A multimode non-sensing resource migration method for an electric power IMS virtualized core network

Abstract

The application discloses a multi-mode non-sensing resource migration method for an electric power IMS virtualized core network, and is used for solving the problems of service interruption, load imbalance and low resource utilization in multi-mode resource migration. The method comprises the following steps: establishing an NFV system model, defining an utility index based on SLA violation, and establishing optimization targets for three modes of virtual machine migration, container migration and application backup respectively; calculating resource utilization and migration deviation in a single mode and a multi-mode fusion environment, and constructing a multi-mode non-sensing migration utility function as a weighted average target; modeling the problem as a Markov decision process model, and converting the migration decision into a strategy; and solving the model by using a deep deterministic policy gradient algorithm to obtain an optimal migration strategy. The application realizes unbiased and non-sensing multi-mode dynamic load balancing, and improves the resource utilization and service reliability of the electric power IMS virtualized core network.

Description

Technical Field

[0001] This invention relates to the field of power communication networks and network function virtualization technology, specifically to a multi-mode seamless resource migration method applied to the virtualization core network of a power IP multimedia subsystem. Background Technology

[0002] With the intelligent development of power communication networks, the core network based on the IP Multimedia Subsystem (IMS) provides integrated voice, data, and multimedia service support for power systems. Network Function Virtualization (NFV) technology, by software-ifying and cloudifying the functions of dedicated network equipment, decouples network functions from the underlying hardware, laying the foundation for building a flexible and scalable power IMS core network. In an NFV environment, Virtualized Network Function (VNF) instances can migrate between different physical nodes based on load, fault, or energy efficiency policies to achieve high availability, load balancing, and resource optimization.

[0003] However, power IMS services have stringent requirements for high reliability and low latency, and traditional resource migration technologies face significant challenges when applied to critical scenarios such as power production control. First, existing migration solutions often focus on a single carrier form, such as optimizing virtual machine (VM) migrations. However, VNFs in the power IMS virtualization core network may coexist in various forms, including virtual machines, containers, and even specific application processes. A single-mode migration strategy struggles to achieve global resource coordination and load balancing across heterogeneous carriers. Second, the migration process itself may introduce service interruptions, performance fluctuations, and additional network and computational overhead, which contradicts the "seamless" (i.e., imperceptible to users and upper-layer services) operational requirements of power services. Existing methods often lack systematic joint modeling and dynamic decision-making capabilities when simultaneously optimizing multiple migration decision objectives (such as resource utilization, SLA violation rate, and migration overhead).

[0004] Specifically, current research on multimodal migration typically treats different carrier forms as independent problems, lacking a unified framework for collaboratively managing virtual machine migration, container migration, and application-level backup / migration strategies. Furthermore, modeling and efficiently solving complex dynamic migration decision-making problems is a major challenge. Traditional optimization methods struggle to adapt to real-time changes in network conditions and service loads, while simple heuristic strategies cannot guarantee optimal long-term returns and perform poorly in meeting multiple stringent constraints (such as SLA, link capacity, and seamless deviation thresholds).

[0005] Therefore, there is an urgent need to propose an innovative resource migration method that can address the scenario of multimodal VNF coexistence in the power IMS virtualization core network. Under the premise of ensuring that the migration process is seamless for the business and the results are unbiased, the method can achieve dynamic load balancing and maximize resource utilization efficiency across modes through intelligent decision-making. Summary of the Invention

[0006] This invention addresses the shortcomings of existing technologies by providing a multi-mode seamless resource migration method for power IMS virtualization core networks. The overall concept of this method is as follows: By establishing a unified NFV system model and a utility index for Service Level Agreement (SLA) violation awareness, it integrates the modeling of three typical modes: virtual machine migration, container migration, and application backup. Then, it quantitatively evaluates resource efficiency and migration deviation under single-mode and multi-mode fusion strategies. Based on this, it constructs an optimization problem with the core objective of maximizing multi-mode weighted average utility and constraints of load, capacity, and seamless deviation. Subsequently, it uses Markov Decision Process (MDP) to model this sequential decision problem and employs Deep Deterministic Policy Gradient (DDPG), a deep reinforcement learning algorithm, to solve it, thereby dynamically outputting the optimal, seamless resource migration strategy.

[0007] To address the aforementioned technical problems, this invention provides a multi-mode seamless resource migration method for a power IMS virtualization core network. Its core lies in the collaborative management of heterogeneous migration modes through a unified modeling, quantification, and intelligent decision-making framework. The method includes the following steps:

[0008] S1. Establish a Network Functions Virtualization (NFV) system model, define a utility index based on Service Level Agreement (SLA) violations, and establish optimization objectives for three single modes: virtual machine migration, container migration, and application backup. The SLA violation model is defined as the proportion of virtualized network functions (VNFs) with SLA violations to the total number of VNFs. The utility index U = 1 - SLA violation value. The optimization objectives for the three single modes are all expressed as maximizing their respective average resource utilization under the constraint that the system load does not exceed a preset threshold.

[0009] More specifically, in the SLA violation model, SLA violations are defined as follows:

[0010]

[0011] In the formula, M0 represents the number of VNFs with SLA violations, and M represents the total number of VNFs;

[0012] The utility index is defined as follows:

[0013]

[0014] Furthermore, the optimization objective for the single-mode virtual machine migration is established as follows:

[0015]

[0016]

[0017] 1≤t≤T;

[0018] In the formula, {V1,V1,…,V n ,…,V N} represents a collection of virtual machines; For VNF sets; For virtual machine V n The m-th VNF node on; This represents the total CPU usage of the node. The amount of memory occupied by the node; This refers to the network bandwidth used by the node. The number of MIPS occupied by the node; The nodes within the running time period t are respectively The amount of CPU, memory, network bandwidth, and MIPS resources used; For nodes The load within the runtime period t; p(t) is a periodic variable, related to the proportion of resources allocated by the system CPU to the corresponding VNF node during the task process; T is the total time period; R VM and L VM These represent the resource utilization and system load of an NFV network with N virtual machines and M VNF nodes, respectively; η VM The load threshold set for the virtual machine migration test environment is independent of the runtime cycle.

[0019] In the single-mode container migration, the weighting coefficients α, β, and γ used to calculate resource utilization satisfy: α + β + γ = 1.

[0020] Furthermore, the optimization objective for the single-mode container migration is established as follows:

[0021]

[0022] α+β+γ=1;

[0023] In the formula, {C1,C1,…,C n ,…,C N} is a collection of containers. For VNF sets, For container C n The m-th VNF node on the network, This represents the total CPU usage of the node. This refers to the bandwidth capacity occupied by the node. R represents the memory capacity occupied by the node, where α, β, and γ are pre-set weight coefficients during the actual migration process; C M represents the average utilization rate of container resources. Cf is the number of VNF nodes in the NFV network used to execute container services; Y L is a binary function used to determine whether a node is used to participate in container services; Y For containerized VNF average load balancing rate; L C η represents the average load balancing rate of container devices. C The load threshold set for container migration in the test environment.

[0024] Furthermore, the optimization objective for the application backup single mode is established as follows:

[0025]

[0026] In the formula, {A1,A1,…,A1} n ,…,A N} represents the application set; For VNF sets; For application A n The m-th VNF node on; This represents the total CPU usage of the node. This refers to the bandwidth capacity occupied by the node. The amount of application memory used by the node; R Z R represents the average resource consumption of applications in application-based VNF backups. A N represents the average resource utilization rate across all applications. A The number of applications used for backup in the NFV network; f A A binary function used to determine whether an application is used in a backup process; η A The load threshold set for application backup in the test environment.

[0027] S2. Calculate the total resource utilization rate of network elements in single-mode and multi-mode converged environments, and calculate the migration deviation between the two; wherein, the migration deviation (E) is calculated as follows: E = |R single -R multiple |,R single R represents the total resource utilization rate in a single-mode environment. multiple This represents the total resource utilization rate in a multi-mode fusion environment.

[0028] More specifically, in step S2, the total resource consumption of the network element node is first calculated:

[0029]

[0030]

[0031] In the formula, CPU total B is the total CPU resource consumed for migration. totalThe total CPU resources consumed for migration, f n This is a binary function used to determine whether a network element in a network link is a VNF migration destination node;

[0032] The formulas for calculating the total resource utilization rate of network element nodes in the single-mode environment and the multi-mode fusion environment are as follows:

[0033]

[0034] In the formula, M VM M C M A These represent the number of VNF nodes in single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively; R VM R C R A These refer to the resource quantities in single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively.

[0035] The formula for calculating the migration bias is:

[0036] E = |R single -R multiple |

[0037] S3. Define the utility functions of the three single modes respectively, and define the utility function of multimode seamless migration as the weighted average of the utility functions of the three single modes, and set the optimization objective as maximizing the utility function of multimode seamless migration; wherein, while maximizing the weighted average utility function, the optimization objective of multimode seamless migration also needs to satisfy the following constraints: the sum of the weighting factors of each single mode is 1; the total CPU and bandwidth resources consumed by migration do not exceed the corresponding capacity threshold of the network link respectively; the migration deviation is less than a preset deviation threshold.

[0038] More specifically, the utility functions for the virtual machine migration single mode, the container migration single mode, and the application backup single mode are as follows:

[0039]

[0040] The optimization objective for the multimodal seamless transfer is established as follows:

[0041]

[0042] In the formula, λ1, λ2, and λ3 are the weighting factors for single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively; CPU chain and B chain These are the CPU resource capacity threshold and bandwidth capacity threshold for NFV network links, respectively; θ>0 is the migration deviation threshold.

[0043] S4. The utility function of the multimode seamless migration is modeled as a Markov Decision Process (MDP) model, and the migration decision of the network element is transformed into the MDP strategy. The state space of the MDP model includes at least the network topology, the CPU resources of the network element nodes, and the link bandwidth resource requirements. The action space is either migrate or not migrate. The reward function is constructed based on the utility function of the multimode seamless migration defined in step S3. The state space of the MDP model specifically includes: the network topology state, the CPU resource state of the network element nodes, and the link bandwidth resource requirement state of the network element nodes.

[0044] More specifically, the MDP model includes a state space {S1, S2, S3}, an action space {A}, and state transition probabilities p(s). t+1 |s t ,a t ,…,s0,a0)=p(s t+1 |s t ,a t and the reward function for time slot t. Wherein, S1, S2, and S3 represent the network topology, network element node CPU resources, and network element node link bandwidth resource requirements, respectively. The state of the VNF node in time slot t is defined as follows:

[0045]

[0046] The set of all actions in the MDP is {A}, and the actions in time slot t are:

[0047] {a t};

[0048]

[0049] The strategy of the MDP is given according to the state of the VNF node in time slot t, and the conditional probability distribution of its action satisfies:

[0050] π(a t |s t )=p(a t |s t );

[0051] The strategy of the MDP is expressed as follows:

[0052]

[0053] Q π (s t ,a t ) = Exp[R t +μ·Q π (s t+1 ,a t+1)|s t ,a t ];

[0054] In the formula, Q π (s t ,a t ) is the action value function, and Exp(·) is the mathematical expectation operation.

[0055] Furthermore, in step S5, the DDPG algorithm adopts the Actor-Critic framework and includes an estimated Actor network, a target Actor network, an estimated Critic network, and a target Critic network.

[0056] The update formula for estimating the Critic network parameters is as follows:

[0057]

[0058] Loss(θ Q )=Exp[y(t)-Q(s t ,a t )|θ Q ] 2 ;

[0059]

[0060] In the formula, θ Q To estimate the parameters of the Critic network; c To estimate the learning rate of the Critic network; Loss(θ) Q ) is the loss function; γ is the discount factor;

[0061] The update formula for estimating the Actor network parameters is as follows:

[0062]

[0063] In the formula, θ π To estimate the parameters of the Actor network; a To estimate the learning rate of the Actor network; d(s t ) represents the state distribution; Let the objective function be the policy objective function; The gradient of the policy objective function;

[0064] The update formulas for the target Critic network parameters and the target Actor network parameters are as follows:

[0065] θ' Q ←τθ Q +(1-τ)θ' Q ;

[0066] θ' π ←τθ π +(1-τ)θ' π ;

[0067] In the formula, θ' Q θ' represents the target Critic network parameters. π τ represents the target Actor network parameters; τ represents the soft update parameters.

[0068] S5. The Deep Deterministic Policy Gradient (DDPG) algorithm is used to solve the MDP model to obtain the optimal transfer policy. The DDPG algorithm employs an Actor-Critic framework, including a dual neural network structure with separate estimation and target networks, and is trained using empirical replay and soft update mechanisms to optimize the policy. Specifically, the DDPG algorithm includes an estimation Actor network, a target Actor network, an estimation Critic network, and a target Critic network; the parameters of the target network are updated using a soft update method.

[0069] Compared with the prior art, the present invention has the following significant advantages:

[0070] It achieves true multi-mode collaboration and unbiased load balancing: by constructing a unified utility function and optimization framework, it incorporates three heterogeneous migration modes—virtual machines, containers, and applications—into the same decision-making system, overcoming the limitations of traditional single-mode strategies. It can achieve fair and efficient load distribution across all modes from a global perspective, avoiding resource silos and unbalanced load phenomena.

[0071] The migration process is made seamless: by introducing and strictly constraining the quantitative indicator of "migration deviation (E)," the abstract requirement of "seamlessness" is transformed into a specific optimization problem constraint, thereby ensuring that the migration strategy has the least impact on the overall resource efficiency of the system and effectively maintaining the continuity and stability of power IMS services.

[0072] It enhances the intelligence and optimality of long-term dynamic decision-making: It uses MDP to model complex dynamic migration problems and uses DDPG, an advanced deep reinforcement learning algorithm, to solve them. This enables the system to adapt to real-time changes in the network and services, and make optimal decisions that not only consider immediate benefits but also long-term cumulative benefits, thereby continuously maximizing resource utilization in dynamic environments.

[0073] The practicality and reliability of the solution are enhanced: the model fully considers multiple practical constraints such as SLA violation, link capacity, and load threshold, and the experience replay and soft update mechanism in the DDPG algorithm also ensures the stable convergence of the learning process. This makes the method proposed in this invention not only theoretically advanced, but also has the potential to be deployed and applied in the actual power IMS virtualization core network environment. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of a single-mode virtual machine migration model provided in an embodiment of the present invention.

[0075] Figure 2 This is a schematic diagram of the NFV network system architecture provided in this embodiment of the invention.

[0076] Figure 3 This is one of the algorithm performance diagrams provided in the embodiments of the present invention (comparison of average resource utilization).

[0077] Figure 4 This is the second schematic diagram of algorithm performance provided in the embodiments of the present invention (system load rate comparison).

[0078] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0079] The invention will now be further described in conjunction with the accompanying drawings and preferred embodiments. For example... Figure 1-4 These are all simplified schematic diagrams, illustrating only the basic structure of the invention in a schematic manner, and therefore only show the components relevant to the invention.

[0080] This invention weighs the feasibility and efficiency of system resource migration in single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively, and requires the design of a reasonable utility index to normalize the performance utility of different modes.

[0081] Quality of Service (QoS) is a network security mechanism used to address network latency and congestion, ensuring the stability and reliability of network communication. QoS is crucial in NFV environments, involving performance guarantees for different types of network traffic during transmission. Its formulation and implementation require consideration of factors such as network functions, network topology, and traffic types. It can be represented by a Service Level Agreement (SLA), using minimum throughput or maximum response time. However, in NFV environments, the representation methods are not entirely consistent for different migration models. This invention defines SLA violations as:

[0082]

[0083] Where M0 represents the number of VNF nodes with SLA violations, and M represents the total number of VNFs in the system. The corresponding SLA violation value represents the proportion of VNFs for which the CPU has not allocated resources when a VNF requests services from the NFV network system. To improve resource allocation efficiency, the SLA value should be minimized; therefore, a resource utilization utility metric is defined as follows:

[0084]

[0085] The goal of virtual machine migration, container migration, and application backup is to allocate VNF resources to maximize the value of U under system load balancing. For each migration single mode, the expression of their respective utility functions also differs because the basic functional performance data they consider are not consistent.

[0086] Example 1

[0087] This example provides a method for establishing a single-mode optimization target for virtual machine migration. For example... Figure 1 As shown, consider an NFV network environment with N virtual machines, where the total number of VNF nodes is M. Define {V1, V2, ..., V...} n ,…,V N} represents a collection of virtual machines. Let be a set of VNFs, where Indicates virtual machine V n The m-th VNF node on the network. For example, due to virtual machine V... N The load exceeds the system's set threshold, and the overloaded node needs to be... From virtual machine V N Migrate to a less-loaded, available virtual machine V2. Similarly, if other less-loaded virtual machines exist in the current system, overloaded nodes can be migrated to other suitable virtual machines. Define the node. The attributes are:

[0088]

[0089] in This represents the total CPU usage of the node. This indicates the amount of memory used by the node. This indicates the network loan used by the node. This indicates the number of MIPS used by the node.

[0090] According to the virtual machine load model, virtual machine load depends on the resource consumption of the tasks processed by the corresponding VNF node. (The last sentence appears to be incomplete and possibly refers to a separate topic: "The node...") The resources used are represented in a standardized form as follows:

[0091]

[0092] in These represent the nodes within the running time period t. The number of CPUs, memory, network bandwidth, and MIPS resources used. Node The load during the operating cycle time t can be expressed as:

[0093]

[0094] Where p(t) represents a periodic variable, related to the proportion of resources allocated by the system CPU to the corresponding VNF node during the task process; T represents the total time period. Therefore, for an NFV network with N virtual machines and M VNF nodes, its resource utilization and system load can be expressed as follows:

[0095]

[0096] Accordingly, the objective function for optimizing the single-mode utility of virtualized VNF migration is set as follows:

[0097] max R VM

[0098] stL VM ≤η VM ;

[0099] 1≤t≤T;

[0100] Where η VM This indicates the load threshold set for the virtual machine migration test environment, and is independent of the runtime cycle.

[0101] Example 2

[0102] This example provides a method for establishing a single-mode optimization goal for container migration. N network devices deployed at the container layer are evenly distributed in an NVF network, containing a total of M VNF nodes. Define {C1, C2, ..., C...} n ,…,C N} is a collection of containers. Let be a VNF set, where Represents container C n The m-th VNF node in the container. When the total resources of the VNF nodes in a container exceed the current resource capacity of the container, the overloaded nodes need to be migrated to other available nodes. Each container can only handle one service request.

[0103] The main utility attributes of a VNF node for container resources are CPU capacity, storage capacity, and memory capacity. Since uneven distribution of container environment resources on nodes can lead to unreliable performance of VNFs with high instantaneous power consumption, the average resource utilization of nodes can be considered as a performance metric for network load balancing. Define nodes. The attributes are:

[0104]

[0105] in This represents the total CPU usage of the node. This indicates the bandwidth capacity used by the node. This indicates the memory capacity occupied by the node.

[0106] The NFV container migration single mode, under the condition of ensuring load balancing of each container, dynamically migrates VNF nodes that are carried by containers based on the corresponding container services, thereby improving the utilization of their CPU quantity, bandwidth and memory resources, so as to optimize the overall system performance and make the final QoS meet the SLA standard.

[0107] According to the load balancing model, the average node utilization in the containerized VNF migration model is determined by the resource consumption of all nodes participating in container services on that container, expressed as:

[0108]

[0109] α+β+γ=1;

[0110] Where α, β, and γ represent pre-set weighting coefficients during the actual migration process. The average utilization rate of container resources can then be expressed as:

[0111]

[0112] Where M C f represents the number of VNF nodes used to execute container services in the NFV network. Y This is a binary function used to determine whether a node is participating in container services. The average load balancing rate of a containerized VNF can be calculated as follows:

[0113]

[0114] Accordingly, the average load balancing rate of the container equipment is:

[0115]

[0116] Based on the above expression, the objective function for optimizing the single-mode utility of virtualized VNF migration is defined as follows:

[0117] max R C

[0118] stL C ≤η C ;

[0119] Where η C This indicates the load threshold set for container migration in the test environment.

[0120] Example 3

[0121] This embodiment provides a method for establishing a single-mode optimization target for application backup. Consider an NFV environment that backs up N applications, where each application is composed of and implemented by one or more VNFs, and the total number of corresponding network element nodes is M. Define {A1, A2, ..., A...} n ,…,A N} represents the application set. Let be a VNF set, where Indicates application A n The m-th VNF node. Since application migration relies on virtual machines or containers, the primary utility attribute of application-specific VNF nodes for application data resources should consider the common attributes of both virtualized and containerized VNFs, i.e., node... The attributes are:

[0122]

[0123] in This represents the total CPU usage of the node. This indicates the bandwidth capacity used by the node. This indicates the amount of application memory used by the node.

[0124] In NFV application backup single-mode, while ensuring load balancing, data of network element nodes is backed up at the virtualization layer as applications, and then the entire application is dynamically migrated using virtual machines or containers. Application-based VNF backup does not necessarily back up all applications; therefore, average resource utilization is used to measure network load. The average resource consumption of applications in application-based VNF backup can be expressed as:

[0125]

[0126] The average resource utilization rate for all applications is:

[0127]

[0128] Where R A f represents the number of applications used for backup in the NFV network. A This is a binary function used to determine whether an application is being used for backup. The average load factor for NFV applications can be calculated as follows:

[0129]

[0130] Based on the above expression, the objective function for optimizing the single-mode utility of applied VNF backup is defined as follows:

[0131] max R A

[0132] stL A ≤ηA ;

[0133] Where η A This indicates the load threshold set for application backup in the test environment.

[0134] Example 4

[0135] This embodiment provides a method for establishing multimodal seamless migration optimization objectives. Considering, for example... Figure 2 The NFV network environment system model shown is illustrated. Define {P1, P2, ..., P...} n ,…,P N} represents the set of network element nodes. Let M be a VNF set, where M VM M C M A These represent the number of VNF nodes in single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively; R VM R C R A These refer to the resource quantities in single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively. This indicates the m-th VNF that originally belonged to a single-pattern F, and N≥M. VM +M C +M A Each network element node stores at most one migrated VNF. For a migrated VNF node, its main utility attributes for data resources should encompass all attributes contained in virtualized VNFs, containerized VNFs, and application-based VNFs. To facilitate problem handling, resource constraints are simplified here based on SFC availability and combined with SDN centralized control, i.e., node P... n The attributes are primarily based on link CPU capacity and bandwidth capacity, and are expressed as follows:

[0136]

[0137] in This indicates that the VNF has been migrated to the network element P. n CPU resources consumed This indicates that the VNF has been migrated to the network element P. n The bandwidth resources consumed. Therefore, the total resource consumption of the network element node is...

[0138]

[0139]

[0140] Where R CPU R represents the total CPU resources consumed during migration. B f represents the total CPU resources consumed by the migration.n To determine network element P in a network link n Is it a binary function for the destination node of a VNF migration?

[0141] To verify the seamlessness of resource migration later, the utility metric here should correspond to the single-mode approach, i.e., resource utilization. Define CPU. chain and B chain Let the CPU capacity and bandwidth capacity of the NFV network link be respectively. Then, the total resource utilization of the network element node in the single-mode environment and the multi-mode converged environment can be expressed as follows:

[0142]

[0143] Therefore, resource migration deviation can be defined as:

[0144] E = |R single -R multiple |;

[0145] The above formula shows that the smaller the value of E, the more significant the migration nonsensicality, and the smaller the corresponding SLA violation, which means that the service quality in the NVF environment is negatively correlated with the multimodal nonsensical migration bias.

[0146] The goal of establishing a multi-mode seamless resource migration model based on NFV is to maximize the resource utilization of network element nodes after migration while satisfying network link load and migration seamlessness. The utility functions for single-mode virtual machine migration, single-mode container migration, and single-mode application backup are expressed as follows:

[0147]

[0148] The multimodal seamless transfer utility function is a weighted average of the three single-modal utility functions, and can be modeled as follows:

[0149] max U=λ1U1+λ2U2+λ3U3

[0150] stλ1+λ2+λ3=1

[0151] 0 <CPU total ≤CPU chain

[0152] 0 total ≤B chain

[0153] E<θ;

[0154] Where λ1, λ2, and λ3 are the weighting factors for single-mode virtual machine migration, single-mode container migration, and single-mode application backup, respectively; CPU chain and B chain ​These are the CPU resource capacity threshold and bandwidth capacity threshold for NFV network links, respectively; θ>0 is the migration deviation threshold.

[0155] Example 5

[0156] This embodiment provides a method for establishing the utility function of multimodal seamless transfer as an MDP model.

[0157] In an NFV environment, considering three state spaces—network topology, network element CPU resources, and network element link bandwidth resource requirements—denoted as S1, S2, and S3 respectively, the set of all state spaces for a Markov Decision Process (MDP) can be represented as {S1, S2, S3}. Furthermore, the state of a VNF node in time slot t is correspondingly defined as:

[0158]

[0159] The actions taken by the VNF, whether to migrate or not, can be summarized into an action space, denoted by {A}. Furthermore, the action in time slot t is defined as:

[0160] {a t};

[0161]

[0162] MDP exhibits Markov properties, and the state transition probabilities satisfy:

[0163] p(s t+1 |s t ,a t ,…,s0,a0)=p(s t+1 |s t ,a t );

[0164] The reward function for time slot t can be expressed as:

[0165] R t =R(s) t ,a t ,s t+1 )=λ1U 1t +λ2U 2t +λ3U 3t ;

[0166] The policy of MDP is given by state, that is, the conditional probability distribution of actions, which satisfies:

[0167] π(a t |s t )=p(a t |s t );

[0168] Therefore, the expression for the action value function is:

[0169] Q π (s t ,a t ) = Exp[R t +μ·Q π (s t+1 ,a t+1 )|s t ,a t ];

[0170] The corresponding optimal VNF migration strategy is expressed as:

[0171]

[0172] Example 6

[0173] This embodiment provides a method for solving an MDP model using the DDPG algorithm to obtain the optimal MDP policy. The MDP model in Embodiment 5 above is a continuous action space problem, and the Deep Deterministic Policy Gradient (DDPG) algorithm from deep reinforcement learning is considered for its design. The algorithm uses an Actor-Critic framework, and the weight parameters are generated through a dual neural network architecture. The symbols and corresponding deep neural networks are shown in Table 1.

[0174] Table 1

[0175]

[0176] In estimating the Critic network, using approximation relationships, we have:

[0177]

[0178] After state and action normalization, the input state s is normalized. t Preprocessing of training samples for VNF-mapped actions accelerates model solving. Here, VNF action a... t The associated Q-value is set to the output of the deep neural network. Introducing a discount factor γ, the loss function becomes:

[0179] Loss(θ Q )=Exp[y(t)-Q(s t ,a t )|θ Q ] 2 ;

[0180]

[0181] The estimated Critic network parameters can then be updated as follows:

[0182]

[0183] Among them l c This represents the estimated learning rate of the Critic network. When constructing the Actor network, a deep neural network is also chosen as its basic structure. For the state distribution d(s) t The policy objective function is:

[0184]

[0185] The gradient of the objective function of the corresponding policy is:

[0186]

[0187] When the approximation relationship is satisfied, the above gradient can be further expressed as:

[0188]

[0189] At the same time, the estimated Actor network parameters are updated, namely:

[0190]

[0191] Among them l a To estimate the learning rate of the Actor network, and to further investigate and identify the optimal action for policy selection during transfer learning, a random perturbation *n* is incorporated into the deep deterministic policy gradient to expand the search space. This action is...

[0192] a t =π(s) t |θ π )+n;

[0193] Where n represents Gaussian random noise with mean n0. Introducing a soft update factor τ, the corresponding target Critic network parameters and target Actor network parameters can be updated as follows:

[0194] θ' Q ←τθ Q +(1-τ)θ' Q ;

[0195] θ' π ←τθ π +(1-τ)θ' π ;

[0196] Table 2 describes the DDPG algorithm:

[0197] Table 2

[0198]

[0199] Test case

[0200] This test case is based on Example 6 above and provides a computational experiment to verify the algorithm's performance. In the simulation parameters, the preset weight coefficients α = β = γ are used during container migration, the application backup process is performed at any NFV network layer, and the preset threshold for SFC link resource capacity is greater than the maximum demand of each single mode. The remaining parameter settings are shown in Table 3.

[0201] Table 3

[0202]

[0203]

[0204] In Table 3, Uniform[a,b] represents a random number generated between a and b (inclusive).

[0205] The seamlessness of deep deterministic policy gradient algorithms for VNF transfers can be measured by the total system latency. In the optimization objective of the multimodal seamless transfer model, the set of weight factors is set as {λ1,λ2,λ3}, and two sets of single-mode corresponding weights are simulated as {0.7,0.2,0.1} and {0.5,0.3,0.2}, respectively. Figure 3 and Figure 4 The figures show the average resource utilization and load rate of the server over one operating cycle when two sets of weight values ​​are obtained. The above graphs demonstrate that, in addition to achieving load balancing within a finite threshold, when the weight allocation tends towards equilibrium, system latency decreases, and the corresponding average resource utilization also increases. Migration deviations are controlled, further illustrating that balanced weights are beneficial for reducing SLA violations during migration and for effectively controlling link resources within the load threshold, thereby improving the overall quality of NFV network services.

[0206] The above embodiments are only for illustrating the technical concept and features of the present invention. Their purpose is to enable those skilled in the art to understand the content of the present invention and implement it. They should not be used to limit the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A multi-mode resource migration method for power IMS virtualized core network, characterized in that, Includes the following steps: S1. Establish a Network Functions Virtualization (NFV) system model, define utility metrics based on Service Level Agreement (SLA) violations, and establish optimization objectives for three single modes: virtual machine migration, container migration, and application backup. S2. Calculate the total resource utilization of network elements in single-mode and multi-mode fusion environments, and calculate the migration deviation between the two. S3. Define the utility functions of the three single modes respectively, define the utility function of multimode seamless transfer as the weighted average of the utility functions of the three single modes, and set the optimization objective as maximizing the utility function of the multimode seamless transfer. S4. The utility function of the multimodal seamless migration is modeled as a Markov Decision Process (MDP) model, and the migration decision of the network element is transformed into the strategy of the MDP. S5. The Deep Deterministic Strategy Gradient (DDPG) algorithm is used to solve the MDP model to obtain the optimal migration strategy.

2. The method of claim 1, wherein, In step S1, the SLA violation model is defined as the proportion of the number of virtualized network functions (VNFs) with SLA violations to the total number of VNFs, and the utility index U = 1 - SLA violation value.

3. The method according to claim 1, characterized in that, In step S1, the optimization objective of the three single modes is to maximize their respective average resource utilization rate under the constraint that the system load does not exceed a preset threshold.

4. The method according to claim 1, characterized in that, In step S3, the optimization objective of the multimode seamless migration, while maximizing the weighted average utility function, also needs to satisfy the following constraints: the sum of the weighting factors of each single mode is 1; the total CPU and bandwidth resources consumed by the migration do not exceed the corresponding capacity threshold of the network link; and the migration deviation is less than a preset deviation threshold.

5. The method according to claim 1, characterized in that, In step S4, the state space of the MDP model includes at least the network topology, CPU resources of network element nodes, and link bandwidth resource requirements; the action space is migration or no migration; the reward function is constructed based on the utility function of the multi-mode seamless migration defined in step S3.

6. The method according to claim 1, characterized in that, In step S5, the DDPG algorithm adopts the Actor-Critic framework, which includes a dual neural network structure with separate estimation and target networks, and is trained through experience replay and soft update mechanisms to optimize the strategy.

7. The method according to claim 1, characterized in that, The step S2, the migration deviation (E) is calculated in the following way: E = |R single -R multiple |; wherein R single is the total resource utilization in a single mode environment, and R multiple is the total resource utilization in a multi-mode fusion environment.

8. The method according to claim 1, characterized in that, In step S4, the state space of the MDP model specifically includes: network topology state S1, network element node CPU resource state S2, and network element node link bandwidth resource requirement state S3.

9. The method according to claim 6, characterized in that, The DDPG algorithm specifically includes an estimated Actor network, a target Actor network, an estimated Critic network, and a target Critic network; the parameters of the target network are updated using a soft update method.

10. The method according to claim 1, characterized in that, In the single-mode container migration, the weighting coefficients α, β, and γ used to calculate resource utilization satisfy: α + β + γ = 1.

Images