Method, apparatus and terminal for dynamic network service reconfiguration

Abstract

The application provides a dynamic network service reconfiguration method, device and terminal. The method comprises the following steps: detecting whether a network service reconfiguration algorithm is triggered; when the network service reconfiguration algorithm is triggered, performing priority sorting on the network service, determining an execution order of the reconfiguration algorithm; according to the execution order, establishing a network service reconfiguration problem model distinguishing between service types, and determining a new configuration scheme of each network service based on a deep reinforcement learning service reconfiguration algorithm; and according to the new configuration scheme of the network service, determining a migration sequence from a current configuration to a new configuration, and reconfiguring all network services. The application can reduce resource consumption, quickly reconfigure network services, and reconfigure services of different service types to meet the needs of different services.

Description

Technical Field

[0001] This invention relates to the field of network configuration technology, and in particular to a dynamic network service reconfiguration method, apparatus and terminal. Background Technology

[0002] SDN is an emerging network paradigm that decouples the network logic control capabilities of the control plane from the data plane. SDN decouples the control and data planes, enabling a global view and facilitating centralized network management. The data plane can be deployed as needed, improving user service quality. Because the control plane can centrally control the physical resources of the entire network, it can perform network-wide traffic control, channel resource allocation, routing management, and load balancing while meeting diverse service requirements. In large-scale networking scenarios, the cost and resource utilization pressure of independently constructing communication networks are significant. Only the technical performance and operational methods provided by SDN can guarantee that the power communication network, while meeting its own service needs, also possesses the capability for service expansion.

[0003] In the context of SDN, the introduction of Network Function Virtualization (NFV) technology enables the construction of isolated dedicated logical networks on general end-to-end networks, providing users with a variety of customized services. Therefore, virtual network functions (VNFs) can be deployed and instantiated at specific locations based on user service needs, achieving rational allocation of service resources, reducing waste of idle resources, and eliminating the need to install new hardware. SDN and NFV-based network service provision separates network functions, traditionally implemented using specific devices, from the actual physical devices, achieving rational allocation of service resources and improving system flexibility. However, dynamic resource allocation remains one of the major challenges that network services still face.

[0004] Existing network service reconfiguration methods propose a make-before-break mechanism to avoid QoS degradation from the perspective of ensuring user service quality during reconfiguration. However, this mechanism requires retaining the new route and installing new virtual resources before reconfiguring the service, thus increasing resource consumption. Summary of the Invention

[0005] This invention provides a method, apparatus, and terminal for dynamic network service reconfiguration to solve the problem in the prior art where retaining new routes during network service reconfiguration leads to increased resource consumption.

[0006] In a first aspect, embodiments of the present invention provide a dynamic network service reconfiguration method, comprising:

[0007] Detect whether the network service reconfiguration algorithm has been triggered;

[0008] When the network service reconfiguration algorithm is triggered, the network services are prioritized to determine the execution order of the reconfiguration algorithm.

[0009] According to the execution order, a network service reconfiguration problem model that distinguishes business types is established, and a new configuration scheme for each network service is determined based on a service reconfiguration algorithm based on deep reinforcement learning.

[0010] Based on the new configuration scheme of the network services, determine the migration sequence from the current configuration to the new configuration, and reconfigure all network services.

[0011] In one possible implementation, methods for triggering the network service reconfiguration algorithm include:

[0012] Obtain the remaining computing resources and remaining link bandwidth resources of each physical node in the infrastructure network;

[0013] Based on the remaining computing resources of each physical node, check whether the variance of the resource utilization rate of the remaining computing resources meets the first condition.

[0014] Based on the remaining link bandwidth resources of each physical node, check whether the variance of the resource utilization of the remaining link bandwidth resources meets the second condition.

[0015] When the variance of the resource utilization of the remaining computing resources meets the first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets the second condition, the network service reconfiguration algorithm is determined to be triggered.

[0016] In one possible implementation, when the variance of the resource utilization of the remaining computing resources meets a first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets a second condition, a network service reconfiguration algorithm is determined to be triggered, including:

[0017] when or At that time, determine the network service reconfiguration algorithm to be triggered;

[0018] Where, N s Represents a set of physical nodes. This represents the i-th physical node. γ represents the remaining computing resources of the i-th physical node. c L represents the variance threshold for the utilization rate of computational resources. s Represents the set of physical links between physical nodes. Represents physical nodes With physical nodes The physical link between them γ represents the remaining link bandwidth resources of the physical link. B The variance threshold representing the utilization rate of bandwidth resources.

[0019] In one possible implementation, prioritizing the network services to determine the execution order of the reconfiguration algorithm includes:

[0020] Calculate the ratio of the remaining lifetime of each network service to its total lifetime;

[0021] The execution order of the reconfiguration algorithm is determined by prioritizing all network services based on their ratios, with smaller ratios indicating higher priority.

[0022] In one possible implementation, following the execution order, a network service reconfiguration problem model distinguishing between service types is established, and a new configuration scheme for each network service is determined based on a deep reinforcement learning-based service reconfiguration algorithm, including:

[0023] If the current network service shares a VNF, then identify the first physical node in the infrastructure network that has the same VNF type as the current network service and shares the VNF, and deploy the VNF there.

[0024] If the current network service does not share a VNF, then the physical node with the lowest load is determined to deploy the VNF based on the service type of the current network service.

[0025] If a VNF is deployed for the current network service, then the state corresponding to the current network service is input into the DQN network to obtain the Q(s) of all actions under that state. t The values ​​of a and θ are used to input the obtained actions into the environment to obtain the new state corresponding to the current network service; the feedback value corresponding to the objective function of the current network service is determined according to the service type of the current network service.

[0026] For the remaining VNFs of the current network service, an action is randomly selected with probability C. If an action cannot be randomly selected using probability C, then the action is selected based on the maximum Q(s). t The values ​​of a and θ are used to determine the action, and the obtained action is input into the environment to obtain a new state; the feedback values ​​after the remaining VNF deployment are calculated;

[0027] Save the quadruple (s,A,R,s') obtained from the above steps as a record; where s represents the state of all network services, A represents the action of all network services, R represents the feedback value after all VNFs are deployed, and s' represents the new state of all network services.

[0028] If the current iteration count has not reached the preset iteration count, then the remaining VNFs of the current network service are selected for action and feedback values ​​are calculated. If all VNFs of the current network service have been selected for action and feedback values ​​have been calculated, then the process jumps to the step of "checking whether the current network service that has not been reconfigured shares a VNF" to process the next network service accordingly.

[0029] If the current iteration count reaches the preset iteration count, a preset number of records are randomly selected from all saved records as training values ​​to train the neural network and obtain the optimized parameters θ.

[0030] Update Q(s) according to the preset strategy and the optimization parameter θ. t The Q value (a; θ) is calculated iteratively based on the updated Q value until the maximum number of iterations is reached.

[0031] In one possible implementation, determining the feedback value corresponding to the objective function of the current network service based on the service type of the current network service includes:

[0032] If the current network service's service type is a uRLLC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0033] If the current network service's service type is an mMTC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0034] If the current network service's service type is an eMBB type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0035] Among them, R uRLLC R represents the feedback value of a network service of type uRLLC. mMTC R represents the feedback value of mMTC type network services. eMBB This represents the feedback value of an eMBB type network service, Z represents the parameter that adjusts the feedback value to a positive number, α, β, γ, and δ represent coefficients, and D represents the feedback value. tot L represents the propagation delay between the current physical node and the previous deployment location for a uRLLC type network service. tot B represents the node load balancing of mMTC type network services. tot C represents the link bandwidth consumption of eMBB type network services. totThis represents the total system overhead of migrating the VNF to the action selection node location, where s represents the set of network servers and k represents the number of edge data centers. k Indicates network services, Indicates physical link Does it belong to network service? k The actual physical forwarding path of the j-th virtual link The value, N represents link delay. s Represents the set of physical nodes on a data center. Represents physical nodes The remaining computing resources Indicates physical link Link bandwidth resources, Indicates physical link The remaining link bandwidth resources; α' and β' represent the overhead coefficients, respectively. Represents the network server of VNF From physical nodes Migrate to Energy consumption expenditure, Represents physical nodes arrive The actual migration path Indicates physical link Does it belong to a physical node? arrive The actual migration path value, Indicates from physical node Migrate to migration overhead, B(s) k This represents the resource consumption on the physical link. Indicates from physical node Migrate to The number of jumps.

[0036] In one possible implementation, determining the migration sequence from the current configuration to the new configuration based on the new configuration scheme of the network service includes:

[0037] Initialize the open and close tables;

[0038] For network services that have not been migrated, set the original configuration of the network service as the starting point and the new configuration as the target point, and add the starting point to the open table;

[0039] Take the starting point with the minimum cost function in the open table as the current node, and check whether the current node contains a target point;

[0040] If the current node is not the target point, move the current node into the close table;

[0041] If the current node is the target node, following the idea of ​​the A* algorithm, start from the current node and backtrack to the starting point through the parent pointer of each node to obtain the migration sequence and output it;

[0042] For unexplored neighboring nodes of the current node, if the neighboring node is a cached infeasible configuration node, then the neighboring node is discarded; if the configuration of the neighboring node is infeasible, then the neighboring node is discarded.

[0043] Add the remaining neighbor nodes to the open table, calculate the cost function of the remaining neighbor nodes, and then proceed to the step of "taking the point with the smallest cost function in the open table as the current node".

[0044] Secondly, embodiments of the present invention provide a dynamic network service reconfiguration apparatus, comprising:

[0045] The detection module is used to detect whether the network service reconfiguration algorithm has been triggered.

[0046] The sorting module is used to prioritize the network services and determine the execution order of the reconfiguration algorithm when the network service reconfiguration algorithm is triggered.

[0047] The reconfiguration module is used to establish a network service reconfiguration problem model that distinguishes business types according to the execution order, and to determine a new configuration scheme for each network service based on a service reconfiguration algorithm based on deep reinforcement learning.

[0048] The migration module is used to determine the migration sequence from the current configuration to the new configuration based on the new configuration scheme of the network service, and to reconfigure all network services.

[0049] Thirdly, embodiments of the present invention provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the dynamic network service reconfiguration method as described in the first aspect or any possible implementation thereof.

[0050] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the dynamic network service reconfiguration method as described in the first aspect or any possible implementation thereof.

[0051] This invention provides a dynamic network service reconfiguration method, apparatus, and terminal. When a network service reconfiguration algorithm is triggered, network services are prioritized to determine the execution order of the algorithm. Based on this order, a network service reconfiguration problem model is established, distinguishing between service types. A new configuration scheme for each network service is determined using a deep reinforcement learning-based service reconfiguration algorithm. According to the new configuration scheme, a migration sequence from the current configuration to the new configuration is determined, and all network services are reconfigured. This allows for software-based network service reconfiguration, eliminating the need to retain new routes and thus reducing resource consumption. It enables rapid network service reconfiguration and allows for the reconfiguration of different service types to meet diverse service needs. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating the implementation of the dynamic network service reconfiguration method provided in this embodiment of the invention.

[0054] Figure 2 This is a schematic diagram of the structure of the dynamic network service reconfiguration device provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0056] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0057] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0058] Figure 1 A flowchart illustrating the implementation of a dynamic network service reconfiguration method according to an embodiment of the present invention is described in detail below:

[0059] Step 101: Check whether the network service reconfiguration algorithm has been triggered.

[0060] In one embodiment, the dynamic network service reconfiguration method is applied in a power grid infrastructure network. The infrastructure system architecture is as follows: the infrastructure network is divided into K edge data centers and M core data centers, and each data center can use a weighted undirected graph. This means that 1 ≤ k ≤ K + M. Where N s Represents a set of physical nodes, where a physical node i is represented by... It means, L s Represents the set of physical links between physical nodes, where physical nodes and Links between Indicate. Use Represents the set of physical nodes on an edge data center. Let represent the set of physical nodes on the core data center. For each physical node i, assume the available computing resources are . Remaining computing resources are The bandwidth resources for each link are The remaining link bandwidth resources are Link latency is

[0061] To find a new configuration scheme, the first step is to set up a mechanism to trigger reconfiguration. In this embodiment, the performance parameters of the problem model mainly considered include the computing resources of the nodes and the bandwidth resources of the links. A multi-threshold triggering method is used to determine the time when reconfiguration is required.

[0062] In one embodiment, the remaining computing resources and remaining link bandwidth resources of each physical node in the infrastructure network are obtained;

[0063] Based on the remaining computing resources of each physical node, check whether the variance of the resource utilization rate of the remaining computing resources meets the first condition.

[0064] Based on the remaining link bandwidth resources of each physical node, check whether the variance of the resource utilization of the remaining link bandwidth resources meets the second condition.

[0065] When the variance of the resource utilization of the remaining computing resources meets the first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets the second condition, the network service reconfiguration algorithm is determined to be triggered.

[0066] In one embodiment, when the variance of the resource utilization of the remaining computing resources meets a first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets a second condition, a network service reconfiguration algorithm is determined to be triggered, including:

[0067] when or When the network service reconfiguration algorithm is triggered, it is determined that the reconfiguration will meet the threshold requirements of the equation for the utilization of computing resources and bandwidth resources, and the migration system overhead of the reconfiguration should be minimized as much as possible.

[0068] In the above formula, N s Represents a set of physical nodes. This represents the i-th physical node. γ represents the remaining computing resources of the i-th physical node. c L represents the variance threshold for the utilization rate of computational resources. s Represents the set of physical links between physical nodes. Represents physical nodes With physical nodes The physical link between them γ represents the remaining link bandwidth resources of the physical link. B The variance threshold representing the utilization rate of bandwidth resources.

[0069] It should be noted that if the variance of the resource utilization of the remaining computing resources does not meet the first condition, and the variance of the resource utilization of the remaining link bandwidth resources does not meet the second condition, the network service reconfiguration algorithm will not be triggered.

[0070] Step 102: When the network service reconfiguration algorithm is triggered, prioritize the network services to determine the execution order of the reconfiguration algorithm.

[0071] In one embodiment, prioritizing network services to determine the execution order of the reconfiguration algorithm may include:

[0072] Calculate the ratio of the remaining lifetime of each network service to its total lifetime;

[0073] The execution order of the reconfiguration algorithm is determined by prioritizing all network services based on their ratios, with smaller ratios indicating higher priority.

[0074] Here, services can be sorted in descending order of priority. The purpose of prioritization is to first reconfigure network services with shorter remaining lifecycles, thereby achieving a reasonable allocation of service resources and improving the system's flexibility.

[0075] Step 103: In accordance with the execution order, establish a network service reconfiguration problem model that distinguishes between business types, and determine the new configuration scheme for each network service based on a service reconfiguration algorithm using deep reinforcement learning.

[0076] Before reconfiguring a network server, it is necessary to clarify some concepts.

[0077] Network services are logically isolated networks on the same infrastructure network, therefore each customized network service... k It represents the set of network functions required for network services. Where M represents the number of required functions. Each network service has a set of functions defined by T(s). k The end-to-end delay threshold is represented by ).

[0078] The reconfiguration result is from and Indicates. If s k The m-th VNF is mapped to the physical node. superior, The value is 1 if it is set to 1, and 0 otherwise. If the physical link... Belongs to service s k The actual physical forwarding path of the j-th virtual link. The value is 1 if it is not 0 otherwise, where service s k The actual physical forwarding path of the j-th virtual link is stored in In the middle. If the physical link Belongs to physical nodes arrive The actual migration path The value is 1 if it is not 0 otherwise, where the physical node arrive The actual migration path is stored in superior.

[0079] Security isolation of network services can be divided into inter-service isolation and intra-service isolation. Based on the security requirements of the applications supporting the services, such as the degree of resource contention between services and information protection needs, inter-service isolation can be further subdivided into physical isolation and logical isolation. Physical isolation involves allocating independent physical resources to network services. Logical isolation is implemented using NFV-based resource isolation technology, using γ(s... k ) indicates service s k Are there any requirements for physical isolation between services? If so... k If physical isolation between services is required, then γ(s) k The value is 1. Different network functions need to provide mutual isolation between them based on their own security requirements and trust relationships. Considering flexibility and economic costs, the intra-chip isolation level of a service can be represented by the ratio of the total number of physical nodes deploying the service to the total number of network functions required by the service. Therefore, K is used. rank This indicates the level of isolation within the service; the higher the value, the higher the isolation requirements within the service.

[0080] The reconfiguration cost defined in this embodiment includes both energy consumption and migration costs. The energy consumption of service reconfiguration increases with the amount of data transmitted. Therefore, the energy consumption of service reconfiguration can be equated to the time required to migrate the running state and data using the physical link. Therefore, VNF From physical nodes Migrate to Energy consumption expenditure is defined as

[0081]

[0082] Migration overhead is related to the number of hops between the original deployment node and the target node; therefore, service migration is defined for the underlying nodes. and The resulting migration overhead is

[0083]

[0084] Including energy consumption costs and relocation costs, Migrate to The total system overhead is

[0085]

[0086] Therefore, the system overhead of migrating the entire network service is

[0087]

[0088] This embodiment aims to comprehensively optimize the different requirements of three typical services while reducing the cost of migration during reconfiguration. Different services have the aforementioned optimization needs, but with different focuses. uRLLC services focus on minimizing service latency, mMTC services focus on achieving load balancing of physical nodes, and eMBB services focus on minimizing link bandwidth consumption.

[0089] Therefore, the objective function for uRLLC type services consists of service latency and the cost of reconfiguration:

[0090] minα*D tot +δ*C tot ;

[0091]

[0092] For mMTC type services, the objective function consists of the cost of node load balancing and reconfiguration:

[0093] minβ*L tot +δ*C tot ;

[0094]

[0095] For eMBB type services, the objective function consists of link bandwidth consumption and reconfiguration cost:

[0096] minγ*B tot +δ*C tot ;

[0097]

[0098] Where α, β, γ, and δ represent coefficients, including weighting factors used to adjust the proportion of the optimization target and data normalization.

[0099] In addition, regarding the reconfiguration issue in this solution, the following concepts are defined:

[0100] State set S: Service reconfiguration needs to consider the resource utilization of the physical network; therefore, we calculate the remaining resource percentage of physical nodes and links in the physical network. Furthermore, considering customized services, the deployment scheme also needs to take security into account as an important factor. In addition, reconfiguration also needs to consider the location of the original configuration; therefore, the state set of service orchestration can be represented as a (M+N+3) dimensional feature:

[0101]

[0102] The first M elements {w1, w2, ... w M} represents the computing resource usage of physical nodes. The next N elements {v1,…v N The third to last element represents the bandwidth usage of the link. Indicates the inter-fragment isolation level, the second to last element K rank The first element indicates the isolation level within the chip, and the last element indicates the original deployment location.

[0103] Action set A: For each state, given the VNF to be deployed and the path between the VNF deployment location and the previous location, a physical node and physical path are selected based on the Q-value function to deploy the VNF and the physical link. Therefore, the action set can be represented as: A = (n, p), where n represents the selected physical node and p represents the selected physical path.

[0104] Feedback Function: Performing different actions A under different states will yield different feedback from the environment, but the chosen action may violate the constraints in the actual problem. Therefore, when the constraints are met, the feedback functions are defined using the optimization objective functions of the three services as shown below; when the chosen action violates the constraints, the feedback value is set to -1.

[0105] R uRLLC=Z-(α*D+δ*C);

[0106]

[0107] R eMBB =Z-(γ*B+δ*C);

[0108] Z is used to adjust the feedback value to a positive number; D represents the system load balance at a certain moment, i.e., the variance of the computing resource usage of all physical nodes; B represents the propagation delay between the selected node and the previous deployment location; C represents the physical link resource consumption; and D represents the total system overhead of migrating the VNF to the action selection node location.

[0109] After defining the relevant concepts, a network service reconfiguration method based on the DQN algorithm in reinforcement learning can be designed. Following the execution order, a network service reconfiguration problem model is established, distinguishing between service types. Then, based on a deep reinforcement learning-based service reconfiguration algorithm, a new configuration scheme for each network service is determined, which may include:

[0110] In the execution sequence, check whether the current network service that has not been reconfigured shares the VNF;

[0111] If the current network service shares a VNF, then identify the first physical node in the infrastructure network that has the same VNF type as the current network service and shares the VNF, and deploy the VNF there.

[0112] If the current network service does not share a VNF, then the physical node with the lowest load is determined to deploy the VNF based on the service type of the current network service.

[0113] If a VNF is deployed for the current network service, then the state corresponding to the current network service is input into the DQN network to obtain the Q(s) of all actions under that state. t The values ​​of a and θ are used to input the obtained actions into the environment to obtain the new state corresponding to the current network service; the feedback value corresponding to the objective function of the current network service is determined according to the service type of the current network service.

[0114] For the remaining VNFs of the current network service, an action is randomly selected with probability C. If an action cannot be randomly selected using probability C, then the action is selected based on the maximum Q(s). t The values ​​of a and θ are used to determine the action, and the obtained action is input into the environment to obtain a new state; the feedback values ​​after the remaining VNF deployment are calculated;

[0115] Save the quadruple (s,A,R,s') obtained from the above steps as a record; where s represents the state of all network services, A represents the action of all network services, R represents the feedback value after all VNFs are deployed, and s' represents the new state of all network services.

[0116] The system checks whether the current iteration count has reached the preset iteration count. If the current iteration count has not reached the preset iteration count, it selects actions and calculates feedback values ​​for the remaining VNFs of the current network service. After all VNFs of the current network service have been selected and their feedback values ​​calculated, it jumps to the "Check whether the current network service that has not been reconfigured shares a VNF" step to process the next network service accordingly. In other words, actions are selected and feedback values ​​are calculated for the VNFs of a network service in sequence, and then the same process is performed for the next network service until all network services have been processed.

[0117] If the current iteration count reaches the preset iteration count, a preset number of records are randomly selected from all saved records as training values ​​to train the neural network and obtain the optimized parameters θ.

[0118] Update Q(s) according to the preset strategy and the optimization parameter θ. t The Q value (a; θ) is calculated iteratively based on the updated Q value until the maximum number of iterations is reached.

[0119] After the above actions are completed, for each network service and for each virtual network function on it, the local migration algorithm is used to calculate the system overhead to all underlying nodes that meet the resource constraints. Select a virtual network function that optimizes both the service differentiation objective and system overhead. Migrate to action node Therefore, by differentiating the objectives and system overhead of each virtual network function in the optimal service, the migration process's differentiation objectives and overall overhead (C) can be achieved as much as possible. tot .

[0120] After the action is completed, the feedback value R after VNF deployment is calculated based on the status of the underlying physical network resources, QoS, and service type. In one embodiment, determining the feedback value corresponding to the objective function of the current network service based on the service type of the current network service may include:

[0121] If the current network service's service type is a uRLLC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0122] If the current network service's service type is an mMTC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0123] If the current network service's service type is an eMBB type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0124] Among them, R uRLLC R represents the feedback value of a network service of type uRLLC. mMTC R represents the feedback value of mMTC type network services. eMBB This represents the feedback value of an eMBB type network service, Z represents the parameter that adjusts the feedback value to a positive number, α, β, γ, and δ represent coefficients, and D represents the feedback value. tot L represents the propagation delay between the current physical node and the previous deployment location for a uRLLC type network service. tot B represents the node load balancing of mMTC type network services. tot C represents the link bandwidth consumption of eMBB type network services. tot This represents the total system overhead of migrating the VNF to the action selection node location, where s represents the set of network servers and k represents the number of edge data centers. k Indicates network services, Indicates physical link Does it belong to network service? k The actual physical forwarding path of the j-th virtual link The value, N represents link delay. s Represents the set of physical nodes on a data center. Represents physical nodes The remaining computing resources Indicates physical link Link bandwidth resources, Indicates physical link The remaining link bandwidth resources; α' and β' represent the overhead coefficients, respectively. Represents the network server of VNF From physical nodes Migrate to Energy consumption expenditure, Represents physical nodes arrive The actual migration path Indicates physical link Does it belong to a physical node? arrive The actual migration path value, Indicates from physical node Migrate to migration overhead, B(s) k This represents the resource consumption on the physical link. Indicates from physical node Migrate to The number of jumps.

[0125] Step 104: Based on the new configuration scheme for network services, determine the migration sequence from the current configuration to the new configuration, and reconfigure all network services.

[0126] For finding the migration sequence from the current configuration to the new configuration, this embodiment proposes to use the A* algorithm. The A* search will be optimized to reduce the search space and improve the speed at which the algorithm completes its steps. The optimization strategy is as follows:

[0127] 1) Reduce exploration space: The open table discards nodes that are found to be infeasible configurations. In this problem, infeasible configurations are useless and violate the constraints, therefore they are not considered nodes in the migration sequence. Discarding infeasible configurations found in the open table reduces memory usage and improves algorithm efficiency.

[0128] 2) Narrowing the scope of neighbor checks: When exploring the neighbors of the current node, it is only necessary to check the constraints related to the migrated VNFs and hosts. To determine whether a node is a feasible configuration, it is necessary to check the constraints related to all VNFs and hosts in that configuration. However, when exploring neighbors, since the current node is a feasible configuration, only the constraints related to the migrated VNFs and hosts need to be checked, which can greatly reduce the checking time.

[0129] 3) Caching of Infeasible Configuration Nodes: Cache detected infeasible configuration nodes. Exploring a migration sequence in a migration graph may involve visiting a node multiple times. Each visit requires checking if it's a feasible configuration, and repeatedly checking nodes can waste time. Therefore, we can cache infeasible configurations to avoid redundant checks.

[0130] Based on the above optimization strategy, in one embodiment, the migration sequence from the current configuration to the new configuration is determined according to the new configuration scheme of the network service, including:

[0131] Initialize the open and close tables;

[0132] For network services that have not been migrated, set the original configuration of the network service as the starting point and the new configuration as the target point, and add the starting point to the open table;

[0133] Take the starting point with the minimum cost function in the open table as the current node, and check whether the current node is the target point;

[0134] If the current node is not the target node, move the current node into the close list;

[0135] If the current node is the target node, following the A* algorithm, start from the current node and backtrack to the starting point through the parent pointer of each node to obtain the migration sequence and output it. It should be noted that the migration sequence here stores the path between the starting point (original configuration) and the target point (new configuration) of all VNFs.

[0136] For neighboring nodes that the current node has not explored, if the neighboring node is a cached infeasible configuration node, then discard the neighboring node; if the neighboring node's configuration is infeasible, then discard the neighboring node.

[0137] Add the remaining neighbor nodes to the open table and calculate the cost function of the remaining neighbor nodes. Then, proceed to the step of "taking the point with the smallest cost function in the open table as the current node" until multiple migration sequences corresponding to all network services are output.

[0138] Based on the heuristic algorithm described above for finding the migration sequence from the current configuration to the new configuration, after the final migration is completed, the network resources are reconfigured, bringing the network back to its optimal operating state in terms of resource utilization.

[0139] This invention, in its embodiments, prioritizes network services and determines the execution order of the reconfiguration algorithm when it is triggered. Based on this order, it establishes a network service reconfiguration problem model that differentiates between service types and uses a deep reinforcement learning-based service reconfiguration algorithm to determine a new configuration scheme for each network service. According to the new configuration scheme, it determines the migration sequence from the current configuration to the new configuration and reconfigures all network services. Based on this network service reconfiguration method, it can learn online based on system status and feedback values ​​from the environment after execution mapping. It also considers the differentiated requirements of network services for latency, bandwidth, and node load balancing, as well as the cost of reconfiguration, proposing specific reconfiguration optimization objectives that match business needs. For the reconfiguration problem, a heuristic algorithm is used to accelerate the search speed, enabling the rapid finding of the migration sequence from the current configuration to the new configuration.

[0140] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0141] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0142] Figure 2A schematic diagram of the dynamic network service reconfiguration device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0143] like Figure 2 As shown, the dynamic network service reconfiguration device 2 includes: a detection module 21, a sorting module 22, a reconfiguration module 23, and a migration module 24.

[0144] Detection module 21 is used to detect whether the network service reconfiguration algorithm is triggered;

[0145] The sorting module 22 is used to prioritize network services and determine the execution order of the reconfiguration algorithm when the network service reconfiguration algorithm is triggered.

[0146] The reconfiguration module 23 is used to establish a network service reconfiguration problem model that distinguishes business types according to the execution order, and to determine the new configuration scheme for each network service based on the service reconfiguration algorithm of deep reinforcement learning.

[0147] Migration module 24 is used to determine the migration sequence from the current configuration to the new configuration based on the new configuration scheme of the network services, and to reconfigure all network services.

[0148] In one possible implementation, when the detection module 21 triggers the network service reconfiguration algorithm, it can be used to:

[0149] Obtain the remaining computing resources and remaining link bandwidth resources of each physical node in the infrastructure network;

[0150] Based on the remaining computing resources of each physical node, check whether the variance of the resource utilization rate of the remaining computing resources meets the first condition.

[0151] Based on the remaining link bandwidth resources of each physical node, check whether the variance of the resource utilization of the remaining link bandwidth resources meets the second condition.

[0152] When the variance of the resource utilization of the remaining computing resources meets the first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets the second condition, the network service reconfiguration algorithm is determined to be triggered.

[0153] In one possible implementation, when the variance of the resource utilization of the remaining computing resources meets a first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets a second condition, the detection module 21 determines that the network service reconfiguration algorithm is triggered, and then performs the following:

[0154] when or At that time, determine the network service reconfiguration algorithm to be triggered;

[0155] Where, Ns Represents a set of physical nodes. This represents the i-th physical node. γ represents the remaining computing resources of the i-th physical node. c L represents the variance threshold for the utilization rate of computational resources. s Represents the set of physical links between physical nodes. Represents physical nodes With physical nodes The physical link between them γ represents the remaining link bandwidth resources of the physical link. B The variance threshold representing the utilization rate of bandwidth resources.

[0156] In one possible implementation, when prioritizing network services and determining the execution order of the reconfiguration algorithm, the sorting module 22 is used for:

[0157] Calculate the ratio of the remaining lifetime of each network service to its total lifetime;

[0158] The execution order of the reconfiguration algorithm is determined by prioritizing all network services based on their ratios, with smaller ratios indicating higher priority.

[0159] In one possible implementation, when the reconfiguration module 23 establishes a network service reconfiguration problem model that distinguishes between service types according to the execution order, and determines the new configuration scheme for each network service based on a deep reinforcement learning-based service reconfiguration algorithm, it is used for:

[0160] In the execution order, it is detected whether the current network service that has not been reconfigured shares a VNF;

[0161] If the current network service shares a VNF, then identify the first physical node in the infrastructure network that has the same VNF type as the current network service and shares the VNF, and deploy the VNF there.

[0162] If the current network service does not share a VNF, then the physical node with the lowest load is determined to deploy the VNF based on the service type of the current network service.

[0163] If a VNF is deployed for the current network service, then the state corresponding to the current network service is input into the DQN network to obtain the Q(s) of all actions under that state. t The values ​​of a and θ are used to input the obtained actions into the environment to obtain the new state corresponding to the current network service; the feedback value corresponding to the objective function of the current network service is determined according to the service type of the current network service.

[0164] For the remaining VNFs of the current network service, an action is randomly selected with probability C. If an action cannot be randomly selected using probability C, then the action is selected based on the maximum Q(s). t The values ​​of a and θ are used to determine the action, and the obtained action is input into the environment to obtain a new state; the feedback values ​​after the remaining VNF deployment are calculated;

[0165] Save the quadruple (s,A,R,s') obtained from the above steps as a record; where s represents the state of all network services, A represents the action of all network services, R represents the feedback value after all VNFs are deployed, and s' represents the new state of all network services.

[0166] If the current iteration count has not reached the preset iteration count, then select actions and calculate feedback values ​​for the remaining VNFs of the current network service. If all VNFs of the current network service have been selected and feedback values ​​have been calculated, then proceed to the "Detect whether the current network service that has not been reconfigured shares VNFs" step to process the next network service accordingly.

[0167] If the current iteration count reaches the preset iteration count, a preset number of records are randomly selected from all saved records as training values ​​to train the neural network and obtain the optimized parameters θ.

[0168] Update Q(s) according to the preset strategy and the optimization parameter θ. t The Q value (a; θ) is calculated iteratively based on the updated Q value until the maximum number of iterations is reached.

[0169] In one possible implementation, when the reconfiguration module 23 determines the feedback value corresponding to the objective function of the current network service based on the service type of the current network service, it is used to:

[0170] If the current network service's service type is a uRLLC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0171] If the current network service's service type is an mMTC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0172] If the current network service's service type is an eMBB type network service, then according to Determine the feedback value corresponding to the objective function of the current network service;

[0173] Among them, R uRLLCR represents the feedback value of a network service of type uRLLC. mMTC R represents the feedback value of mMTC type network services. eMBB This represents the feedback value of an eMBB type network service, Z represents the parameter that adjusts the feedback value to a positive number, α, β, γ, and δ represent coefficients, and D represents the feedback value. tot L represents the propagation delay between the current physical node and the previous deployment location for a uRLLC type network service. tot B represents the node load balancing of mMTC type network services. tot C represents the link bandwidth consumption of eMBB type network services. tot This represents the total system overhead of migrating the VNF to the action selection node location, where s represents the set of network servers and k represents the number of edge data centers. k Indicates network services, Indicates physical link Does it belong to network service? k The actual physical forwarding path of the j-th virtual link The value, N represents link delay. s Represents the set of physical nodes on a data center. Represents physical nodes The remaining computing resources Indicates physical link Link bandwidth resources, Indicates physical link The remaining link bandwidth resources; α' and β' represent the overhead coefficients, respectively. Represents the network server of VNF From physical nodes Migrate to Energy consumption expenditure, Represents physical nodes arrive The actual migration path Indicates physical link Does it belong to a physical node? arrive The actual migration path value, Indicates from physical node Migrate to migration overhead, B(s) k This represents the resource consumption on the physical link. Indicates from physical node Migrate to The number of jumps.

[0174] In one possible implementation, when migration module 24 determines the migration sequence from the current configuration to the new configuration based on the new configuration scheme of the network service, it is used to:

[0175] Initialize the open and close tables;

[0176] For network services that have not been migrated, set the original configuration of the network service as the starting point and the new configuration as the target point, and add the starting point to the open table;

[0177] Take the starting point with the minimum cost function in the open table as the current node, and check whether the current node contains a target point;

[0178] If the current node is not the target node, move the current node into the close table;

[0179] If the current node is the target point, following the idea of ​​the A* algorithm, start from the current node, backtrack to the starting point through the parent pointer of each node, obtain the migration sequence and output it;

[0180] For unexplored neighboring nodes of the current node, if the neighboring node is a cached infeasible configuration node, then the neighboring node is discarded; if the configuration of the neighboring node is infeasible, then the neighboring node is discarded.

[0181] Add the remaining neighbor nodes to the open table, calculate the cost function of the remaining neighbor nodes, and then proceed to the step of "taking the point with the smallest cost function in the open table as the current node".

[0182] The aforementioned dynamic network service reconfiguration device, when the network service reconfiguration algorithm is triggered, uses a sorting module to prioritize network services and determine the execution order of the reconfiguration algorithm. The reconfiguration module, following the execution order, establishes a network service reconfiguration problem model that differentiates between service types and determines a new configuration scheme for each network service based on a deep reinforcement learning-based service reconfiguration algorithm. The migration module, based on the new configuration scheme, determines the migration sequence from the current configuration to the new configuration and reconfigures all network services. Based on this network service reconfiguration method, it can learn online based on system status and feedback values ​​from the environment after execution mapping. It also considers the differentiated requirements of network services for latency, bandwidth, and node load balancing, as well as the cost of reconfiguration, proposing specific reconfiguration optimization objectives that match business needs. For the reconfiguration problem, a heuristic algorithm is used to accelerate the search speed, enabling the rapid finding of the migration sequence from the current configuration to the new configuration.

[0183] Figure 3 This is a schematic diagram of a terminal provided in an embodiment of the present invention. For example... Figure 3As shown, the terminal 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, it implements the steps in the various dynamic network service reconfiguration method embodiments described above, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of modules / units 21 to 24 shown.

[0184] For example, the computer program 32 can be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal 3. For example, the computer program 32 can be divided into... Figure 2 Modules / units 21 to 24 are shown.

[0185] The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0186] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0187] The memory 31 can be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 can also be an external storage device of the terminal 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal 3. Furthermore, the memory 31 can include both internal storage units and external storage devices of the terminal 3. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0188] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0189] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0190] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0191] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0194] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various dynamic network service reconfiguration method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0195] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for dynamic network service reconfiguration, characterized in that, include: Detect whether the network service reconfiguration algorithm has been triggered; When the network service reconfiguration algorithm is triggered, the network services are prioritized to determine the execution order of the reconfiguration algorithm. Following the execution order, a network service reconfiguration problem model is established that distinguishes between business types, and a new configuration scheme for each network service is determined based on a deep reinforcement learning-based service reconfiguration algorithm, including: In the execution order, it is detected whether the current network service that has not been reconfigured shares a VNF; If the current network service shares a VNF, then identify the first physical node in the infrastructure network that has the same VNF type as the current network service and shares the VNF, and deploy the VNF there. If the current network service does not share a VNF, then the physical node with the lowest load is determined to deploy the VNF based on the service type of the current network service. If a VNF is deployed for the current network service, then the state corresponding to the current network service is input into the DQN network to obtain all actions under that state. The value is obtained by inputting the action into the environment to obtain the new state corresponding to the current network service; the feedback value corresponding to the objective function of the current network service is determined according to the service type of the current network service; wherein, This indicates the current status of the network service. express All actions in the state, Indicates the optimization parameters; For the remaining VNFs of the current network service, an action is randomly selected with probability C. If an action cannot be randomly selected using probability C, then the action with the highest probability is selected. The value determines the action, and the obtained action is input into the environment to obtain a new state; and the feedback value after the remaining VNF deployment is calculated; Get the quadruple Saved as a record; among which, This indicates the status of all network services. This represents the actions corresponding to all network services. This represents the feedback value after all VNF deployments are completed. This indicates the new state of all network services; Check if the current iteration count has reached the preset iteration count; If the current iteration count reaches the preset iteration count, a preset number of records are randomly selected from all saved records as training values ​​to train the neural network and obtain optimized parameters. ; According to the preset strategy and the optimization parameters ,renew The value is calculated iteratively based on the updated Q value until the maximum number of iterations is reached. Based on the new configuration scheme of the network services, determine the migration sequence from the current configuration to the new configuration, and reconfigure all network services.

2. The dynamic network service reconfiguration method according to claim 1, characterized in that, Methods that trigger network service reconfiguration algorithms include: Obtain the remaining computing resources and remaining link bandwidth resources of each physical node in the infrastructure network; Based on the remaining computing resources of each physical node, check whether the variance of the resource utilization rate of the remaining computing resources meets the first condition. Based on the remaining link bandwidth resources of each physical node, check whether the variance of the resource utilization of the remaining link bandwidth resources meets the second condition. When the variance of the resource utilization of the remaining computing resources meets the first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets the second condition, the network service reconfiguration algorithm is determined to be triggered.

3. The dynamic network service reconfiguration method according to claim 2, characterized in that, When the variance of the resource utilization of the remaining computing resources meets the first condition, or the variance of the resource utilization of the remaining link bandwidth resources meets the second condition, the network service reconfiguration algorithm is determined to be triggered, including: when ,or At that time, determine the network service reconfiguration algorithm to be triggered; in, Represents a set of physical nodes. Indicates the first One physical node Indicates the first The remaining computing resources of each physical node, The variance threshold representing the utilization rate of computing resources; Represents the set of physical links between physical nodes. Represents physical nodes With physical nodes The physical link between them This represents the remaining link bandwidth resources of the physical link. The variance threshold representing the utilization rate of bandwidth resources.

4. The dynamic network service reconfiguration method according to claim 1, characterized in that, Prioritizing the network services to determine the execution order of the reconfiguration algorithm includes: Calculate the ratio of the remaining lifetime of each network service to its total lifetime; The execution order of the reconfiguration algorithm is determined by prioritizing all network services based on their ratios, with smaller ratios indicating higher priority.

5. The dynamic network service reconfiguration method according to claim 3, characterized in that, After detecting whether the current iteration number has reached the preset iteration number, the method further includes: If the current iteration count has not reached the preset iteration count, then actions are selected and feedback values ​​are calculated for the remaining VNFs of the current network service; if all VNFs of the current network service have completed action selection and feedback value calculation, then the process jumps to the "detect whether the current network service that has not been reconfigured shares VNFs" step to process the next network service accordingly.

6. The dynamic network service reconfiguration method according to claim 5, characterized in that, Determining the feedback value corresponding to the objective function of the current network service based on the service type of the current network service includes: If the current network service's service type is a uRLLC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service; If the current network service's service type is an mMTC type network service, then according to Determine the feedback value corresponding to the objective function of the current network service; If the current network service's service type is an eMBB type network service, then according to Determine the feedback value corresponding to the objective function of the current network service; in, This represents the feedback value of a network service of type uRLLC. This represents the feedback value of a network service of type mMTC. This represents the feedback value of a network service of type eMBB. This parameter indicates that the feedback value will be adjusted to a positive number. They represent coefficients respectively. This represents the propagation delay between the current physical node and the previous deployment location for a uRLLC type network service. This indicates the node load balancing of mMTC type network services. This indicates the link bandwidth consumption of eMBB type network services. This represents the total system overhead of migrating the VNF to the action selection node location. Represents a collection of network servers. Indicates the number of edge data centers. Indicates network services, Indicates physical link Does it fall under the category of network services? The actual physical forwarding path of the j-th virtual link The value, Indicates link latency. Represents the set of physical nodes on a data center. Represents physical nodes The remaining computing resources Indicates physical link Link bandwidth resources, Indicates physical link The remaining link bandwidth resources; , These represent the cost coefficients, Represents the network server of VNF From physical nodes Migrate to Energy consumption expenditure, Represents physical nodes arrive The actual migration path Indicates physical link Does it belong to a physical node? arrive The actual migration path value, Indicates from physical node Migrate to migration overhead, This indicates the amount of resources consumed on the physical link. Indicates from physical node Migrate to The number of jumps.

7. The dynamic network service reconfiguration method according to claim 1, characterized in that, Based on the new configuration scheme of the network service, determine the migration sequence from the current configuration to the new configuration, including: Initialize the open and close tables; For network services that have not been migrated, set the original configuration of the network service as the starting point and the new configuration as the target point, and add the starting point to the open table; Take the starting point with the minimum cost function in the open table as the current node, and check whether the current node contains a target point; If the current node is not the target point, move the current node into the close table; If the current node is the target node, following the idea of ​​the A* algorithm, start from the current node and backtrack to the starting point through the parent pointer of each node to obtain the migration sequence and output it; For unexplored neighboring nodes of the current node, if the neighboring node is a cached infeasible configuration node, then the neighboring node is discarded; if the configuration of the neighboring node is infeasible, then the neighboring node is discarded. Add the remaining neighbor nodes to the open table, calculate the cost function of the remaining neighbor nodes, and then proceed to the step of "taking the point with the smallest cost function in the open table as the current node".

8. A dynamic network service reconfiguration device, characterized in that, include: The detection module is used to detect whether the network service reconfiguration algorithm has been triggered. The sorting module is used to prioritize the network services and determine the execution order of the reconfiguration algorithm when the network service reconfiguration algorithm is triggered. The reconfiguration module is used to establish a network service reconfiguration problem model that distinguishes service types according to the execution order, and to determine a new configuration scheme for each network service based on a service reconfiguration algorithm based on deep reinforcement learning; the reconfiguration module is used to detect whether the current network services that have not been reconfigured share VNFs under the execution order. If the current network service shares a VNF, then identify the first physical node in the infrastructure network that has the same VNF type as the current network service and shares the VNF, and deploy the VNF there. If the current network service does not share a VNF, then the physical node with the lowest load is determined to deploy the VNF based on the service type of the current network service. If a VNF is deployed for the current network service, then the state corresponding to the current network service is input into the DQN network to obtain all actions under that state. The value is input into the environment to obtain the new state corresponding to the current network service; Determine the feedback value corresponding to the objective function of the current network service based on the service type of the current network service; For the remaining VNFs of the current network service, an action is randomly selected with probability C. If an action cannot be randomly selected using probability C, then the action with the highest probability is selected. The value determines the action, and the obtained action is input into the environment to obtain a new state; and the feedback value after the remaining VNF deployment is calculated; wherein, This indicates the current status of the network service. express All actions in the state, Indicates the optimization parameters; Get the quadruple Saved as a record; among which, This indicates the status of all network services. This represents the actions corresponding to all network services. This represents the feedback value after all VNF deployments are completed. This indicates the new state of all network services; Check if the current iteration count has reached the preset iteration count; If the current iteration count reaches the preset iteration count, a preset number of records are randomly selected from all saved records as training values ​​to train the neural network and obtain optimized parameters. ; According to the preset strategy and the optimization parameters ,renew The value is calculated iteratively based on the updated Q value until the maximum number of iterations is reached. The migration module is used to determine the migration sequence from the current configuration to the new configuration based on the new configuration scheme of the network service, and to reconfigure all network services.

9. A terminal, comprising a memory and a processor, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the dynamic network service reconfiguration method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the dynamic network service reconfiguration method as described in any one of claims 1 to 7.

Images