Digital twinning assisted VNF migration method based on space-time large model resource demand prediction in industrial Internet of Things

Abstract

The invention relates to a VNF migration method based on space-time large model resource demand prediction for digital twinning assistance in an industrial Internet of Things, and belongs to the technical field of mobile communication. Aiming at the problems of poor service continuity, migration strategy lagging and high synchronization delay caused by resource demand fluctuation, the following scheme is provided: constructing a digital twin network architecture to collect node information in real time; spatial features are extracted through a graph convolutional network, time dependence is captured in combination with a multi-head self-attention mechanism, and future resource requirements are predicted by using a large language model; and establishing a system optimization model fusing energy consumption, time delay and load balancing, and adopting a heterogeneous multi-agent reinforcement learning algorithm to decide VNF migration and digital twin node re-association. According to the method, the resource prediction precision is remarkably improved, the migration energy consumption is reduced by more than 15%, the synchronization delay is reduced by 20%, the network load balance is effectively optimized, and the overall effectiveness of the system is improved by 30%.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications, and relates to a method for migrating VNF assisted by digital twin based on spatio-temporal large model resource demand prediction in industrial Internet of Things. Background Art

[0002] With the development of industrial Internet of Things technology, a large number of new applications and services have emerged, and the number of users has been continuously increasing, resulting in a substantial increase in network traffic. The location of mobile service requests changes over time, and the network traffic will also change dynamically, which may cause the existing network configuration to fail to meet the service quality (Quality of Service, QoS) requirements of services, and further lead to service interruptions. To avoid such problems, it is necessary to migrate the successfully deployed VNFs in the network slice to meet the QoS requirements of corresponding mobile services. Proactive migration formulates migration strategies based on pre-predicted resource requirements, which can effectively solve the problem of lagging migration strategies. Large Language Model (LLM) has demonstrated excellent capabilities in time series prediction. Through its powerful context understanding and long-term dependence modeling capabilities, LLM can provide accurate resource demand predictions in large-scale network environments.

[0003] Digital Twin (DT), as an emerging technology, realizes real-time mapping of physical entities by constructing digital models and conducts simulations in the virtual space to reduce actual application costs. With the development of communication technology, the integration of DT and network slices is gradually becoming a trend in future network development. By using DT technology, network node information, topological structure, and service requirements can be obtained in real time. Through the integration of technologies such as deep learning, data mining, and edge computing, DT can predict the future development trends of physical entities throughout their life cycles and assist Software Defined Network (SDN) controllers in providing intelligent support in resource optimization decisions.

[0004] Due to the dynamic changes in service requirements, user locations, and node states, it is necessary to design prediction algorithms in large-scale networks to accurately predict the future resource requirements of VNFs on each DT synchronization node. At the same time, there may be errors in resource prediction, and there is a risk of migrating VNFs to the wrong servers, further leading to a sharp increase in energy consumption and even service interruptions. In addition, after the VNF migration, the original DT synchronization node associated with it may fail to synchronize due to excessive synchronization delay. Selecting appropriate DT synchronization nodes for re-association can ensure the reliable mapping of node information. Although some current studies consider combining DT and network slice technologies, how to make more accurate and intelligent decisions during the VNF migration process to ensure service QoS remains to be studied. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a VNF migration method assisted by digital twins for resource demand prediction based on spatio-temporal large models in the industrial Internet of Things, which can optimize the VNF migration strategy and DT re-association strategy to ensure business deterministic QoS to the greatest extent and improve system utility.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A VNF migration method assisted by digital twins for resource demand prediction based on spatio-temporal large models in the industrial Internet of Things, which is characterized by including the following steps:

[0008] S1: In the industrial Internet of Things scenario, construct a VNF migration architecture in the digital twin-assisted network slice, and use digital twin synchronization nodes to collect network node information, including network devices such as user equipment, base stations, and servers.

[0009] S2: Extract the spatio-temporal correlation in the historical data of VNF nodes, and then perform resource demand prediction based on the large language model.

[0010] S3: Construct a system utility optimization model for digital twin-assisted VNF migration from aspects such as energy consumption, service delay, DT synchronization delay, and load balancing.

[0011] S4: Combine the predicted future resource demands and slice priorities to propose a digital twin-assisted HAPPO algorithm to solve the migration problem, and solve the digital twin re-association problem based on the COMA algorithm.

[0012] Further, the digital twin-assisted VNF migration architecture includes a physical network layer and a digital twin layer;

[0013] The physical network layer includes physical network elements, such as BSs in the RAN domain, MEC servers close to BSs, Internet of Things terminals, and multiple servers in the CN domain. The physical network elements support both software-defined and virtualization technologies, and service requests in the physical network layer are sent to the digital twin layer in the form of service function chains.

[0014] The digital twin layer includes a digital twin basic model and a digital twin function model. The data warehouse in the digital twin function model collects various information of the physical network through DT-VNF nodes, thereby constructing the digital twin basic model, and interacting with the function model to achieve policy simulation and verification, and optimize the VNF migration decision.

[0015] Further, the network resources include computing resources, storage resources, and link bandwidth resources.

[0016] Furthermore, multiple VNFs within a network slice are connected in a specified order to form a service link, and a service link is dedicated to the transmission of service data between specific source and destination nodes.

[0017] Furthermore, in S2, a graph convolutional network and multi-head self-attention are used to capture spatial features and temporal features respectively:

[0018]

[0019] In the formula, H s represents the spatial feature matrix; represents the normalized adjacency matrix of the graph, which is used to avoid the influence of node degree differences on information propagation to ensure that each node can retain its own information. D is the node degree matrix; A is the original adjacency matrix; I is the identity matrix; l ∈ {0, 1} represents the number of neural network layers, and W l represents the weight matrix of the l-th layer.

[0020]

[0021] In the formula, H t,h represents the temporal feature matrix of the h-th head in the multi-head attention; d k represents the dimension of the key vector, where X represents the input data, are the weight matrices of query, key, and value respectively.

[0022] Furthermore, in S2, the VNF resource demand prediction algorithm includes the following steps:

[0023] S61: Initialize the network topology and node feature matrix;

[0024] S62: Determine whether the set number of iterations has been exceeded. If so, stop the iteration; otherwise, continue to execute S83;

[0025] S63: Determine whether the number of VNFs has been exceeded. If so, stop the iteration; otherwise, continue to execute S84;

[0026] S64: Sample the historical resource demand data of VNFs to construct an input sequence;

[0027] S65: Perform multi-layer graph convolutional operations of the GCN to extract spatial features;

[0028] S66: Calculate the self-attention scores and perform multi-head attention calculations to extract temporal features;

[0029] S67: Fuse the spatial and temporal features through a fully connected layer to obtain a spatio-temporal feature vector;

[0030] S68: Input the spatio-temporal feature vector into the large language model module, and update the resource demand prediction value by freezing and fine-tuning the pre-trained large language model.

[0031] Further, in S3, the system utility optimization problem of the digital twin-assisted VNF migration is expressed as:

[0032]

[0033] In the formula, the optimization variables include the node mapping matrix F, the link mapping matrix L, and the node association matrix A. U(t) represents the total system utility in time slot t.

[0034] Further, in S4, the optimization problem is transformed into a Markov decision process model, including state, action, transition probability, and reward, expressed as where represents the state space, represents the action space, represents the state transition probability, represents the reward function. The agent is the service request initiated by each Internet of Things terminal in the system scenario. Each agent selects an action from the action space by considering the network state. In the VNF migration problem, each service in the system is used as an agent, and in the DT re-association problem, each DT synchronization node in the system is used as an agent.

[0035] In the VNF migration phase, the state space includes the physical node network state information and physical link network state information in time slot t - 1, the service request set, the demand prediction values of VNF computing resources and storage resources, and the demand prediction value of the bandwidth resource of the virtual link.

[0036] The state space in the DT re-association problem includes the network state information of DT nodes, the link state information between DT nodes, the association relationship between DT nodes and VNF nodes in time slot t - 1, the VNF mapping relationship, the mapping relationship of virtual links, the remaining computing resources of DT nodes, the remaining storage resources, and the remaining bandwidth resources between DT nodes in time slot t.

[0037] The beneficial effects of the present invention are as follows: In view of the problem that the lack of guarantee for service continuity is faced due to the fluctuating resource requirements and complex node dependency relationships in the industrial Internet of Things, the present invention proposes a DT-assisted VNF migration algorithm based on spatio-temporal large language model resource requirement prediction. First, a spatio-temporal large language model resource prediction method combining graph convolutional network and multi-head self-attention is proposed. Secondly, a DT-assisted VNF migration model composed of energy consumption, delay, load balancing, etc. is constructed, and a joint optimization model of VNF migration and DT re-association aiming to maximize the long-term utility of the system is established. Finally, a migration algorithm based on heterogeneous multi-agent proximal policy optimization is proposed to solve the VNF migration problem according to the resource requirements of the prediction model, and a counterfactual multi-agent algorithm is proposed to solve the DT re-association problem.

[0038] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. Brief Description of the Drawings

[0039] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0040] Figure 1 is a schematic diagram of the digital twin-assisted VNF migration architecture;

[0041] Figure 2 is a resource prediction framework diagram of the spatio-temporal large model;

[0042] Figure 3 is a framework diagram of the DT-HAPPO algorithm;

[0043] Figure 4 is a framework diagram of the COMA algorithm. Detailed Embodiments

[0044] The following specific examples illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention schematically. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0045] Among them, the attached drawings are only for illustrative purposes, showing only schematic diagrams rather than physical diagrams, and should not be construed as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the attached drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the attached drawings may be omitted.

[0046] In the attached drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the attached drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the attached drawings are only for illustrative purposes and should not be construed as a limitation to the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0047] The embodiments of the present invention propose a method for VNF migration assisted by digital twin based on spatio-temporal large model resource demand prediction in industrial Internet of Things, which can optimize the VNF migration strategy and DT re-association strategy to ensure the business deterministic QoS to the greatest extent and improve the system utility.

[0048] The steps of the method are as follows:

[0049] S1: In the industrial Internet of Things scenario, construct a VNF migration architecture assisted by digital twin in the network slice, and use the digital twin synchronization node to collect network node information, including network devices such as user equipment, base stations, and servers.

[0050] S2: Extract the spatio-temporal correlation in the historical data of the VNF node, and then perform resource demand prediction based on the large language model.

[0051] S3: Construct a system utility optimization model for digital twin assisted VNF migration from aspects such as energy consumption, service delay, DT synchronization delay, and load balancing.

[0052] S4: Combine the predicted future resource requirements and slice priorities to propose a digital twin assisted HAPPO algorithm to solve the migration problem, and solve the digital twin re-association problem based on the COMA algorithm.

[0053] Further, the digital twin assisted VNF migration architecture includes a physical network layer and a digital twin layer. Figure 1 It is a schematic diagram of the digital twin assisted VNF migration architecture.

[0054] The physical network layer includes physical network elements, such as BS in the RAN domain, MEC servers close to the BS, IoT terminals, and multiple servers in the CN domain. The physical network elements support both software and virtualization technologies. The service requests in the physical network layer are sent to the digital twin layer in the form of a service function chain.

[0055] The digital twin layer includes a digital twin basic model and a digital twin functional model. The data warehouse in the digital twin functional model collects various types of information of the physical network through the DT-VNF node to build a digital twin basic model, and interacts with the functional model to realize strategy simulation and verification, and optimize VNF migration decisions.

[0056] Furthermore, the network resources include computing resources, storage resources and link bandwidth resources.

[0057] The network slice set is defined as I = {1, 2, ..., i, ..., I}, where a single network slice i contains several virtual network nodes and related link resources, and can overlap with other slices in terms of geographical location. Each base station has an MEC server, and the base station set is defined as J = {1, 2, ..., j, ..., J}, and the location coordinates of base station j are fixed as (x b,j ,y b,j ). The position of the user equipment initiating the service request changes dynamically in the time slot t∈T, triggering the resource demand fluctuation or migration of the corresponding slice.

[0058] Assuming that each service is provided by a slice, the location coordinates of the service requesting device in slice i are expressed as (x u,i (t),y u,i (t)). At the same time, different types of services have different priorities. The priority of slice i is represented as pri i ∈[0,1],pri i The larger the value, the higher the service priority. Define the service request set of time slot t as Where R i (t) represents the service request rate of service i in time slot t, T i and J i They represent the delay requirement and jitter requirement of service i respectively.

[0059] The user device that initiates the service request first establishes a connection request through the BS, and then routes it to the virtual host hosting the corresponding VNF through the optical fiber link. Since the current state of the user device is related to its state at the previous moment, it is assumed that there is a time interval of δ between two time slots. t,t-1 The position of the user equipment in slice i remains unchanged, and the speed v of the user equipment in slice i is calculated using the Gauss-Markov model. i (t) and the moving direction θ i′(t) are respectively expressed as:

[0060]

[0061] Wherein, α v ∈ [0, 1], α θ′ ∈ [0, 1] respectively represent the influence degrees of the historical state on the current state in terms of speed and moving direction; are respectively the average speed and average moving direction of the user equipment; Φ i , Ψ i follow independent Gaussian distributions with two different means and variances.

[0062] The position of the service request device in slice i is updated as:

[0063] x u,i (t) = x u,i (t - 1) + v i (t - 1)cos(θ i (t - 1))δ t,t-1

[0064] y u,i (t) = y u,i (t - 1) + v i (t - 1)sin(θ i (t - 1))δ t,t-1

[0065] Furthermore, multiple VNFs in the network slice are connected in a specified order to form a service link, and a service link is dedicated to the transmission of service data between specific source and destination nodes.

[0066] The network in the physical network layer is defined as an undirected graph G = (V, E), where V is the set of physical nodes and E is the set of links between nodes. To better collect network node information, the physical network nodes include several VNF node sets for deploying VNF instances and the DT - VNF node set composed of DT synchronization nodes Therefore, the set of physical nodes V = V DT ∪V v . If the set of links between VNF nodes is E V , and the set of links between DT nodes is E DT , then the set of links E = E DT ∪E v .

[0067] The SFC request in the digital twin layer is represented as an undirected graph The VNF set of slice i is represented as The set of nodes Where n′ represents the number of VNFs in slice i, and the types of resource requirements on the VNFs include computing resource requirements and storage resource requirements Multiple VNFs in slice i are connected in a specified order to form a service link L i , and the link set The bandwidth resource requirement between two nodes on the link is

[0068] Each VNF has a corresponding node mapping relationship in the digital twin network. The node mapping matrix is represented as Where the binary variable represents the mapping relationship between the physical node hosting the VNF and the virtual node in the digital twin layer. If then the VNF node n1 in the physical network is mapped to the VNF n′ in the DTN. Otherwise

[0069] The link mapping between the link between two physical nodes and the corresponding virtual nodes in the digital twin layer uses the binary variable to represent. If the link mapping is successful, then Otherwise Therefore, the link mapping matrix is represented as Meanwhile, the VNF node needs to be associated with the DT-VNF node to maintain the information synchronization of the digital twin. The node association matrix is represented as Where the binary variable represents whether the VNF n′ on the VNF node n1 is associated with the DT-VNF node n2. If then the VNF n′ on the VNF node n1 is associated with the DT-VNF node n2. Otherwise

[0070] When the user moves, or a service node fails or is overloaded, the VNFs requested by the relevant slice need to perform service migration to meet the service quality. Therefore, the binary variable σ n′ (t) ∈ {0, 1} is defined to represent whether the VNF node n′ in the DT network migrates the service. If σ n′ (t) = 1, it means the node will perform migration. Otherwise, σ n′ (t) = 0. Meanwhile, due to the VNF migration, the originally associated DT-VNF node may not be able to accurately map the node state due to the increased synchronization delay. The DT-VNF needs to be migrated. Therefore, the binary variable is defined to represent whether the DT-VNF node migrates. If it means the DT-VNF node will perform migration. Otherwise

[0071] The load distribution between nodes and the network topology are spatially correlated. The load fluctuations of neighboring nodes will affect the slice resource requirements, and the link characteristics are also valuable in the network topology information. At the same time, the node resources on each path are affected by their historical states. This temporal correlation and spatial correlation do not exist independently, but continuously change and interact during the resource allocation process. Therefore, it is necessary to consider both spatial and temporal characteristics to more accurately predict resource requirements. Figure 2 It is a resource prediction framework diagram for the spatio-temporal large model.

[0072] The input data at time slot t is represented as where represents the CPU resource, storage resource, and bandwidth resource requirements of slice i at time slot t. Given the past time step length l and the prediction window size q, the resource requirements of the past l time steps within network slice i are The predicted resource requirements for the next q time steps are represented as The predicted data set is represented as Y t ={y1, y2, …, y i}.

[0073] Furthermore, in S2, the graph convolutional network and the multi-head self-attention are used to capture spatial features and temporal features respectively:

[0074] The graph convolutional network (GCN) aggregates the neighbor information of each node through the adjacency matrix to learn the spatial dependence. Two layers of GCN are used, where the first-layer neural network uses ReLu (Rectified Linear Unit) as the activation function, and the second-layer neural network uses Softmax as the activation function:

[0075]

[0076] In the formula, H s represents the spatial feature matrix; represents the normalized adjacency matrix of the graph, which is used to avoid the influence of the node degree difference on the information propagation to ensure that each node can retain its own information. D is the node degree matrix; A is the original adjacency matrix; I is the identity matrix; l ∈ {0, 1} represents the neural network layer number, and W l represents the weight matrix of the l-th layer.

[0077] Multi-head Self-Attention (MHSA) has significant advantages in processing large-scale time series data. It can perform parallel computing and efficiently utilize computing resources. The resource demand data of the past l time steps is used as the input of MHSA to extract time features, and then spatio-temporal features are obtained through feature fusion. The self-attention mechanism in MHSA is used to calculate the similarity between each time step and other time steps, and based on these similarities, the input features are weighted:

[0078]

[0079] In the formula, H t,h represents the time feature matrix of the h-th head in the multi-head attention; d k represents the dimension of the key vector, where X represents the input data, are the weight matrices of query, key, and value respectively.

[0080] When H t,h , h ∈ [1, N h is obtained, the time feature representation of the sliced resource data is:

[0081]

[0082] In the formula, [*, …, *] represents the concatenation operation; N h represents the number of attention heads; W O represents the linear transformation weight of the multi-head output, which is used to map the concatenated result back to the target space.

[0083] Based on the fully connected layer, the outputs of the two modules are fused to obtain the spatio-temporal feature H st as the input of the LLM module to predict future resource demands.

[0084] Since the received data has high-dimensional spatio-temporal features and GPT-2 cannot directly process high-dimensional spatio-temporal features, first, the channel independence is used to convert the multi-node spatio-temporal features into spatio-temporal features of multiple independent nodes. Each VNF in the network slice is regarded as a channel, denoted as Then, in order to reduce the feature dimension, the spatio-temporal features of adjacent time steps are grouped as a patch, and feature compression is performed on it through an encoding operation.

[0085] The adjacent time steps l a are used as a patch, and each patch is represented as:

[0086]

[0087] To make the dimension of the patch consistent with the embedding dimension of the GPT-2 model, a 1D convolutional layer Conv is used as the patch encoding layer, and the patch encoding is Meanwhile, to enable the model to understand the order of the sequence, position encoding is introduced. Given the patch position index of the node Use a learnable position encoding lookup table E pos , each position index corresponds to a vector in the lookup table, and this vector will add position encoding to each patch Add the position encoding and the patch encoding to obtain Then use it as the input of the GPT-2 model.

[0088] During fine-tuning, excessive parameter updates may cause the model to overfit or lose the general features learned during pre-training. Therefore, most parameters (such as the feed-forward layer and MHSA) are frozen, and only layer normalization is fine-tuned to avoid the above problems. Given the input e n′ , first send it into the pre-trained GPT-2 model, where the pre-trained large model contains L Transformer blocks, and the expected resource requirement of VNF n′ on network slice i is z i,n′ .

[0089] Flattening and rearrangement operations are performed through the output layer. The flattening operation is used to convert the model output into a 1D vector, and the rearrangement operation is used to rearrange the target output shape. Given the learnable weight matrix W out , if the resource requirement of the VNF node at the future q time step is y i,n′ ∈y i , it is expressed as:

[0090] y i,n′ =Rearrange(Flatten(z i,n′ )·W out )

[0091] Furthermore, in S2, the VNF resource requirement prediction algorithm includes the following steps:

[0092] S61: Initialize the network topology and the node feature matrix;

[0093] S62: Determine whether the set number of iterations is exceeded. If so, stop the iteration; otherwise, continue to execute S83;

[0094] S63: Determine whether the number of VNFs is exceeded. If so, stop the iteration; otherwise, continue to execute S84;

[0095] S64: Sample the historical resource requirement data of the VNF and construct the input sequence;

[0096] S65: Perform multi-layer graph convolution operations of the GCN to extract spatial features;

[0097] S66: Calculate self-attention scores, perform multi-head attention calculation, and extract temporal features;

[0098] S67: Fuse spatial features and temporal features through a fully connected layer to obtain a spatio-temporal feature vector;

[0099] S68: Use the spatio-temporal feature vector as input to the large language model module, and update the resource demand prediction value by freezing and fine-tuning the pre-trained large language model.

[0100] When the VNF in the slice request is successfully mapped, the corresponding host node is in the powered-on state. Assuming that a physical node can host at most one VNF in the same slice request, define the energy consumption required for the CPU computing resource of the computing unit as e c , so the computing energy consumption of network slice i is expressed as:

[0101]

[0102] Binary variable represents the working state of physical host node n1. When all VNFs on the node are not running, the working state of the host node Otherwise Specifically expressed as:

[0103]

[0104] Therefore, the node switching energy consumption of the physical host hosting the VNF in network slice i is:

[0105]

[0106] In the formula, V i represents the set of physical nodes required to deploy the VNF of slice i; e s represents the switching energy consumption of the physical host from the idle or sleep state to the working state.

[0107] In addition, when the physical host node is in the powered-on state, power needs to be consumed to maintain the working state, which is expressed as the basic energy consumption where e b represents the energy consumption required to maintain the node.

[0108] In the case of migration, the energy consumption of VNF migration consists of data transmission energy consumption and node switching energy consumption. First, the physical host corresponding to the newly determined VNF needs to switch its operating state, allocate corresponding storage space and computing resources to initialize the VNF. Then, the original VNF node transmits its historical relevant data to the new VNF node. Finally, the new VNF node receives and stores the data, and updates the configuration according to the relevant data.

[0109] The data transmission energy consumption is related to the packet size and the transmission distance. For the VNF in network slice i If the mapping state in the current time slot is different from that in the previous time slot, VNF migration will be triggered. Therefore, the VNF migration energy consumption is expressed as:

[0110]

[0111] In the formula, e d represents the energy consumed for transmitting a unit data packet; c mig (t) represents the size of the migrated data packet. If the migrated VNF can process and transmit service data normally in the follow-up, the VNF energy consumption in the case of normal migration is:

[0112]

[0113] Due to possible errors in resource prediction, there is a risk of migrating the source VNF node to the wrong destination node, which further causes high-energy-consuming service nodes to block the migration request, resulting in migration failure and even service interruption.

[68] . Although it is possible to migrate to other host nodes again to ensure service quality, frequent migration will result in additional energy consumption overhead, and a relatively large amount of energy consumption is unacceptable. Therefore, the energy consumption caused by migration errors needs to be considered. Migration errors are divided into three types: premature migration, delayed migration, and wrong migration.

[0114] Premature migration means that the destination node of migration is the server that will be used by the current service request subsequently. Define the binary variable f pre (t) ∈ {0, 1} to indicate whether there is premature migration. When there is a premature migration state, the time point when the migration is completed is earlier than the first use time of the target node in the migration strategy, f pre (t) = 1, otherwise f pre (t) = 0. In this case, the target node may not be able to bear the load of the actual service, and the data must be transmitted from the prematurely migrated target node to the current node, and this backhaul process will cause additional node switching energy consumption.

[0115] Delayed migration means that the node migration is executed too late. The target node is expected to complete the migration at time slot t, but the actual VNF migration is completed in the next time slot. Define the binary variable f late(t) ∈ {0, 1} indicates whether there is a delayed migration. When there is a delayed migration status, the actual migration completion time is later than the expected time, f late (t) = 1, otherwise f late (t) = 0. In this case, the service request may continue to be processed by the overloaded or non-optimal source node, and the VNF needs to re-transmit the new historical resource data. The size of the re-migrated data packet is c′ mig (t), so additional data transmission energy consumption is required.

[0116] An incorrect migration means that the destination node of the migration is neither the service node to be accessed in the future nor the service node in the previous time slot. Define the binary variable f wrong (t) ∈ {0, 1} indicates whether there is an incorrect migration. When there is an incorrect migration status, f wrong (t) = 1, otherwise f wrong (t) = 0. In this case, the energy consumption of the entire migration process is wasted. Finally, the energy consumption of the VNF node is expressed as:

[0117]

[0118] The physical network also includes DT-VNF nodes for constructing the digital twin network. Maintaining the state of the digital twin requires consuming relevant computing resources. If is the computing resource requirement on the DT-VNF node, then the computing energy consumption of the DT-VNF is expressed as:

[0119]

[0120] Similar to the VNF node, the energy consumption of the DT-VNF node is expressed as:

[0121]

[0122] In the formula, represents the energy consumption required to maintain the basic working state after the DT node is activated; represents the DT migration energy consumption for migrating the DT node, c mig,DT (t) represents the size of the data packet for migrating the DT; the binary variable represents whether there is a DT switching power consumption. When , that is, when there is a VNF associated with the DT-VNF node, otherwise

[0123] Finally, the total energy consumption in the system is expressed as:

[0124]

[0125] In the successfully deployed network slice migration, the VNF can meet the QoS requirements of the corresponding mobile services. Delay is one of the important indicators of QoS, and effective measures need to be taken to ensure the delay requirements of the slice.

[0126] The user equipment accesses the base station through the wireless link and is assumed to be connected to the nearest base station. The uplink transmission rate between the service request device of network slice i and the nearest base station j is:

[0127]

[0128] In the formula, B i is the wireless channel bandwidth allocated to the device; p i , h i are the transmission power and channel gain respectively; is the path loss, θ is the path loss factor, represents the distance between the service request device i and the base station j; σ 2 = N0B i is the noise power, N0 is the noise power spectral density; I′ is the interference noise power from other devices.

[0129] In network slice i, if the size of the transmitted data packet is c i (t), then the wireless communication delay of the user equipment connected to the base station is:

[0130]

[0131] Assume that the VNF mapped on the physical node has been associated with the corresponding DT-VNF node, and the DTN construction has been completed. If the migration path set of network slice i is P n′ , and the migration path set of the associated DT-VNF node is When the source node is the same as the destination node, the migration path set is empty. The migration delay includes the transmission delay and the processing delay. The transmission delay T i trans (t) is the transmission time of the migration data on the link, and the processing delay T i p (t) is the processing time of the migration data on the node. The migration delay can be expressed as:

[0132] T i mig (t) = T i trans (t) + T i p (t)

[0133] Packet transmission between multiple nodes in a slice incurs transmission delay. The transmission delay for migration in network slice i is the sum of the delays on the transmission link, expressed as:

[0134]

[0135] Processing delay refers to the delay incurred when a VNF node executes a data processing task. It depends on the processing capacity of the service node, the size of the data packet in the service request, and the number of service requests on the service node. If represents the CPU cycles required to process a single bit of data, the processing delay is expressed as:

[0136]

[0137] Therefore, the end-to-end service delay at time slot t in the case of VNF migration is:

[0138] T i (t) = T i com (t) + T i mig (t)

[0139] The end-to-end service delay of network slice i should satisfy the delay constraint, expressed as:

[0140] T i (t) ≤ T i

[0141] In the scenario of VNF dynamic migration, due to changes in service requirements, network environment, and terminal location, there are certain fluctuations in service delay. Jitter is a measure of the service delay fluctuation. The jitter of slice i is defined as:

[0142]

[0143] where l is the time step of a time slot, is the average delay of slice i within the time step.

[0144] To ensure the determinacy of service data transmission, it is necessary to ensure that the jitter is within an acceptable range. If J i,th represents the maximum threshold of allowable jitter, the jitter constraint of slice i is expressed as:

[0145] J i (t) ≤ J i,th

[0146] Similarly, the DT-VNF node decides whether to migrate based on the destination node of the VNF migration. The DT migration delay is expressed as where:

[0147]

[0148] Represents the link bandwidth resources between DT migration nodes. Represents the computing resources required to synchronize VNF n′ on the DT-VNF node.

[0149] To measure the state consistency between the digital twin and its mapped VNF before and after VNF migration, the synchronization delay T syn (t) is introduced to ensure that the state of DT-VNF matches the state of VNF operation. The DT synchronization delay is defined as the sum of the DT migration delay and the synchronization data upload delay between the DT model and its mapped VNF node. To maintain the real-time synchronization of DT, the system synchronization delay should be less than the synchronization delay upper limit. That is The average system synchronization delay is expressed as:

[0150]

[0151] In the formula, Is the time required to transmit unit data per unit distance; c syn (t) is the size of the synchronization data packet; d(n1,n2) is the physical distance between the VNF node and the DT-VNF node.

[0152] During VNF migration, different nodes have different resource carrying capacities, including computing resources and storage resources. Without load balancing, some nodes may be overloaded with resources due to receiving too many service requests or traffic, thus affecting the network stability and service quality. Through load balancing, the service requests or traffic can be evenly distributed to multiple nodes, avoiding resource overload of individual nodes and improving the system stability and reliability. At time slot t, the resource load on the VNF node is defined as:

[0153]

[0154] In the formula, ω c And ω m Are the weights corresponding to the computing resources and storage resources respectively, and ω c +ω m = 1; And Are the maximum values of the computing resources and storage resources on the VNF node respectively. Similarly, at time slot t, the resource load on the DT-VNF node is:

[0155]

[0156] In the formula, And They are the maximum values of computing resources and storage resources on the DT-VNF nodes respectively. Therefore, in time slot t, the average resource load on the entire network nodes is:

[0157]

[0158] The standard deviation of the network resources of the physical nodes is used to measure the load balancing ability of the network, and the resource load balancing ability of the network is expressed as:

[0159]

[0160] To ensure the real-time load balancing ability of the network, if μ th represents the network load balancing threshold, then the load balancing of the network in each time slot needs to satisfy:

[0161] μ(t) ≤ μ th

[0162] Furthermore, in S3, the system utility optimization problem of the digital twin-assisted VNF migration is expressed as:

[0163]

[0164] In the formula, the optimization variables include the node mapping matrix F, the link mapping matrix L, and the node association matrix A. U(t) represents the total system utility in time slot t.

[0165] The priority of services usually directly affects the service charges and the operator's revenue. Generally speaking, the higher the service priority, the higher the MNO pricing and the higher the revenue. If γ in is the unit revenue, the service revenue is expressed as:

[0166]

[0167] The system service cost includes the delay cost and the system energy consumption. If ω e , ω t and ω d are the weight coefficients of energy consumption, service delay, and DT delay respectively, the system service cost is expressed as:

[0168]

[0169] In time slot t, the utility of the entire system is expressed as:

[0170] U(t) = RE(t) - CO(t)

[0171] Furthermore, in S4, the optimization problem is transformed into a Markov decision process model, including state, action, transition probability, and reward, expressed as where Denote the state space, Denote the action space, Denote the state transition probability, Denote the reward function. The agent is for the service requests initiated by each IoT terminal in the system scenario. Each agent selects an action from the action space by considering the network state. In the VNF migration problem, each service of the system is regarded as an agent, and in the DT reassociation problem, each DT synchronization node in the system is regarded as an agent.

[0172] In the VNF migration phase, the state space includes the physical node network state information and the network state information of the physical links at time slot t - 1, the service request set, the predicted values of the computing resources and storage resources required by the VNF, and the predicted values of the bandwidth resources required by the virtual links.

[0173] Among them, The network state information of physical node n1 is If Denote the state information of the physical link between any two VNF nodes , then the set of physical link state information is The predicted value of the computing resource requirement of the node is The predicted value of the storage resource requirement of the node is The predicted value of the bandwidth resource requirement of the link is

[0174] The action space at time slot t is represented as the set of mapping actions a v (t) = {F(t), L(t)}, including the mapping scheme of the VNF and the mapping scheme of the virtual link

[0175] The reward function in the migration phase is related to the energy consumption and latency of service migration, and is represented as r v,i (t) = RE i (t) - ω e E i (t) - ω t T i (t) + ι, where RE i (t) is the revenue of network slice i, ι is the penalty factor, and its value is 0 if the relevant constraints are satisfied, otherwise it is negative infinity. For all agents, the global reward function of the entire system is

[0176] The policy gradient method of the Heterogeneous-Agent Proximal Policy Optimization (HAPPO) algorithm improves and updates the policies of each agent sequentially, where the advantage function of each subsequent agent is iteratively adjusted according to the importance of the current sampling and the advantage function of the previous agent. After the update of the first i-1 agents is completed, the ratio of the new and old policies of the i-th agent is as follows:

[0177]

[0178] where is the candidate policy; is the old policy; the jointly updated policy function of the first i-1 agents is expressed as

[0179] The actor-critic network of the agent is updated through a clipping mechanism based on PPO, and the loss function for policy optimization is expressed as:

[0180]

[0181] where clip() represents the clipping function, which is used to limit the gap between the new and old policies within the clipping range ε; represents the estimated value of the advantage function of agent i, which is calculated through the Generalized Advantage Estimation method and is specifically expressed as:

[0182]

[0183] where the hyperparameter γ ∈ [0, 1] is the discount factor used to weigh future rewards; the hyperparameter χ ∈ [0, 1] is the Generalized Advantage Estimation smoothing parameter, which affects the trade-off between bias and variance during training. The critic network updates the network parameters by minimizing the global loss function, and its loss function is expressed as:

[0184]

[0185] Figure 3 is the framework diagram of the DT-HAPPO algorithm. The DT-HAPPO algorithm for VNF migration specifically includes the following steps:

[0186] (1) Initialize the actor-critic network, the global critic network, and the experience replay pool;

[0187] (2) All agents obtain the observation state, select actions according to the old joint policy and execute them, obtain rewards and the next state, and put the data into the experience pool;

[0188] (3) Randomly extract sample data from the experience pool and calculate the estimated value of the advantage function;

[0189] (4) Randomly arrange the order of the agents, calculate the loss function of the actor-critic network, and update the actor-critic network;

[0190] (5) Calculate the loss function of the critic network and update the critic network.

[0191] The state space in the DT re-association problem includes the network state information of DT nodes at time slot t - 1, the link state information between DT nodes, the association relationship between DT nodes and VNF nodes, the VNF mapping relationship at time slot t, the mapping relationship of virtual links, the remaining computing resources of DT nodes, the remaining storage resources, and the remaining bandwidth resources between DT nodes.

[0192] At time slot t, Denote the set of actions where the VNF on the VNF node is associated with the DT node, and represent the action space of the DT re-association problem as a d (t) = A(t).

[0193] The goal of the DT re-association problem is to minimize the cost composed of synchronization delay and energy consumption under constraints. Similarly, considering the penalty factor ι, define the reward function as r d (t) = -ω e E DT (t) - ω d T syn (t) + ι.

[0194] The Counterfactual Multi-Agent (COMA) algorithm solves the non-stationarity problem by using a counterfactual baseline to improve the stability in the multi-agent learning process. The algorithm contains a global critic network and multiple actor-critic networks. The global critic network is used to estimate the value of the global joint action. To avoid divergence during the training process and ensure the stability of the algorithm, a target critic network consistent with the critic network is used, and the parameters of the target critic network are updated every C steps. For agent i, it is stipulated that the actions a -i (t) of other agents remain unchanged, and the advantage function is expressed through the counterfactual baseline as:

[0195]

[0196] The counterfactual baseline judges the quality of actions by comparing the reward for executing the current action with the average reward of other actions. For agent i, the gradient of its actor-critic network is expressed as:

[0197]

[0198] The critic network of the COMA algorithm uses temporal difference for gradient update, and its loss function is expressed as:

[0199]

[0200] wherein represents the λ-return used by TD(λ), and all possible n-step weightings are used to approximate the actual reward

[0201] Figure 4 is the framework diagram of the COMA algorithm. The COMA algorithm for DT reassociation specifically includes the following steps:

[0202] (1) Initialize the parameters of the actor network, the critic network, and the target critic network, and initialize the experience replay pool;

[0203] (2) All agents obtain the observed state and take actions according to the current policy, obtain the reward and the next observed state, and store the data in the experience pool;

[0204] (3) Randomly sample from the experience pool, calculate the loss function of the critic network and update the parameters of the critic network;

[0205] (4) Update the target critic network every C steps;

[0206] (5) Each agent calculates the counterfactual baseline and the policy gradient respectively, and updates the parameters of the actor network.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A digital twin-assisted VNF migration method based on spatiotemporal large model resource demand prediction in industrial Internet of Things, characterized by: S1: Build a digital twin-assisted network slicing architecture in the industrial Internet of Things scenario, and collect node information of the physical network layer in real time through the digital twin synchronization node. The node information includes user device location, base station status, and server resource usage data; S2: Extract the spatiotemporal correlation features of the historical resource demand data of VNF nodes, use a graph convolutional network to capture spatial features, extract temporal features through a multi-head self-attention mechanism, and predict resource demand for future time slots based on a large language model; S3: Establish a system utility optimization model that integrates energy consumption, service delay, digital twin synchronization delay and load balancing, with the goal of maximizing long-term system utility. The optimization variables include the node mapping matrix F, the link mapping matrix L and the node association matrix A. S4: A Markov decision process model is constructed based on predicted resource demand, a heterogeneous multi-agent proximal policy optimization algorithm is used to make VNF migration decisions, and the reassociation strategy of digital twin nodes is optimized through a counterfactual multi-agent algorithm.

2. According to claim 1, the VNF migration method assisted by digital twins in the industrial Internet of Things based on spatiotemporal large model resource demand prediction is characterized by: In S1, the network slice architecture includes: The physical network layer is composed of base stations, MEC servers, IoT terminals and core network servers that support virtualization technology. Each physical node is connected through an optical fiber link to form a service function chain. The digital twin layer includes a digital twin basic model and a functional model. The functional model collects physical layer data through DT-VNF nodes to build a network topology and uses the data warehouse to simulate and verify the migration strategy. The node association relationship satisfies , which indicates that the VNF n′ of the physical node n1 maintains synchronous mapping with the DT node n2.

3. The VNF migration method based on spatiotemporal large model resource demand prediction assisted by digital twins in industrial Internet of Things according to claim 2 is characterized in that: The spatial feature extraction formula of the graph convolutional network is: Among them, H s represents the spatial feature matrix; represents the normalized adjacency matrix of the graph, D is the node degree matrix; A is the original adjacency matrix; l∈{0,1} represents the number of neural network layers, W l represents the weight matrix of the lth layer, X is the input feature matrix, W 1 , W 2 They are the weight matrices of the first and second layers respectively; ReLu represents the rectified linear unit activation function; softmax represents the normalized exponential function.

4. The VNF migration method based on spatiotemporal large model resource demand prediction assisted by digital twins in industrial Internet of Things according to claim 3 is characterized by: The temporal feature extraction formula of the multi-head self-attention mechanism is: Among them, H t,h represents the temporal feature matrix of the h-th attention head; denote the query, key, and value matrices of the h-th attention head, respectively, and d k is the dimension of the key vector and X is the input feature matrix.

5. The VNF migration method based on spatiotemporal large model resource demand prediction assisted by digital twins in industrial Internet of Things according to claim 1 is characterized in that: In S2, resource demand prediction specifically includes: S21: Initialize the adjacency matrix A and node feature matrix X of the network topology; S22: Sample the resource demand data of each VNF node for l historical time slots to construct an input sequence; S23: Extracting spatial dependency features between nodes through a two-layer graph convolutional network; S24: Use 8-head self-attention mechanism to calculate the attention weight of time dimension; S25: Input the spatiotemporal feature vector into the pre-trained large language model for fine-tuning, and adapt the high-dimensional features to the model input dimension through channel independence and patch encoding.

6. The VNF migration method based on spatiotemporal large model resource demand prediction assisted by digital twins in industrial Internet of Things according to claim 1 is characterized in that: The objective function of the system utility optimization model is: The optimization variables include the node mapping matrix F, the link mapping matrix L and the node association matrix A; U(t) represents the total system utility at time slot t, and T represents the time period set.

7. The method for VNF migration based on spatiotemporal large model resource demand prediction assisted by digital twin in industrial Internet of Things according to claim 1 is characterized in that: In S4, the state space of the Markov decision process includes: VNF migration phase: Physical node computing / storage resource margin, link bandwidth utilization, and predicted resource demand y i represents the predicted resource demand value of the VNF node in the next q time slots; n represents the number of VNF nodes; DT reassociation phase: DT node remaining resources, synchronization link delay status, and time-varying characteristics of the association matrix A.

8. The VNF migration method based on spatiotemporal large model resource demand prediction assisted by digital twins in industrial Internet of Things according to claim 7 is characterized in that: The heterogeneous multi-agent proximal strategy optimization algorithm adopts: Calculate the ratio of new and old strategies when updating the policy gradient M 1:i Indicates the ratio of the new and old strategies, used for strategy gradient update; represents candidate strategies; Indicates the old strategy; a i represents the action of agent i; s represents the system state; Advantage function estimation χ represents the smoothing parameter of the generalized advantage estimate; r v represents the global reward function; V φ Represents the value function of the critic network; the discount factor γ is 0.95~0.

99.

9. The method for VNF migration based on spatiotemporal large model resource demand prediction assisted by digital twin in industrial Internet of Things according to claim 1 is characterized in that: The counterfactual multi-agent algorithm comprises: Counterfactual baseline calculation: Using TD(λ) returns Update critic network; in A i represents the advantage function of agent i; Q(o(t),a(t)) represents the Q-value estimate of the global critic network for the joint action a(t); a -i (t) represents the set of actions that exclude agent i; λ represents the attenuation factor of the TD(λ) algorithm; G n (t) represents the n-step return.

10. A system for implementing the method according to any one of claims 1 to 9, characterized in that: include: The physical network layer includes a cluster of virtualized function nodes deployed on the base station side. Each node is equipped with a resource monitoring module to report CPU / memory / bandwidth usage in real time. The digital twin engine is deployed on the distributed computing nodes on the core network side, including the spatiotemporal feature extraction module, the large language model reasoning module, and the strategy simulation verification module; The migration decision controller integrates the HAPPO and COMA algorithm implementers, connects to the physical network layer through the southbound interface, and receives the QoS policy of the network slice management system through the northbound interface.

Images