A Digital Twin Function Virtualization Method for 6G Mobile Networks

Abstract

This invention relates to a digital twin function virtualization method for 6G mobile networks, comprising the following steps: constructing a core architecture for 6G digital twin function virtualization based on SDN; building heterogeneous basic digital twins in the 6G twin network layer based on mobile terminal devices and establishing communication connections between them; the digital twin function virtualization layer decouples the basic digital twins into hardware resources and twin resources, and through data identification and model structure analysis, places resources with the most recent similarity scores in the same virtual resource pool, and reconstructs the virtual digital twins to achieve virtualized management of digital twin resources; when the 6G service layer makes a service request, the digital twin function virtualization layer uses a virtualized digital twin resource orchestration algorithm to optimally orchestrate the digital twin resources to respond to dynamic 6G service requests in real time and adaptively. Compared with existing technologies, this invention has advantages such as flexible resource configuration and the ability to adaptively respond to service demands.

Description

Technical Field

[0001] This invention relates to the field of digital twin technology, and in particular to a digital twin function virtualization method for 6G mobile networks. Background Technology

[0002] As 5G mobile networks are gradually phased out, 6G mobile networks are emerging. 6G networks are expected to support many innovative services, such as autonomous driving, automated production, and the metaverse, through millisecond-level latency, data transmission rates exceeding 100GB / s, and ultra-high reliability. However, with the continued development of the Internet of Things (IoE), future mobile networks will tend to be denser, and even minor changes can lead to downtime and failures. Therefore, 6G networks require predictive maintenance and rapid fault response to maintain optimal mobile network operation. In this context, digital twin mobile networks are considered beneficial for the development and operation of 6G networks. Digital twins are considered a promising technology for realizing self-evolving complex systems. Integrating with digital twin technology, digital twin networks can provide continuous prototyping, testing, and optimization for 6G mobile networks to better support emerging 6G services. The digital twin of mobile network equipment at the edge network side can be viewed as a special resource that can be traded in the Digital Twin as a Service (DTaaS) market. The tradability of digital twin resources enables on-demand and personalized digital twin-assisted mobile services in digital twin mobile networks. However, the increased number of mobile devices and service requests in 6G presents potential challenges to the service demand response of DTMNs. First, frequent DT migrations lead to changes in the distribution of edge digital twin resources, posing challenges to digital twin resource management and orchestration. Second, due to the complexity of application scenarios, 6G mobile network services typically have random and dynamic digital twin resource demands. Optimizing global QoS and adaptively responding to massive service requests is challenging. Therefore, the performance of service response in digital twin mobile networks is hampered by the varying distribution of edge digital twin resources and unpredictable service demands.

[0003] On the other hand, Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) are still considered two promising technologies in 6G. By decoupling hardware resources from heterogeneous network devices, NFV reconstructs virtual network functions in software to facilitate network upgrades and updates. Furthermore, SDN provides a convenient network management approach by separating the network data plane and control plane. However, in digital twin mobile networks, mobile network devices on edge servers and their twins are tightly coupled through real-time synchronization. Simply virtualizing the physical network device hardware using existing NFV is insufficient. Moreover, frequent edge digital twin migrations complicate resource orchestration in digital twin mobile networks, requiring more intelligent and real-time solutions. Therefore, traditional NFV is difficult to directly apply to 6G digital twin mobile networks. Summary of the Invention

[0004] The purpose of this invention is to focus on providing flexible resource management and intelligent service response for 6G digital twin mobile networks. It offers a method for virtualizing digital twin functions for 6G mobile networks. This method involves constructing virtual digital twin functions, analyzing and aggregating dynamic and heterogeneous edge digital twin resources, and then allowing twins with similar resource levels to utilize decoupled resources to reconstruct the data synchronization, model update, and predictive control functions of the digital twin. This enhances the functionality of the basic digital twin and makes it more scalable. Simultaneously, deep reinforcement learning algorithms are used to intelligently orchestrate virtualized resources, thereby adaptively meeting dynamic service demands and optimizing the benefits of large-scale service response.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for virtualizing digital twin functions for 6G mobile networks includes the following steps:

[0007] A core architecture for 6G digital twin function virtualization based on SDN is constructed. The architecture consists of a 6G physical network layer, a 6G twin network layer, a digital twin function virtualization layer, and a 6G service layer. The digital twin function virtualization layer uses a centralized SDN controller to provide virtualization management of digital twin resources in the 6G twin network layer and respond to service requests from the 6G service layer.

[0008] Based on mobile terminal devices in the 6G physical network layer, a basic digital twin of heterogeneous mobile terminal devices is constructed in the 6G twin network layer, and a communication connection is established between the basic digital twin and the corresponding mobile terminal devices to maintain real-time synchronization between the two.

[0009] The digital twin functional virtualization layer decouples the basic digital twin into hardware resources and twin resources. Through data identification and model structure analysis, resources with the most recent similarity scores are placed in the same virtual resource pool. The virtual digital twin is reconstructed based on the resources in the virtual resource pool, thereby realizing the virtualized management of digital twin resources.

[0010] When the 6G service layer makes a service request, the digital twin function virtualization layer uses a deep reinforcement learning-based virtualized digital twin resource orchestration algorithm to optimally orchestrate the digital twin resources in order to respond to 6G service requests with dynamic needs in real time and adaptively.

[0011] The 6G twin network layer includes an edge server, and the basic digital twin is hosted on the edge server and migrates with the mobile terminal device.

[0012] The basic digital twin's functions include data synchronization, model updating, and physical control. The virtual digital twin reconstructs and extends these functions based on virtualized resources.

[0013] For the twin data synchronization function, the virtual digital twin is realized through data sharing between the associated basic digital twins. The data of each pair of basic digital twins will be synchronized to other associated basic digital twins in real time, thereby enhancing the twin data synchronization function.

[0014] The twin model update function is performed jointly by all basic digital twins associated with the virtual digital twin, and the basic digital twins share hardware resources to support model update calculations;

[0015] The twin physical control function is implemented by virtual digital twins making individual control decisions for each basic digital twin.

[0016] The virtualized digital twin resource orchestration algorithm based on deep reinforcement learning includes the following steps:

[0017] S41: Continuously capture the dynamic service needs of edge users and divide the service response process in the digital twin mobile network into several time slots T = {t1, t2, ..., t...} |T| The arrival of service requests is described using a Poisson process.

[0018] S42: In any time slot t, agents deployed on the SDN controller successively orchestrate resources for the service requests that have arrived. In the digital twin mobile network, each arriving service is assigned a virtual digital twin to respond to the request.

[0019] S43: The agent collects environmental state information from the environment. t ;

[0020] S44: The agent takes the current environment state as input according to a predetermined strategy, and selects a basic digital twin on the edge server for association for each virtual digital twin. For a specific virtual digital twin request, the agent outputs the sequence number 'a' of an edge basic digital twin each time. t The remaining resource requests of the virtual digital twin are updated until the total resources of the associated basic digital twin are greater than the resource requests of the virtual digital twin, at which point the orchestration is considered complete.

[0021] S45: After the intelligent agent makes a programmed action, it calculates the service cost and service revenue under that action, wherein the service cost includes twin data synchronization cost, twin association cost and server operation cost;

[0022] S46: The agent calculates the reward for the current action based on the service cost and service revenue.

[0023] r(s t ,a t )=w0E s -w1C syn +w2C aso +w3C opt )

[0024] Where w0, w1, w2, w3 are weights, and E s For service revenue, C syn For twin data synchronization costs, C aso For twin-related costs, C opt For server operating costs;

[0025] S47: The agent updates its policy online using the AM-PPO algorithm based on the experience (S, A, R) consisting of state, action, and reward collected within a pre-configured time interval. The objective function for policy update is expressed as:

[0026]

[0027] Where, π H (*) is the agent's policy function after adding action space mapping, A H (*) is the advantage function after adding action space mapping, F KL It is the Kullback-Leibler divergence function, θ P These are the updated policy parameters. These are the policy parameters before the update, and ω is the penalty factor;

[0028] S48: The trained agent collects new data while orchestrating digital twin mobile network resources, preparing for the next round of training and updates.

[0029] The environmental status information includes the status of edge basic digital twin resources and service resource requirements. The status of edge basic digital twin resources includes twin location, number of twin resources, and twin association status.

[0030] To prevent the agent from repeatedly selecting an already associated basic digital twin to output an illegal action, in step S44, a modulo cofactor is used to map the illegal action to the remaining set of available legal actions. For the illegal action a′ t Map it to the set of remaining legal actions A legal In the pth = a′ t mod|A legal | elements.

[0031] The twin data synchronization cost refers to the average latency C required for all virtualized digital twins to synchronize data under the current virtualized digital twin resource orchestration scheme. syn This includes the communication latency T during the virtual twin data synchronization process. com and processing delay T proc ,in,

[0032] The data synchronization communication delay of the k-th virtual twin is expressed as:

[0033]

[0034] in, This represents the communication latency between the physical devices of all basic digital twins associated with the k-th virtual digital twin and their twin objects. This represents the communication latency between all basic digital twins associated with the k-th virtual digital twin. This represents the set of basic digital twins associated with the k-th virtual digital twin. Represents edge server m j The basic set of digital twins on the surface, Indicates server m i The dataset resource size of the j-th digital twin. Represents a basic digital twin The data transfer rate between physical devices and twin objects. x represents the total dataset resources of all basic digital twins associated with the k-th virtual digital twin. i,j,k Indicator of basic digital twin Is it associated with the k-th virtual digital twin? This represents the link distance from each basic digital twin to the communication relay node. This indicates the minimum transmission bandwidth required for the service, and M represents the number of edge servers.

[0035] The data synchronization processing delay of the k-th virtual twin is expressed as:

[0036]

[0037] in, This represents the total computing resources of all basic digital twins associated with the k-th virtual digital twin. Indicates server m i The amount of computational resources required for the j-th digital twin.

[0038] The twin-related cost refers to the portion of the total resources of the virtualized digital twin that exceeds the total service demand, expressed as:

[0039]

[0040] Where, δ d and δ c These are the prices per unit of twin dataset resources and per unit of computing resources. It is a service SR k The required amount of dataset resources, It is a service SR k The amount of computing resources required.

[0041] The server operating cost is expressed as follows:

[0042]

[0043] Where |SR| represents the number of service requests in the current network, and the step function Heav(*) takes the value 1 when the parameter is positive and 0 otherwise, indicating that when the edge server does not host any basic digital twin associated with the virtual digital twin, the server is temporarily shut down to save operating costs.

[0044] The service revenue is related to the total amount of resources and bandwidth used by the service during the request period:

[0045]

[0046] Among them, e s It is service revenue, δ b It is the price per unit of bandwidth resource. It is the response time of the kth service.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] (1) This invention proposes a core architecture for 6G digital twin function virtualization based on software-defined networking (SDN), and proposes a virtualized digital twin resource orchestration algorithm based on deep reinforcement learning in this architecture. The algorithm uses a novel action space mapping near-end policy optimization (AM-PPO) method to learn the best orchestration strategy for virtualized digital twin resources in a highly dynamic digital twin mobile network, thereby responding to 6G service requests with dynamic needs in real time and adaptively, and optimizing the response benefits of large-scale 6G service requests. This enables the 6G digital twin mobile network to configure edge digital twin resources more flexibly and to respond to large-scale 6G service requests adaptively and intelligently.

[0049] (2) In order to further improve the learning speed and optimization effect of the algorithm, the present invention uses the modulo cofactor function to map the illegal action set in the action space to the legal action set during the training process, so that the algorithm can effectively shield illegal actions and converge stably.

[0050] (3) The virtual digital twin function of the present invention enhances the basic digital twin function. At the same time, since the associated basic digital twin can be arbitrarily selected, the virtual digital twin function has good flexibility and scalability.

[0051] (4) The virtualized digital twin resource orchestration algorithm of the present invention is based on online learning and orchestration of the distribution of digital twin resources in the network and specific service requirements. It can adapt to different network topologies and therefore can be deployed in any 6G digital twin network to adaptively respond to service requirements, with good compatibility. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the core architecture of the 6G digital twin function virtualization of the present invention.

[0053] Figure 2 This is a flowchart illustrating the workflow of the virtualized digital twin resource orchestration algorithm of this invention.

[0054] Figure 3 This is a comparison diagram of the training process of the present invention and other methods in one embodiment.

[0055] Figure 4 This is a comparison chart of the response benefits of the present invention and other methods under different service request scales in one embodiment.

[0056] Figure 5 This is a comparison chart of the response benefits of the present invention and other methods under different network resource scales in one embodiment. Detailed Implementation

[0057] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0058] With the continuous evolution of 6G mobile networks, digital twin technology is considered to have advantages in optimizing 6G network performance, gradually forming the new concept of "digital twin mobile network." To bridge the physical and digital spaces and provide real-time simulation, prediction, and control for 6G networks, the integration of digital twins and 6G mobile networks has become a global hot topic in the field. First, due to the frequent migration of edge digital twins and the highly dynamic nature of 6G service demands, how to flexibly configure digital twin resources and adaptively respond to service demands in a digital twin mobile network is a challenge. Second, optimizing the benefits of global service response under large-scale service requests is another challenge.

[0059] Therefore, this invention focuses on providing flexible resource management and intelligent service response benefit optimization for 6G digital twin mobile networks. First, a core architecture for 6G digital twin function virtualization based on Software-Defined Networking (SDN) is proposed. Then, within this architecture, a virtualized digital twin resource orchestration algorithm based on deep reinforcement learning is proposed. This algorithm uses a novel Action Space Mapping Proximal Policy Optimization (AM-PPO) method to learn the optimal orchestration strategy for virtualized digital twin resources in a highly dynamic digital twin mobile network, thereby enabling real-time and adaptive responses to dynamically demanding 6G service requests and optimizing the response benefits for large-scale 6G service requests. With the support of this invention, 6G digital twin mobile networks can more flexibly configure edge digital twin resources and adaptively and intelligently respond to large-scale 6G service requests.

[0060] Specifically, this embodiment provides a digital twin function virtualization method for 6G mobile networks, including the following steps:

[0061] S1: Construct a core architecture for 6G digital twin functional virtualization based on SDN, such as Figure 1 As shown.

[0062] In this embodiment, the architecture consists of a 6G physical network layer, a 6G twin network layer, a digital twin function virtualization layer, and a 6G service layer.

[0063] (1) 6G physical network layer

[0064] The 6G physical network layer showcases terminal devices within the 6G physical network, including vehicles, drones, and industrial robots. These mobile terminal devices continuously communicate with twins hosted on edge servers to maintain real-time synchronization.

[0065] (2) 6G twin network layer

[0066] The 6G twin network layer mainly consists of distributed edge servers and basic digital twins of heterogeneous mobile terminal devices. To maintain a seamless digital twin connection with the mobile terminal devices, the basic digital twins at the edge need to move in real time with the mobile terminal devices.

[0067] (3) Digital Twin Function Virtualization Layer

[0068] The digital twin functionality virtualization layer uses a centralized SDN controller to provide virtualization management of digital twin resources and respond to service requests in the digital twin mobile network. Heterogeneous device digital twins in the twin network layer are first decoupled into hardware resources (such as computing and storage resources) and twin resources (such as model resources and dataset resources). Through data identification and model structure analysis, resources with the most recent similarity scores are placed in the same virtual resource pool, and then virtual digital twins can be reconstructed based on these virtual resources. The virtual digital twin reconstructs and extends the functionality of the basic digital twin based on virtualized resources. Since the associated basic digital twin can be arbitrarily selected, the virtual digital twin functionality has good flexibility and scalability. Simultaneously, to intelligently orchestrate virtualized digital twin resources to respond to large-scale dynamic service requests, this embodiment deploys a deep reinforcement learning algorithm based on Action Space Mapping Proximal Policy Optimization (AM-PPO) in the digital twin functionality virtualization layer for twin resource orchestration, achieving response benefit optimization.

[0069] (4) 6G service layer

[0070] The 6G service layer includes emerging 6G services such as autonomous driving, unmanned factories, and the metaverse. These applications, due to their complex application scenarios, have dynamic and real-time requirements for digital twin mobile network resources.

[0071] S2: Based on mobile terminal devices in the 6G physical network layer, construct basic digital twins of heterogeneous mobile terminal devices in the 6G twin network layer, and establish communication connections between the basic digital twins and the corresponding mobile terminal devices to maintain real-time synchronization between the two.

[0072] S3: The digital twin function virtualization layer decouples the basic digital twin into hardware resources and twin resources. Through data identification and model structure analysis, resources with the most recent similarity scores are placed in the same virtual resource pool. The virtual digital twin is reconstructed based on the resources in the virtual resource pool, thereby realizing the virtualized management of digital twin resources.

[0073] The basic functions of a digital twin include data synchronization, model updates, and physical control. A virtual digital twin, based on virtualized resources, reconstructs and extends these functions.

[0074] For the twin data synchronization function, the virtual digital twins achieve data sharing between the associated basic digital twins. The data of each pair of basic digital twins will be synchronized to other associated basic digital twins in real time, thereby enhancing the twin data synchronization function.

[0075] The twin model update function is performed jointly by all the basic digital twins associated with the virtual digital twin, and the basic digital twins share hardware resources to support the model update calculation.

[0076] The twin physical control function is implemented by virtual digital twins making individual control decisions for each basic digital twin.

[0077] Virtual digital twin functionality enhances basic digital twin functionality, and because the associated basic digital twin can be arbitrarily selected, it offers excellent flexibility and scalability. At the 6G mobile network service layer, emerging 6G services such as autonomous driving, unmanned factories, and metaverse were showcased. These applications, due to their complex application scenarios, have dynamic and real-time requirements for digital twin mobile network resources.

[0078] S4: When the 6G service layer makes a service request, the digital twin function virtualization layer uses a deep reinforcement learning-based virtualized digital twin resource orchestration algorithm to optimally orchestrate the digital twin resources in order to respond to 6G service requests with dynamic needs in real time and adaptively.

[0079] This embodiment employs a virtualized resource orchestration algorithm based on AM-PPO. Traditional PPO algorithms do not consider shielding against illegal actions, meaning the agent can repeatedly select illegal actions as output. However, in the proposed digital twin functional virtualization architecture, the same basic digital twin cannot be associated with multiple virtual digital twins simultaneously, as different virtual digital twins may issue conflicting control commands to the basic digital twin. Therefore, during virtual digital twin resource orchestration, the agent should avoid selecting already associated basic digital twins. While penalties for selecting illegal actions can be added during training with the traditional PPO algorithm, this suffers from difficulties in parameter control and slow training. Therefore, in this method, based on training with the traditional PPO algorithm, a modulo cofactor is used to map the output illegal action values ​​to the remaining legal action space, accelerating algorithm convergence and improving the effectiveness of the orchestration algorithm.

[0080] Specifically, such as Figure 2 As shown, the virtualization resource orchestration algorithm includes the following steps:

[0081] S41: Continuously capture the dynamic service needs of edge users and divide the service response process in the digital twin mobile network into several time slots T = {t1, t2, ..., t...} |T| The arrival of the service request is described using a Poisson process.

[0082] S42: In any time slot t, agents deployed on the SDN controller successively orchestrate resources for the service requests that have arrived. In the digital twin mobile network, each arriving service is assigned a virtual digital twin to respond to the request.

[0083] S43: The agent collects environmental state information from the environment. t s t This includes the status of basic digital twin resources at the edge (twin location, number of twin resources, twin association status) and service resource requirements.

[0084] S44: The agent takes the current environment state as input according to a predetermined strategy, and selects a basic digital twin on the edge server for association for each virtual digital twin. For a specific virtual digital twin request, the agent outputs the sequence number 'a' of an edge basic digital twin each time. t The remaining resource requests of the virtual digital twin are updated until the total resources of the associated basic digital twin exceed the resource requests of the virtual digital twin, at which point the orchestration is considered complete.

[0085] To prevent the agent from repeatedly selecting an already associated basic digital twin to output an illegal action, a modulo cofactor is used to map the illegal action to the remaining set of legal actions. For the illegal action a′ t Map it to the set of remaining legal actions A legal In the pth = a′ t mod|A legal | elements.

[0086] S45: After the agent makes a programmed action, it calculates the service cost and service revenue under that action.

[0087] In this embodiment, the service cost includes twin data synchronization cost, twin association cost, and server operation cost.

[0088] a) Cost of twin data synchronization

[0089] Twin data synchronization cost refers to the average latency C required for data synchronization among all virtualized digital twins under the current virtualized digital twin resource orchestration scheme. synThis includes the communication latency T during the virtual twin data synchronization process. com and processing delay T proc ,in,

[0090] The data synchronization communication delay of the k-th virtual twin is expressed as:

[0091]

[0092] in, This represents the communication latency between the physical devices of all basic digital twins associated with the k-th virtual digital twin and their twin objects. This represents the communication latency between all basic digital twins associated with the k-th virtual digital twin. This represents the set of basic digital twins associated with the k-th virtual digital twin. Represents edge server m j The basic set of digital twins on the surface, Indicates server m i The dataset resource size of the j-th digital twin. Represents a basic digital twin The data transfer rate between physical devices and twin objects. x represents the total dataset resources of all basic digital twins associated with the k-th virtual digital twin. i,j,k Indicator of basic digital twin Is it associated with the k-th virtual digital twin? This represents the link distance from each basic digital twin to the communication relay node. This indicates the minimum transmission bandwidth required for the service, and M represents the number of edge servers.

[0093] The data synchronization processing delay of the k-th virtual twin is expressed as:

[0094]

[0095] in, This represents the total computing resources of all basic digital twins associated with the k-th virtual digital twin. Indicates server m i The amount of computational resources required for the j-th digital twin.

[0096] b) Twin-related costs

[0097] Twin-related costs refer to the portion of the total resources of a virtualized digital twin that exceeds the total service demand, expressed as:

[0098]

[0099] Where, δ d and δ c These are the prices per unit of twin dataset resources and per unit of computing resources. It is a service SR k The required amount of dataset resources, It is a service SR k The amount of computing resources required.

[0100] c) Server operating costs

[0101] Server operating costs are expressed as follows:

[0102]

[0103] Where |SR| represents the number of service requests in the current network, and the step function Heav(*) takes the value 1 when the parameter is positive and 0 otherwise, indicating that when the edge server does not host any basic digital twin associated with the virtual digital twin, the server is temporarily shut down to save operating costs.

[0104] d) Service revenue

[0105] Service revenue is related to the total amount of resources and bandwidth used by the service during the request period:

[0106]

[0107] Among them, E s It is service revenue, δ b It is the price per unit of bandwidth resource. It is the response time of the kth service.

[0108] S46: The agent calculates the reward for the current action based on the service cost and service revenue.

[0109] r(s t , a t )=w0E s -(w1C syn +w2C aso +w3C opt )

[0110] Where w0, w1, w2, w3 are weights, and E s For service revenue, C syn For twin data synchronization costs, C aso For twin-related costs, C opt For server operating costs.

[0111] S47: The agent updates its policy online using the AM-PPO algorithm based on the experience (S, A, R) consisting of state, action, and reward collected within a pre-configured time interval. The objective function for policy update is expressed as:

[0112]

[0113] Where, π H (*) is the agent's policy function after adding action space mapping, A H (*) is the advantage function after adding action space mapping, F KL It is the Kullback-Leibler divergence function, θ P These are the updated policy parameters. These are the policy parameters before the update, and ω is the penalty factor.

[0114] S48: The trained agent collects new data while orchestrating digital twin mobile network resources, preparing for the next round of training and updates.

[0115] Based on the above method, this embodiment establishes an experimental environment, which mainly consists of two parts. The first step is to generate a randomized digital twin mobile network simulation environment, including generating the network topology and the distribution of digital twin resources within the network. The second step is to simulate a deep reinforcement learning agent running the AM-PPO policy on an SDN controller. The agent is then allowed to continuously interact and learn from the environment, ultimately training a converged service response benefit optimization strategy. This strategy can effectively orchestrate virtualized digital twin resources and optimize the benefits of large-scale service responses.

[0116] (I) Establishing a digital twin mobile network simulation environment

[0117] This paper uses Python programming to build a digital twin mobile network simulation environment. The topology of the digital twin mobile network is randomly generated. First, an edge digital twin server node is established, which includes several digital twin objects and a total available bandwidth. Each digital twin object has a certain amount of computing resources (hardware resources) and dataset resources (twin resources). However, simply establishing one server node is insufficient; multiple nodes are needed to form the digital twin mobile network. Only when the network topology is formed and data transmission occurs between nodes can the operation of a real network be simulated. To meet the experimental requirements, the total number of edge server nodes, the connection rate between nodes, the maximum link latency, the maximum number of digital twins on the server, and the resource quantity range for each digital twin are specified. Then, the connection topology between the edge servers is randomly generated.

[0118] Furthermore, it is necessary to simulate service requests arriving in the digital twin mobile network. To this end, this embodiment employs a round-robin structure, dividing the time interval of each round into several time slots. Assuming that service requests follow a Poisson distribution, service arrival rate and service rate parameters are set, and then a service request flow following a Poisson distribution is randomly generated within the considered time interval.

[0119] (II) Training a deep reinforcement learning agent using the AM-PPO strategy

[0120] After running the digital twin mobile network simulation environment, an agent can interact with the environment to learn the optimal virtualization resource orchestration strategy. In this embodiment, the agent's learning strategy is set to AM-PPO, and then trained to achieve the desired effect. Figure 3 This shows a comparison of the reward function using the AM-PPO algorithm, the original PPO algorithm, and another deep reinforcement learning algorithm, DQN. According to... Figure 3 It can be observed that after adopting the improved method of action space mapping, the learning effect is more stable and converges to a higher reward value.

[0121] After training is complete, the trained agent using the AM-PPO algorithm is deployed on SDN nodes in the digital twin mobile network. Simulation experiments are used to test the orchestration effect of the AM-PPO algorithm. This embodiment tests the service response performance of different algorithms by changing the scale of digital twin resources and service requests in the digital twin mobile network environment.

[0122] exist Figure 4 The experiment demonstrates the effectiveness of the AM-PPO algorithm on agents with varying numbers of service requests. This embodiment underwent several tests on the same network topology. For comparison, this embodiment also included another reinforcement learning algorithm, DQN, and two other classic heuristic resource orchestration algorithms, FirstFit and BestFit. As the number of service requests increases, the AM-PPO algorithm employed in this invention maintains relative stability compared to the baseline scheme.

[0123] To further test the effectiveness of the AM-PPO orchestration algorithm, this embodiment was tested under different network digital twin resource scales. Figure 5 The experimental results are shown. From Figure 5 Two main trends can be observed:

[0124] Trend 1: As the scale of network digital twin resources expands, the response benefits gradually increase. This is in line with expectations, because as the scale of network digital twin resources increases, the action space for agents to orchestrate also increases, meaning that the probability of agents finding the optimal orchestration method also increases. Therefore, compared to a smaller resource scale, the response benefits obtained by agents gradually increase.

[0125] Trend Two: As the scale of network digital twin resources expands, the AM-PPO algorithm maintains a stable advantage over other algorithms. Although the scale of network resources increases, the AM-PPO-based resource orchestration algorithm of this invention can still learn the optimal orchestration method well. The expansion of network resource scale has little negative impact on this invention, thus proving that this invention remains effective for larger networks. This means that the AM-PPO algorithm can be deployed in larger-scale digital twin mobile networks, supporting large-scale resource orchestration and service response.

[0126] In summary, the proposed digital twin function virtualization method for 6G mobile networks, within a simulated digital twin mobile network environment, utilizes an agent based on the AM-PPO learning algorithm to continuously interact with the environment. Based on this interaction experience, the agent learns the optimal virtual digital twin resource orchestration strategy. The trained reinforcement learning agent can then achieve adaptive responses to large-scale digital twin resource distribution and service requests. Experimental results demonstrate that the expected results have been achieved.

[0127] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for digital twin function virtualization for 6G mobile networks, characterized in that, Includes the following steps: A core architecture for 6G digital twin function virtualization based on SDN is constructed. The architecture consists of a 6G physical network layer, a 6G twin network layer, a digital twin function virtualization layer, and a 6G service layer. The digital twin function virtualization layer uses a centralized SDN controller to provide virtualization management of digital twin resources in the 6G twin network layer and respond to service requests from the 6G service layer. Based on mobile terminal devices in the 6G physical network layer, a basic digital twin of heterogeneous mobile terminal devices is constructed in the 6G twin network layer, and a communication connection is established between the basic digital twin and the corresponding mobile terminal devices to maintain real-time synchronization between the two. The digital twin function virtualization layer decouples the basic digital twin into hardware resources and twin resources. Through data identification and model structure analysis, resources with the most recent similarity scores are placed in the same virtual resource pool. The virtual digital twin is reconstructed based on the resources in the virtual resource pool, thereby realizing the virtualized management of digital twin resources. When the 6G service layer makes a service request, the digital twin function virtualization layer uses a deep reinforcement learning-based virtualized digital twin resource orchestration algorithm to optimally orchestrate the digital twin resources in order to respond to 6G service requests with dynamic needs in real time and adaptively. The virtualized digital twin resource orchestration algorithm based on deep reinforcement learning includes the following steps: S41: Continuously capture the dynamic service needs of edge users and divide the service response process in the digital twin mobile network into several time slots. And use a Poisson process to describe the arrival of service requests; S42: In any time slot t The agents deployed on the SDN controller successively orchestrate resources for the service requests that have arrived. In the digital twin mobile network, each arriving service is assigned a virtual digital twin to respond to the request. S43: The agent collects environmental state information from the environment. ; S44: The agent takes the current environment state as input according to a predetermined strategy, selects a basic digital twin on the edge server for association for each virtual digital twin, and outputs the sequence number of a basic edge digital twin for each virtual digital twin request. The remaining resource requests of the virtual digital twin are updated until the total resources of the associated basic digital twin are greater than the resource requests of the virtual digital twin, at which point the orchestration is considered complete. S45: After the intelligent agent makes a programmed action, it calculates the service cost and service revenue under that action, wherein the service cost includes twin data synchronization cost, twin association cost and server operation cost; S46: The agent calculates the reward for the current action based on the service cost and service revenue. in, As weight, For service revenue, Cost of twin data synchronization For twin-related costs, For server operating costs; S47: The agent uses experience consisting of states, actions, and rewards collected within a pre-configured time interval. The AM-PPO algorithm is used for online policy updates. The objective function for policy updates is expressed as: in, It is the agent's policy function after adding action space mapping. It is the advantage function after adding action space mapping. It is the Kullback-Leibler divergence function. These are the updated policy parameters. These are the policy parameters before the update. It is a punishment factor; S48: The trained agent collects new data while orchestrating digital twin mobile network resources, preparing for the next round of training and updates.

2. The digital twin function virtualization method for 6G mobile networks according to claim 1, characterized in that, The 6G twin network layer includes an edge server, and the basic digital twin is hosted on the edge server and migrates with the mobile terminal device.

3. The digital twin function virtualization method for 6G mobile networks according to claim 1, characterized in that, The basic digital twin's functions include data synchronization, model updating, and physical control. The virtual digital twin reconstructs and extends these functions based on virtualized resources. For the twin data synchronization function, the virtual digital twin is realized through data sharing between the associated basic digital twins. The data of each pair of basic digital twins will be synchronized to other associated basic digital twins in real time, thereby enhancing the twin data synchronization function. The twin model update function is performed jointly by all basic digital twins associated with the virtual digital twin, and the basic digital twins share hardware resources to support model update calculations; The twin physical control function is implemented by virtual digital twins making individual control decisions for each basic digital twin.

4. The digital twin function virtualization method for 6G mobile networks according to claim 1, characterized in that, The environmental status information includes the status of edge basic digital twin resources and service resource requirements. The status of edge basic digital twin resources includes twin location, number of twin resources, and twin association status.

5. A digital twin function virtualization method for 6G mobile networks according to claim 1, characterized in that, To prevent the agent from repeatedly selecting an already associated basic digital twin to output an illegal action, in step S44, a modulo cofactor is used to map the illegal action to the remaining set of available legal actions. Map it to the set of remaining legal actions. The first in Each element.

6. A digital twin function virtualization method for 6G mobile networks according to claim 1, characterized in that, The twin data synchronization cost refers to the average latency required for all virtualized digital twins to synchronize data under the current virtualized digital twin resource orchestration scheme. This includes communication latency during the virtual twin data synchronization process. and processing latency ,in, No. k The data synchronization communication delay of a virtual twin is expressed as: in, Indicates the first k The communication latency between the physical devices of all basic digital twins associated with a virtual digital twin and their twin objects. Indicates the first k Communication latency between all basic digital twins associated with a virtual digital twin, Indicates the first k A set of basic digital twins associated with a virtual digital twin. Represents edge server The basic set of digital twins on the surface, Indicates server Upper j The amount of data resources for a digital twin dataset. Represents a basic digital twin The data transfer rate between physical devices and twin objects, Indicates all those that are the first k The total dataset resources of all basic digital twins associated with a virtual digital twin. Indicator of basic digital twin Is it related to the first k A virtual digital twin is associated with it. This represents the link distance from each basic digital twin to the communication relay node. Indicates the minimum transmission bandwidth required for the service. M Indicates the number of edge servers; No. k The latency for data synchronization processing of a virtual twin is expressed as: in, Indicates all those that are the first k The total computing resources of all basic digital twins associated with a virtual digital twin. Indicates server Upper j The amount of computing resources required for a digital twin.

7. A digital twin function virtualization method for 6G mobile networks according to claim 6, characterized in that, The twin-related cost refers to the portion of the total resources of the virtualized digital twin that exceeds the total service demand, expressed as: in, and These are the prices per unit of twin dataset resources and per unit of computing resources. It is a service sr k The required amount of dataset resources, It is a service sr k The amount of computing resources required.

8. A digital twin function virtualization method for 6G mobile networks according to claim 6, characterized in that, The server operating cost is expressed as follows: in, The step function represents the number of service requests in the current network. The parameter is set to 1 if it is positive and 0 otherwise, indicating that the server is temporarily shut down to save operating costs when no basic digital twin associated with the virtual digital twin is hosted on the edge server.

9. A digital twin function virtualization method for 6G mobile networks according to claim 7, characterized in that, The service revenue is related to the total amount of resources and bandwidth used by the service during the request period: in, It is service revenue. It is the price per unit of bandwidth resource. It is the first k The response time of each service.

Images