Virtualized digital twin traffic resource scheduling model based on cloud edge collaboration

Abstract

The invention discloses a virtual digital twinborn traffic resource scheduling model based on cloud edge collaboration, and aims to further improve the resource operation efficiency of a traffic system. According to the model, on the basis of fully utilizing the characteristics of a digital twinning technology, efficient operation of resources is realized in a virtualized resource mode through a cloud edge collaborative architecture so as to adapt to on-demand scheduling of various traffic complex scenes. Besides, in order to further optimize scheduling, the invention provides a value distribution strategy gradient map deep reinforcement learning model based on a node importance strategy, and efficient collaboration and scheduling of digital cloud edge comprehensive traffic resources are realized through deep learning of key nodes and relationships and adaptive feedback of scheduling requirements. Experimental results show that the model provided by the invention is superior to a current mainstream resource scheduling model in multiple aspects.

Description

Technical field:

[0001] The present invention relates to the field of traffic resource scheduling, and in particular to a virtualized digital twin traffic resource scheduling model based on cloud-edge collaboration. The model integrates a node importance strategy graph neural network and a value distribution reinforcement learning algorithm to achieve efficient resource optimization scheduling. Background technology:

[0002] In the era of rapid development of communication technology, a large number of intensive computing service programs have gradually emerged in the transportation network, including natural language processing, intelligent navigation, autonomous driving and other fields. The resource consumption of these applications and the demand for data processing, communication and processing are constantly increasing. At the same time, with the continuous evolution of various computing environments such as cloud computing, edge computing, and the Internet of Things, different computing architectures and resource forms are constantly merging, making the complexity of resource scheduling in the transportation system increasingly complex. The current intelligent transportation system needs to process a large amount of real-time data, and needs to coordinate the data flow and resource allocation between different systems to achieve overall collaborative work. However, the computing resources of the vehicle itself are relatively limited, and a large amount of resources will be occupied when performing complex computing tasks, thereby increasing the demand for resource scheduling. Therefore, it is necessary to build a model architecture that can integrate multiple resources and achieve efficient scheduling to improve the management level of the intelligent transportation system.

[0003] The emergence of digital twin technology provides new technical support and development direction for traffic resource scheduling. Digital twins can build a bridge between the physical world and the real world. By collecting and integrating data from various sensors, combined with geographic information systems (GIS) and big data analysis tools, a digital model that matches the actual traffic system can be built in the virtual space. This model can not only reflect the current state of the physical system, but also predict possible future conditions, thereby providing support for further scheduling decisions. It creates a high-fidelity virtual environment through a full-scale simulation of the actual traffic network, achieving seamless connection between physical space and digital space. Using the data generated by digital twin technology, more accurate prediction models and decision support tools can be trained. These tools can repeatedly test different scheduling strategies in a simulated environment to find the optimal solution, thereby guiding actual operations.

[0004] As the two core supporting components in the digital twin system, cloud computing and edge computing provide powerful computing power and low-latency service response respectively. How to collaboratively manage and schedule network resources between the cloud and the edge has become a hot topic for researchers. On the one hand, cloud computing centers have abundant computing and storage resources, which effectively solves the problem of limited vehicle computing resources. On the other hand, edge computing distributes part of the data processing tasks such as computing, communication, and caching to edge devices close to the data source, thereby alleviating the computing pressure on the vehicle side, reducing data transmission delays, and ensuring low latency and high throughput. Due to the unavoidable delay caused by long-distance task transmission, traffic application scenarios with high real-time requirements, such as emergency obstacle avoidance in autonomous driving and real-time road condition analysis, may be seriously affected. At the same time, the use of edge computing resources in smart transportation systems is limited, and it is difficult to handle complex tasks.

[0005] In order to further optimize resource scheduling, researchers have conducted in-depth explorations in the field of smart transportation and introduced a series of intelligent scheduling algorithms. Most of these algorithms are based on heuristic rules, machine learning, or optimization theory, and are designed to intelligently decide the allocation and execution location of tasks based on the urgency of the task, computational complexity, network bandwidth conditions, and energy consumption characteristics of edge devices and cloud servers. Some algorithms prioritize tasks with high real-time requirements to be processed by nearby edge nodes to reduce transmission delays; while computationally intensive tasks may be scheduled to cloud servers for parallel processing to take advantage of the powerful computing power of the cloud.

[0006] Although these methods have improved the overall efficiency and response speed of the system and reduced energy consumption to a certain extent, they still face some limitations. First, traditional resource scheduling algorithms often rely on pre-designed rules or models, which are difficult to accurately capture the complex and changeable dynamic environment in the transportation system and have insufficient response capabilities to changes in traffic resources, which may lead to non-optimal scheduling decisions. Secondly, the compatibility and interoperability between different methods are poor, making it difficult to achieve unified scheduling across systems and platforms. In addition, with the continuous expansion and complexity of the transportation system, the computational complexity and resource consumption of the algorithm are also increasing, which puts higher requirements on the scalability and sustainability of the system. Summary of the invention:

[0007] A new resource scheduling framework is proposed - a virtualized scheduling model of cloud-edge collaborative resources for smart transportation based on deep reinforcement learning of graphs. The model fully integrates the resource characteristics of the cloud, edge, and end to form a comprehensive resource optimization architecture, improves the scalability of the system, and realizes real-time simulation and monitoring of the physical transportation system through virtualized digital twin technology. At the same time, by combining the deep representation ability of graph neural networks and the decision-making and scheduling capabilities of reinforcement learning, it can more accurately capture the dynamic changes and complex relationships of the transportation system, thereby making more accurate and flexible scheduling decisions. The use of virtualized resource organization and management mode improves the flexibility and compatibility of resource management, making resource scheduling more efficient and intelligent. Through experimental verification, the model has demonstrated good performance in practical applications, significantly outperforming other existing scheduling models. Specifically, it includes the following steps:

[0008] Step 1: Use cloud-edge collaborative architecture to build a digital twin system for the transportation system, which takes advantage of the powerful computing power of the cloud and the low latency characteristics of the edge. Through global optimization in the cloud and real-time processing at the edge, effective allocation and utilization of resources are achieved. At the same time, virtualization technology is introduced into the digital twin system to enhance the flexibility and scalability of the system, making resource scheduling more efficient and intelligent.

[0009] Step 2: Model the network resource environment. First, define the state space s. A virtual model that reflects the state of the physical system in real time is established in the digital twin system. The state space s includes the cloud resource usage, edge resource usage, virtual network request queue, and embedded virtual network status. This information is collected and updated in real time by the digital twin system to ensure that scheduling decisions are based on the latest and accurate data. Then define the action space a of the agent, and finally design the reward function r.

[0010] Step 3: The present invention designs and proposes a key node graph neural network SIGNet, which is improved on the basis of the graph neural network (NiGCN) model based on node importance. In addition to considering node features, the original model also adds a node importance strategy in model training to explore more information related to node importance from the graph structure. Since the degree centrality of NiGCN cannot fully reflect the impact of topological structure on node importance, resulting in low accuracy of calculation results, this patent improves it into a semi-local centrality measurement method, which evaluates node importance by considering the direct neighbors and second-order neighbors of the node, and its calculation accuracy is improved to a certain extent compared with the original model. By using a key node graph neural network to process the nodes in the network and their interconnections, each node represents a physical entity in the digital twin system, and the link represents the connection relationship and communication path between these entities. This patent introduces a node importance strategy in the algorithm, which assigns different weights to nodes according to factors such as the node's computing resources, storage resources, and real-time requirements of processing tasks. In this way, the algorithm can give priority to those nodes that are critical to global stability and performance during resource scheduling.

[0011] Step 4: The output of the graph neural network with key node features is used as the environmental input of the reinforcement learning scheduling algorithm. At the same time, a value distribution policy gradient reinforcement learning algorithm DSORL based on the DSAC-T algorithm is designed for the uncertainty of virtualized network resource allocation in the cloud-edge collaborative framework and the dynamic changes in resource availability. DSAC-T is an advanced reinforcement learning algorithm that has good performance in control tasks in continuous action space, especially when dealing with complex environments containing high-dimensional states and actions. It incorporates the maximum entropy principle and aims to find strategies that are both high-return and highly uncertain, so as to promote exploration and avoid falling into local optimal solutions. The Q function in DSAC-T is updated using random target returns. This randomness will introduce additional variance into the update process, making the learning process unstable. Therefore, the present invention uses the Bellman equation to update the Q function and V function, and adopts an offline policy learning method, which means that it can flexibly use the data collected according to any strategy for learning, greatly improving the data utilization efficiency and the robustness of the algorithm. By maintaining an experience replay pool and continuously sampling historical transformations from it for learning, this not only reduces the correlation of the data, but also allows the algorithm to gradually learn long-term causal relationships, thereby performing better in dynamically changing environments.

[0012] Compared with the existing models, the present invention has the following advantages:

[0013] 1. Aiming at the problem of intelligent transportation resource optimization, a cloud-edge collaborative virtualized resource scheduling model enabled by digital twins is proposed. On the premise of ensuring the effective allocation of multi-layer resources of cloud, edge and end in digital form, a unified resource view is constructed from a global perspective by means of virtualization technology, which is compatible with resource differences of different levels, dimensions, structures, etc., and improves the efficiency of resource representation.

[0014] 2. A Significant Nodes Graph Neural Network (SIGNet) is proposed to learn complex traffic resource structures. The semi-local centrality measurement method is used to improve the model's ability to understand node relationships in different resource configurations and structural levels, as well as its ability to mine resource relationships in real-time traffic scheduling, providing structured semantic support for comprehensive resource scheduling.

[0015] 3. An improved value distribution strategy gradient reinforcement learning algorithm DSORL (Distributed Strategy Optimization with Reinforcement Learning) is proposed. By adopting the value distribution strategy iteration framework of the Bellman equation, the over-estimation problem is effectively suppressed to make the scheduling strategy more accurate. It is combined with multi-dimensional joint optimization of the cloud-edge to achieve real-time and efficient scheduling of complex transportation resource systems. Description of the drawings:

[0016] Figure 1 Implementation process of the present invention;

[0017] Figure 2 This is the framework diagram of the intelligent transportation digital twin cloud-edge system;

[0018] Figure 3 It is a diagram of the virtualization resource scheduling process;

[0019] Figure 4 This is a diagram of the graph reinforcement learning algorithm framework;

[0020] Figure 5 This is a comparison chart of simulation results. Specific implementation method:

[0021] The content of the present invention will be further described below in conjunction with the accompanying drawings and simulation examples.

[0022] Reference Figure 1 The implementation flowchart of the invention scheme shown provides a detailed description of the implementation steps.

[0023] like Figure 2The structure of the digital twin system using cloud-edge collaboration in intelligent transportation is shown. It consists of three parts: the physical entity layer, the edge twin layer, and the cloud twin layer. The physical entity layer is mainly composed of various vehicles, traffic lights, cameras, roadside units (RSUs), and electronic devices in the hands of pedestrians. The main feature of this layer is that the computing and storage resources are relatively limited. The edge twin layer is composed of edge servers containing multiple computing and storage devices. This layer can calculate, store, and transmit various information in the physical entity layer. The cloud twin layer is composed of large-scale cloud servers, which manage all edge service devices and perform large-scale computing and storage services. The entire digital twin system shows a hierarchical structure of resource allocation, with the cloud processing complex large-scale tasks and the edge nodes responsible for real-time local processing. The data flow between vehicles, sensors, edge nodes, and the cloud ensures that the system can make timely information-based decisions.

[0024] The cloud server is located at the outermost layer of the three-layer structure. As the central point, it is responsible for complex data processing, long-term storage and high-level decision-making tasks. It also has three edge nodes, which are located in the second edge twin layer and distributed in the city. The edge nodes are responsible for real-time data processing near the data source. They can reduce the burden on the cloud, reduce latency and increase decision-making speed. The traffic lights, cameras and sensors in the figure are located in the physical entity layer and are responsible for collecting real-time traffic data, including images, vehicle counts and environmental conditions. They are the main data source of the system. Vehicles equipped with OBU (on-board unit) can communicate with roadside units (RSU) and handheld devices of other vehicles and pedestrians.

[0025] The process of virtualization resource scheduling is as follows: Figure 3 As shown in the figure, the physical entity layer, edge nodes, and cloud server resources are first virtualized into a network resource topology diagram, where the darker nodes represent virtualized cloud nodes with more computing resources; and the lighter edges are edge nodes and devices in the physical entity layer. The edges in the figure represent physical links, showing the connection relationship between nodes. Each edge represents a physical link between two nodes, transmitting data or virtualizing network resource requests. The color of the edge represents the link resource capacity. The darker the color, the higher the resource capacity of the link. On the right is the color bar of the nodes and edges, showing the resource capacity range.

[0026] An embedding process of virtualized resources is as follows Figure 2As shown in the lower part, there are two virtualized network resource requests waiting to be embedded in a physical entity network resource. The numbers next to the nodes and links in the virtualized network represent the demand for resources, while the numbers next to the nodes and links in the physical network represent the remaining amount of resources. The left side shows the status of the three networks before embedding, and the right side shows the status after the virtual network is successfully embedded in the physical network. After the virtualized network in the digital twin system is embedded, it can be fed back to the physical entity layer to update the status of the entire system.

[0027] The state of all entity network resources in the three-layer architecture is uniformly represented as a weighted undirected graph Where N P It represents the node set composed of all devices in all physical entity layers. P It represents the set of links connecting various devices. Represents a collection of properties of a physical network device. Represents the attribute set of the physical link, where sumN and sumE represent the total number of physical network nodes and physical links respectively. This experiment will use computing resources and bandwidth resources as the attributes of physical network nodes and physical link nodes respectively.

[0028] Similarly, the virtual network is modeled as a weighted undirected graph in the digital twin system. Where N V It represents the set of all virtual network nodes, E V It represents the set of all virtual links. It represents the properties of the virtual network node. It represents the properties of the virtual network link, where sumn and sume represent the total number of virtual network nodes and virtual links respectively.

[0029] The graph deep reinforcement learning framework designed by the present invention is as follows Figure 2 As shown. In the graph neural network module, the information of nodes in the network can be effectively transmitted and integrated through the convolution operation between nodes. The nodes in the cloud, edge, and physical entity are established as a graph structure as the input of the graph convolution layer. The basic layer of the graph neural network can be expressed by the following formula:

[0030]

[0031] Among them, H (l) is the node feature matrix of the lth layer, each row represents the feature vector of a node; H (l+1) is the feature matrix after the graph convolution layer transformation; W (l) is the learnable weight matrix of this layer, used for feature transformation; σ is the nonlinear activation function; It is the adjacency matrix A plus the identity matrix I to take into account the characteristics of the node itself; is the degree matrix, where Used for normalization to ensure smooth transfer of information; It is called the symmetric normalized adjacency matrix, which is used to consider the influence of node degree when aggregating neighbor information and prevent nodes with high degrees from excessively dominating information propagation.

[0032] In the digital twin system, there are some abnormal nodes or noisy data, and the traditional graph convolutional neural network will be affected to a certain extent when processing these states. Therefore, this patent adopts a graph neural network with a node importance strategy in the graph convolutional neural network model. By assigning different importance weights to different nodes, the model focuses on the nodes and relationships that are more important to global resource scheduling, and can allocate resources more reasonably in the resource scheduling process. By assigning different weights to each neighbor node when aggregating information at each level of the cloud-edge system, these weights can reflect the importance of the node. In this way, the node importance strategy is added, and the original graph convolutional neural network formula is modified to obtain:

[0033]

[0034] Among them, ⊙ represents the Hadamard product, that is, the multiplication of corresponding elements; is a symmetric normalized matrix that includes self-connections; S is the node importance matrix, which reflects the relative importance between nodes and is pre-calculated based on the degree information of the nodes.

[0035] SIGNet designed in this patent introduces a semi-local centrality measurement method to better reflect the actual importance of nodes in the graph. The semi-local centrality measurement not only considers the direct neighbors of the node, but also the influence of the second-order neighbors. Define the semi-local centrality C of a node v semi-loc (v) as follows:

[0036]

[0037] Where d(v) represents the degree of node v, that is, the number of nodes directly connected to v; N1(v) and N2(v) represent the first-order and second-order neighbor sets of node v, respectively; coefficients α and β are used to adjust the importance weights of neighbors at different levels, and the optimal value is found through cross-validation. By calculating the semi-local centrality C of each virtual device v semi-loc (v) It is possible to more clearly understand which devices in the cloud-edge network play a more important role in the network, thereby making more reasonable allocation decisions when resources are limited. In addition, to prevent overfitting, a regularization term is added when calculating semi-local centrality:

[0038]

[0039] Among them, γ is the regularization coefficient.

[0040] In the cloud-edge collaboration scenario, edge devices are usually responsible for real-time data processing, while the cloud is responsible for complex computing and big data storage. SIGNet's propagation layer can be used to simulate the flow of data between edge devices and the cloud. The new representation of node v Represented by the current And the representation of the neighbors is obtained after transformation, and the transformation formula is as follows:

[0041]

[0042] Where W (l) and b (l) are the weight matrix and bias vector of the lth layer respectively; ||N(v)|| represents the number of neighbors of node v; the activation function σ adopts the ReLU function.

[0043] In the reinforcement learning part, the action space a defines the decisions that the DSORL algorithm can take at each time step. In the cloud-edge collaborative environment, the action space includes the decision of mapping virtual network requests to the cloud or edge. Define each action as a binary tuple: a cloud Indicates whether to map the current virtual network request to the cloud. edge Indicates whether to map the current virtual network request to the edge. These decisions need to be dynamically adjusted based on the real-time feedback provided by the digital twin model to ensure optimal resource allocation.

[0044] The reward function r(s,a) defines the immediate reward obtained by taking action a in state s. In the digital twin system, the reward function is defined as follows:

[0045] r(s,a)=α·r(s′)+β·(1-d(s′))-γ·e(s′)

[0046] Where s′ is the new state after executing action a, α, β, γ are weight coefficients used to adjust the importance of different goals, and r, d, and e represent the resource utilization, delay, and consumption in the new state, respectively.

[0047] The goal of the DSORL algorithm is to maximize the cumulative expected reward while maximizing the policy entropy. The optimization objective function can be expressed as:

[0048]

[0049]

[0050] Among them, r(s t ,a t ) is in state st Take action a t The instant reward obtained, α is the entropy temperature coefficient, which is used to control the entropy term The weight of strategy π in state s t The entropy under .

[0051] In order to handle the continuous state and action space in the digital twin system, this patent parameterizes the value distribution and random strategy as and π φ (·|s), where θ and φ are network parameters, and these two parameterized functions are modeled as diagonal Gaussian functions. The key technologies of the DSORL algorithm include the cycle of policy evaluation (Critic update) and policy improvement (Actor update). An objective function is designed in the policy evaluation network to update the Critic network:

[0052]

[0053] in, Represents the playback buffer area of ​​the digital twin system’s historical data. and are the parameters of the target network, is the estimated value of the soft Q network, Z is the conditional probability, y z is the target value returned randomly:

[0054]

[0055] if If it obeys the Gaussian distribution, it can be transformed into Where Q θ (s,a) and σ θ (s, a) represent the mean and standard deviation of the value distribution respectively. Combining it with the above formula, we can get the updated gradient of the Critic network as:

[0056]

[0057] The critical gradient consists of two parts: the mean gradient and the variance gradient. z and Q θ The difference of (s,a) will be treated as a random version of the TD error.

[0058] The objective function in the policy improvement actor network will be set as:

[0059]

[0060] Among them, Q θ(s,a) is the soft Q function, and α is the entropy adjustment parameter. While maximizing the cumulative reward, the entropy of the strategy is also maximized to ensure sufficient exploration. In addition, the algorithm introduces a soft target that combines the expected return and the policy entropy term to ensure continuous exploration during the optimization process. This design helps maintain the diversity of strategies during the policy iteration process and promotes a deeper understanding of the environment.

[0061] The present invention uses the Bellman equation to update the Q function and the V function, wherein the update of the Q function can be based on the following form of soft Bellman operation:

[0062]

[0063] Further use the smallest two Q values ​​to reduce overestimation:

[0064]

[0065] The above formula represents the expected cumulative discounted reward in the future after executing action a in state s and acting according to strategy π. Among them, γ is the discount factor, which is between 0 and 1 and determines the current value of future rewards.

[0066] In order to dynamically balance exploration and exploitation, a parameter called entropy temperature is set. This parameter controls the randomness of the strategy and allows the algorithm to automatically adjust the degree of exploration according to the progress of learning. As training progresses, the entropy temperature will gradually adjust. This patent implements this through an additional loss function, which aims to make the entropy of the strategy close to a target entropy value:

[0067]

[0068] in, is the target entropy level; Indicates the target entropy level The logarithmic term of is used to set the target value of entropy. By optimizing this loss function, α can be automatically adjusted to achieve the desired entropy level.

[0069] The algorithm framework designed by the present invention is as follows Figure 4As shown in the figure, the structure of the graph convolutional neural network with node importance strategy is shown. First, the input data includes the adjacency matrix and the feature matrix. The adjacency matrix reflects the connection between cloud nodes and edge nodes, and the feature matrix contains the attribute information of each node. This information comes from the digital twin model, truly reflects the state of the physical system, and provides the necessary basis for resource scheduling decisions. Then the data flows through a series of graph convolutional layers to aggregate the node information. Each layer will perform a weighted average on the information of adjacent nodes, and then add it to its own features to form a new feature vector. This process can not only effectively capture local structural information, but also ensure that the algorithm can dynamically adjust according to the latest network status through the feedback mechanism of digital twin technology. The node importance strategy is added to each convolutional layer. This strategy evaluates the importance of nodes and readjusts the weights between nodes based on the evaluation results. This design allows those more important nodes to play a greater role in the subsequent aggregation process, thereby optimizing the resource scheduling strategy. Finally, the features processed by multiple layers of SIGNet will be summarized into a new feature matrix. This matrix contains information about the original features and their complex relationships, and will serve as the input of the reinforcement learning environment.

[0070] The resource allocation and strategy generation of reinforcement learning are implemented by the value network and the policy network respectively. In this framework, the DSORL algorithm first updates the distribution value network based on the historical sample data collected from the digital twin system. Digital twin technology provides accurate environmental state information for the reinforcement learning algorithm by simulating the state of the physical system in real time. This process continuously optimizes the parameters of the Actor-Critic network so that the predicted state-action value function can more accurately reflect the state of the actual traffic system. Then, after the value distribution network is updated, its output results will be used to guide the update of the policy network. The policy network will use this information to adjust its own parameters to better adapt to the current environment and maximize the benefits of the actions taken. By combining the ability of graph neural networks to capture complex network structures and the intelligent decision-making advantages of reinforcement learning, the DSORL algorithm realizes the adaptive and intelligent resource scheduling in the cloud-edge collaborative system. The algorithm uses the value distribution policy gradient algorithm to reduce the overestimation problem and optimize the scheduling strategy, and introduces the entropy regularization term to balance exploration and utilization to ensure that the best decision can be made in a complex and changing environment.

[0071] The effect of the present invention will be further described below in conjunction with simulation experiments:

[0072] This patent uses the simulation experiment tool GT-ITM and the deep learning development framework PyTorch to complete experimental verification. GT-ITM (Georgia Tech Internetwork Topology Models) is a tool for generating network topology models. It is mainly used to create virtual network structures in digital twins. These structures imitate the characteristics and properties of actual networks. Through a series of parameterized configurations, users are allowed to control the characteristics of the generated topology, thereby creating a network topology structure that meets specific research needs for network experiments and analysis.

[0073] A physical network with 100 physical nodes and 600 physical links is generated by adopting the Waxman topology model, which simulates the network configuration of the physical entity layer and the cloud-edge twin layer. In the Waxman model, the probability of connecting any two nodes is 0.5. The node resource and link resource capacity in the twin system follow a uniform distribution of 50 to 100 units. In each time step, 2000 virtual network resource requests are sent from the physical entity layer to the twin network in a Poisson process sequence, with an average arrival rate of 0.2. Each virtual network resource request contains 2 to 10 different numbers of virtual networks, and the virtual node computing resource requirements and virtual link resource requirements are randomly and uniformly distributed in 1 to 50 units. The simulation time is set to 20,000 time units.

[0074] The environment of the present invention is built on the Linux operating system, and the hardware resources are set to 64GB memory, CPU Intel Corei7-13600, and GPU NVIDIARTX4070.

[0075] After many simulation experiments, combined with Figure 5 The simulation results are used to further illustrate the effect of the present invention.

[0076] Figure 5 It is a line graph showing the changes in the acceptance rate of different algorithms over time, where the horizontal axis represents 28 time steps, and the time step length of this experiment is set to 1000 seconds; the vertical axis represents the acceptance rate of each algorithm. It can be seen from the figure that with the passage of time, the acceptance rate of different algorithms has decreased, but the new method proposed in this invention has better performance in both acceptance rate and stability compared with the traditional method, while the performance of the other methods is relatively low or fluctuates greatly.

[0077] Most of the other algorithms compared use strategies based on weights and priorities. Therefore, the initialization performance of each metric is relatively good, but as time goes by, traditional heuristic algorithms cannot play a better advantage in scheduling problems. Therefore, the algorithm of the present invention significantly improves the robustness and effect of the algorithm through the joint support of graph neural networks and reinforcement learning.

Claims

1. A virtualized digital twin traffic resource scheduling model based on cloud-edge collaboration, characterized in that: The following steps are involved: S1: Design a digital twin model to model network resources in cloud computing and edge computing environments as a graph structure. S2: Design a graph neural network to extract model features. S3: Design a reinforcement learning model to obtain a scheduling solution through training.

2. The method of modeling network resources in cloud computing and edge computing environments according to claim 1, characterized in that: The S1 comprises the following steps: Step 1-1: Abstract the cloud-edge network resources under the digital twin system into a graph structure, where nodes represent computing resources or storage devices, and edges represent connections between resources or data transmission paths. Step 1-2: Generate a random graph of network topology through the Waxman model, in which the network request process arrives in the order of the Poisson process. Step 1-3: Model the network resource environment, define the state space s, including the resource occupancy of each node, inter-node communication delay, node importance index, etc., define the action space a of the agent, and design the reward function r.

3. According to the design graph neural network model of claim 1, network features are extracted, characterized in that: The S2 comprises the following steps: Step 2-1: Design a graph convolutional neural network model SIGNet to capture the topological structure characteristics of the network and the dynamic interaction information between nodes. Step 2-2: Through the graph convolution operation, the feature representation of each node is effectively encoded, which not only contains its own state information, but also integrates the influence of neighboring nodes, thereby achieving a deep understanding of the global state of the network. Step 2-3: A node importance strategy is also added. In the first layer of SIGNet, in addition to the conventional adjacency matrix and node features, an additional node importance feature vector is added as input. An additional weight distribution mechanism is used to ensure the priority of key nodes in resource scheduling, thereby improving the overall stability and efficiency of the system. Step 2-4: Using the semi-local centrality measurement method, the model's ability to understand node relationships in different resource configurations and structural levels, as well as its ability to mine resource relationships in the real-time traffic scheduling process, provides structured semantic support for comprehensive resource scheduling. Step 2-5: Through multiple layers of SIGNet iteration, the state information of the network is integrated to obtain the feature matrix, providing a feature representation from a global perspective for subsequent decision making.

4. The design reinforcement learning model according to claim 1 obtains a scheduling solution through training, characterized in that: The S3 comprises the following steps: Step 3-1: Based on the network state encoded by the graph neural network, establish a value function network, including the state value function V(s) and the state-action value function Q(s,a), which are used to evaluate the quality of a given state or state-action pair. Step 3-2: Establish a Q network to evaluate the expected return after taking a certain action, and design a V network to evaluate the value of the current state. Step 3-3: At the same time, add the negative term of policy entropy to the objective function to encourage the algorithm to maintain sufficient exploration while seeking high returns and avoid converging to suboptimal strategies too early. Step 3-4: By adopting the value distribution strategy iteration framework of the Bellman equation, the over-estimation problem is effectively suppressed, making the scheduling strategy more accurate, and multi-dimensional joint optimization with the cloud-edge terminal is carried out to achieve real-time and efficient scheduling of complex transportation resource systems.

Images