Edge cloud collaborative federal digital twin model construction method based on dynamic hierarchical game

Abstract

The invention discloses an edge cloud collaborative federated digital twin model construction method based on a dynamic hierarchical game, and aims to construct an edge cloud collaborative federated model generation framework based on distributed sensing data to solve the problems of data islands, high time and energy overhead, poor model performance and the like existing in central dynamic digital twin model construction. And an online decision optimization problem is modeled by taking maximization of long-term model quality and reduction of long-term resource overhead as targets, and dynamic decision optimization is realized. In a federated digital twinborn model generation framework, a bottom sensor collects feature data of a physical entity and transmits the feature data to an edge server to construct a local digital twinborn model, and a cloud server collects all the local digital twinborn models and integrates the local digital twinborn models into a global digital twinborn model. According to the method, the technical problems of model heterogeneous dynamic evolution, sensor resource overlapping and sharing and computing communication resource joint optimization in federated digital twinborn construction are effectively solved, the quality of the digital twinborn model is remarkably improved, and the system energy consumption and configuration cost are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of digital twins, and specifically relates to building a federated digital twin model generation architecture based on distributed perception data in edge-cloud collaborative communication, and on this basis dynamically optimizing local digital twin model construction task offloading, sensor selection and resource allocation with the goal of maximizing long-term system performance, and in particular relates to a method for building an edge-cloud collaborative federated digital twin model based on dynamic hierarchical game. Background Art

[0002] As a breakthrough technology, Digital Twin (DT) can build high-fidelity, ultra-realistic virtual mappings for physical entities, reshaping the interaction paradigm between the physical and digital worlds. With the help of real-time simulation and predictive analysis, this technology can promote innovations in telemedicine, intelligent transportation, autonomous driving and other fields. Among them, the construction of digital twins (i.e., creating dynamic self-evolving visual models through artificial intelligence algorithms and deep neural networks) is the core link in realizing its potential. This process requires not only real-time collection of sensor data in dynamic environments, but also continuous updating of model status to ensure accurate synchronization between virtual models and physical entities.

[0003] Although existing studies have explored the construction of digital twins based on distributed perception, they all adopt a centralized framework. Among them, after the sensor collects data, it is directly uploaded to the cloud server, which completes the model docking, rendering and visualization processing. However, the state of physical entities continues to evolve, and the centralized framework requires frequent uploading of all data and updating of the global model, resulting in high communication and computing costs. In addition, the data island phenomenon that is prevalent in wide-area wireless network environments will hinder the use of real-time data, thereby affecting the quality of the model. To this end, the present invention pioneers a federal digital twin construction framework that collaborates with edge servers and cloud servers. The framework disassembles the global digital twin model in the cloud into multiple functional components, namely local digital twin models, whose creation and evolution are completed by edge servers in a distributed manner. The specific process is as follows: First, the cloud server splits the global model into several local models and distributes them to the edge server; second, each edge server uses the associated sensor data to create a local model in parallel; finally, the cloud integrates all local models and forms a complete global model through connection and coordination. This distributed architecture significantly improves construction efficiency and model accuracy through task decomposition and parallel processing.

[0004] The digital twin model is a complex combination of multiple models such as artificial intelligence, visualization, and rule engines. Its federated construction is essentially different from the existing distributed model training architecture (such as federated learning). First, the local models in federated learning are isomorphic, while the local digital twin models will show significant heterogeneity due to the division of functions. For example, in the traffic digital twin system, the local models for vehicle trajectory prediction, pedestrian behavior recognition, and road condition perception may use different algorithm architectures, but they need to collaborate to build the overall traffic situation. Second, federated learning focuses on static model training, while digital twins need to be dynamically updated according to the entity state, requiring the construction process to have real-time adaptive capabilities. For example, the digital twin of a smart factory needs to be reconstructed in real time as the production line status changes to accurately map the equipment operation status. In order to achieve the efficient operation of the above framework, three core problems need to be solved: the local model allocation strategy of the cloud server to the edge server, the dynamic association mechanism between the edge server and the sensor, and the resource optimization configuration of the global model integration. Its challenges are mainly reflected in two aspects:

[0005] a. Challenge 1: Multi-level coupled optimization and dynamic uncertainty. Obviously, under the collaborative architecture of edge servers and cloud servers, the digital twin model splitting, local digital twin model allocation, edge server and sensor association, local digital twin model training and global integration are executed in sequence but tightly coupled, which means that all related resource allocations must be jointly optimized. In addition, unlike the static weighting of each local model by preset coefficients in federated learning, the local digital twin model created by the edge server has different contributions to the cloud global model integration and has different importance weights. Due to the uncertainty of digital twin evolution, such weights may change dynamically over time. This time-varying characteristic, combined with unpredictable wireless network conditions and fluctuations in sensor capabilities, makes the optimization problem uncertain and dynamic, which requires the design of advanced online algorithms with strong exploration capabilities to maximize the long-term performance of the system.

[0006] b. Challenge 2: Resource competition under overlapping relationships. In actual scenarios, local digital twin models, which are functional components of global digital twin models, may exhibit certain shared features (such as the same rendering materials and core functions), which means that there is partial overlap in the data required for their creation. Therefore, in the federated digital twin construction framework proposed in the present invention, each sensor can be associated with multiple edge servers at the same time to participate in the creation of different local digital twin models. This leads to the formation of complex overlapping associations between edge servers and sensors, requiring us to extend the analysis and solution methods of traditional independent alliances to overlapping alliance scenarios and integrate them into online optimization algorithms. Summary of the invention

[0007] Purpose of the invention: In view of the challenges that the above-mentioned existing digital twin model construction centralized architecture does not fully utilize edge-cloud resources and considers the dynamic evolution of digital twins, the present invention aims to provide an edge-cloud collaborative federated digital twin construction method based on hierarchical game and deep reinforcement learning.

[0008] Technical solution: A method for constructing an edge-cloud collaborative federated digital twin model based on dynamic hierarchical game, wherein the method is to construct a digital twin model based on distributed sensing data in an edge-cloud collaborative network. The federated digital twin model includes the following third aspects for data interaction:

[0009] First, in each time slice, the central cloud server is responsible for building a global model and dividing the complete model into at least one local digital twin according to its function;

[0010] Second, each local digital twin model construction task is assigned to an edge server, which uses the feature data provided by its associated sensors to create and update the local digital twin model;

[0011] Third, after all local digital twin models are built, the edge server transmits them to the cloud server for global integration, forming a complete global digital twin model by connecting and coordinating the local models;

[0012] The method further includes:

[0013] The task offloading in the global digital twin model and the sensor selection and resource allocation decision-making problems in the local digital twin model are respectively handled through bilateral matching game and overlapping coalition game.

[0014] A system utility function including local and global digital twin model quality, transmission speed, computing energy consumption, latency, and configuration cost is constructed to characterize system performance. In order to maximize system performance, this method adopts the Gale-Shapley algorithm and overlapping coalition selection algorithm to solve the short-term optimal construction task offloading and sensor selection decisions respectively, and combines the deep reinforcement learning algorithm to generate the long-term optimal preference sequence and resource allocation decision, thereby extending the short-term optimal decision to the long-term optimal.

[0015] Furthermore, the method for establishing a network model based on a federated digital twin model of distributed sensing data includes the following computational processing:

[0016] (1) In each time slice, calculate the physical quantities related to the benefits of the edge collaborative system, including the quality of the digital twin model, the amount of collected data, and the upper limit of the number of training rounds, including:

[0017] Instantaneous characteristic data collection amount d c The calculation method of (t) is:

[0018]

[0019] Among them, d n,c (t) represents the local digital twin model that each sensor n can obtain in time slice t The amount of feature data.

[0020] Model quality of the global digital twin model A Global The calculation method of (t) is:

[0021]

[0022] Model quality of local digital twin model A c,b The calculation method of (t) is:

[0023]

[0024] in, Indicates the construction of a local digital twin model The maximum model quality required, Represents each local digital twin model The model quality that can be achieved after multiple rounds of training using the accumulated data, It represents the impact of the accumulated feature data on the model quality, quantified as a logarithmic normalization function with parameter β. Represents the local digital twin model as of time slice t The amount of accumulated feature data collected, It represents the approximate value of the model accuracy calculated according to the logistic loss function. L, δ, γ are predefined hyperparameters, indicating that the loss function of the AI ​​model for training digital twins satisfies L-Lipschitz continuity and γ-strong convexity, and δ∈(0,2 / L) is the corresponding learning rate.

[0025] The upper limit of edge server calculation rounds The calculation method is:

[0026]

[0027] (2) Calculate the physical quantities related to system cost, including system energy consumption and delay.

[0028] Total delay of a single time slice in the edge-cloud collaborative system τ Total (t) and total energy consumption E Total The calculation methods of (t) are:

[0029]

[0030] Sensor data acquisition delay and energy consumption The calculation methods are:

[0031]

[0032] Characteristic data upload rate The calculation method is:

[0033]

[0034] Where W represents the bandwidth of each subcarrier, H n,b,w (t) is the instantaneous channel gain from wireless sensor n to edge server b on subcarrier w, represents the co-channel interference at edge server b, P n represents the transmission power of wireless sensor n, is the additive white Gaussian noise at the edge server b.

[0035] Historical data migration delay and energy consumption The calculation methods are:

[0036]

[0037] in, Represents edge server To another edge server The transmission rate of the optical fiber link, P b' Represents edge server of transmission power.

[0038] Computational latency of local digital twin models and energy consumption The calculation methods are:

[0039]

[0040] in, Represents edge server Calculate the number of CPU cycles required per byte of data (in seconds / byte), F b For edge servers CPU frequency (in cycles / second), ρ b Represents edge server The effective switching capacitance.

[0041] Local digital twin model parameter upload delay and energy consumption The calculation methods are:

[0042]

[0043] Among them, D c Local digital twin model The model size.

[0044] Global Digital Twin Model Integration Latency and energy consumption The calculation methods are:

[0045]

[0046] in, The number of CPU cycles required to integrate each byte of local digital twin model data, F CS is the CPU frequency of the cloud, CS is the access rate of a single processing unit in the cloud, H CS,c To integrate local digital twin models The average number of instructions, and is the hardware parameter of the cloud processing unit;

[0047] (3) Calculate the system utility function based on the physical quantities related to system benefits and costs. Short-term utility function of the federated digital twin model It is expressed as:

[0048]

[0049] Among them, ζ and κ are weight coefficients. Long-term utility function U Sys It is expressed as:

[0050]

[0051] Among them, C Conf is the configuration cost coefficient, represents the total number of sensor selection changes of the edge server within time slice t, and ⊕ is the exclusive OR operator.

[0052] (4) Establishing the system’s decision variable set It contains the local digital twin model construction task offloading decision x c,b (t), the sensor selection y of each edge server b,n (t), subcarrier allocation z b,n,w (t), and the number of computation rounds allocated to each edge server T b (t).

[0053] The online optimization problem for maximizing the performance of the long-term federated digital twin construction system is expressed as:

[0054]

[0055] τ Total (t)≤δ T ,

[0056]

[0057] In the formula, and denotes the set of all edge servers, sensors and subcarriers respectively, δ T Indicates the maximum threshold of delay within a time slice. Indicates the minimum communication rate to maintain stable communication between the sensor and the edge server.

[0058] Transform the original problem Two parallel sub-problems, including the cloud server side sub-problem And the edge server side issue Respectively expressed as:

[0059]

[0060] δ Total (t)≤δ T ,

[0061]

[0062] τ Total (t)≤δ T ,

[0063]

[0064] (5) Construct a dynamic two-layer hierarchical game to transform sub-problems and The game expansion is as follows:

[0065]

[0066] in, express The set of participants, namely the cloud server and all edge servers, represents the upper subgame in each time slice t, It represents the corresponding upper-level game strategy. represents the lower-level subgame in each time slice t, represents the corresponding lower-level game strategy, and They represent the long-term utility of the cloud server and all edge servers respectively.

[0067] In order to solve For the local digital twin model, the bilateral matching game is used to model it, and its expression is as follows:

[0068]

[0069] in, express The set of participants, i.e., all local digital twin models and edge servers, denote the preferences of the local digital twin model and the edge server for each other, φ t (i) Represents the bidirectional matching mapping between the local digital twin model and the edge server.

[0070] In order to solve In the resource allocation problem of sensor selection for edge servers and construction of local digital twin models, overlapping coalition formation game is used to model the problem, and its expression is as follows:

[0071]

[0072] in, Express participation All sensors and edge servers, Δ represents all possible sensor clusters (i.e. alliances), represents the set of all alliances in time slice t, represents the set of resource allocation decisions, Indicates that all sensors selected by edge server b are members of the alliance Co b The utility function, U n (t) represents the contribution of sensor n to each alliance. Evolves in each time slot, thus achieving dynamic strategy adjustment. In each time slice t, Making short-term decisions as a leader c,b (t), and on this basis, As a follower, after observing the decision x c,b (t) and then determine the short-term decision y b,n (t),z b,n,w (t) and T b (t). This process is repeated from one time slice to another, thus integrating the two short-term subgames into In the long-term hierarchical game framework;

[0073] (6) Design a short-term game equilibrium solution method based on the Gale-Shapley algorithm and the overlapping coalition selection algorithm, by solving the sub-game in each time slice. and The equilibrium solution is used to realize the optimal local digital twin model construction task offloading of the federated digital twin construction system in each time slice and the optimal sensor selection of the edge server (i.e., x c,b (t) and y b,n (t)). The overlapping alliance formation algorithm used to solve the optimal edge server sensor selection in each time slice is a distributed operation. In the same time slice, each party independently calculates the alliance selection of each edge server. In essence, it solves the optimal edge server sensor selection in each time slice. equilibrium, i.e. stable alliance division The algorithm is implemented based on the following overlapping coalition switching criteria:

[0074] Transfer criteria: Sensor n from Alliance Co a Transfer to Alliance Co b , that is, execute All of the following conditions must be met:

[0075] 1) The remaining subcarriers of edge server b are sufficient;

[0076] 2)

[0077] 3)

[0078] 4)

[0079]

[0080] Joining Guidelines: Sensors from Alliance Co a Overlap to join the allianceCo b , that is, execute All of the following conditions must be met:

[0081] 1) The remaining subcarriers of edge server b are sufficient, and sensor n can continue to join new alliances;

[0082] 2)

[0083] 3)

[0084] 4)

[0085] Departure Guidelines: Sensors from Alliance Co a Unilateral departure means execution All of the following conditions must be met:

[0086] 1)

[0087] 2)

[0088] (6) Design an intelligent decision-making algorithm based on deep reinforcement learning to solve multi-stage sequential games The global equilibrium solution (i.e., the long-term optimal decision) in the entire system running time 0≤t≤T realizes the expansion of the short-term equilibrium solution to the long-term equilibrium solution. Solving the hierarchical game The deep reinforcement learning algorithm for the global equilibrium solution during the entire system runtime is based on the Proximal Policy Optimization (PPO) algorithm and the Actor-Critic (AC) framework. The state space of the reinforcement learning process comprehensively considers the importance weights, the amount of historical data collected, historical decisions, and the coalition state (i.e., I c (t),d c (t-\1), and ). In addition, the intelligent decision-making algorithm based on deep reinforcement learning integrates distributed training and centralized training. c (t),p b (t)) and edge server-side decision (resource allocation z b,n,w (t),T b (t)) Use different agents to train the best strategy.

[0089] Beneficial effects: Compared with the prior art, the present invention has the following three significant features and substantial improvements:

[0090] First, the present invention pioneered a federated digital twin hierarchical game construction model based on edge-cloud collaboration, which solved the joint optimization problems of digital twin model splitting, local model allocation, multi-sensor overlapping association and resource scheduling in a dynamic evolution environment. In the modeling process, it fully covers the decision-making interaction relationship between edge servers and sensors, the time-varying contribution weight of local models to the global, and the comprehensive trade-off between communication energy consumption and computing cost in resource allocation, so as to achieve accurate characterization of optimization goals under multi-dimensional dynamic constraints;

[0091] Second, the present invention proposes a distributed optimization method based on stable matching and overlapping alliance formation in response to the overlapping data requirements and resource competition characteristics of local models. This method innovatively designs the local model dynamic allocation mechanism of edge servers and the multi-attribution association strategy of sensors, adopts a cloud-edge two-layer game architecture, and realizes efficient collaboration between edge servers and sensors through the Gale-Shapley algorithm and the overlapping alliance selection algorithm based on switching rules. Compared with traditional centralized optimization, this method supports distributed autonomous decision-making and has the advantages of low latency and high scalability;

[0092] Third, the present invention proposes a dynamic multi-objective optimization method that integrates deep reinforcement learning to address the uncertainty of digital twin evolution. This method captures the time dependence of system states through reinforcement learning agents, maps the short-term equilibrium of hierarchical games into long-term optimal decisions, and can still achieve multi-objective balance of global digital twin model quality, energy consumption and cost under dynamic channel conditions, data quality fluctuations and time-varying model weights, and has strong robustness and environmental adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a schematic diagram of the system structure and device interaction of the method of the present invention;

[0094] Figure 2 It is a multi-stage sequential game flow chart of the present invention;

[0095] Figure 3 It is a schematic diagram of the framework of the intelligent decision-making method based on reinforcement learning in the present invention;

[0096] Figure 4 This is a comparison chart of the system cumulative utility of the intelligent decision-making method based on reinforcement learning and the existing methods. DETAILED DESCRIPTION

[0097] In order to illustrate the technical solution disclosed by the present invention in detail, the present invention is further elaborated below in conjunction with the accompanying drawings and specific implementation methods.

[0098] First, the technical problem that the method described in the present invention focuses on is: in the edge-cloud collaborative federated digital twin construction scenario, how to achieve a long-term multi-objective balance between maximizing the quality of the global digital twin model and minimizing the construction cost by dynamically optimizing the allocation of local digital twin models, the overlapping association strategy of edge servers and sensors, and the scheduling of computing and communication resources. This problem needs to overcome the core challenges of uncertainty in the evolution of digital twins, heterogeneity of local models, and resource competition caused by multi-attribution association of sensors.

[0099] The main idea of ​​the present invention is to propose a federated digital twin hierarchical optimization framework for the complex requirements of dynamic construction of digital twin models in edge-cloud collaborative environments, aiming to maximize the quality of the global model while minimizing energy consumption and configuration costs. First, a long-term online optimization problem is constructed to dynamically coordinate the allocation of local digital twin models, edge server-sensor association strategies, and computing and communication resource allocation, and characterize the impact of uncertainty in the evolution of digital twins. To solve this problem, the global optimization is decomposed into a cloud-edge two-layer collaborative architecture: the cloud layer optimizes the task offloading of local digital twin model construction through a bilateral matching model, and the edge layer optimizes the sensor selection and resource scheduling of edge servers based on an overlapping alliance model. On this basis, a distributed short-term optimization algorithm (a cloud task allocation algorithm based on stable matching and an edge alliance formation algorithm based on switching rules) is designed to quickly generate local optimal solutions. Finally, in order to cope with the long-term decision-making challenges in a dynamic environment, an online decision optimization algorithm integrating deep reinforcement learning is proposed. By capturing the state dependency relationship in the time dimension, the hierarchical short-term decision is extended to a long-term equilibrium strategy to achieve an adaptive balance between the quality of global digital twin construction, resource efficiency, and cost constraints.

[0100] Specifically, a method for constructing an edge-cloud collaborative federated digital twin model based on dynamic hierarchical game is implemented in the following steps:

[0101] Step 1: Design a framework for building a federated digital twin model based on distributed perception data.

[0102] First, build a system model, such as Figure 1 As shown in the figure, the present invention considers a federated digital twin construction framework based on an edge-cloud collaborative system, which consists of a central cloud server (CS), a group of local edge servers (Quantity is ), and a randomly distributed set of sensors (Quantity is In this framework, the digital twin model of the target is regarded as a global model, which can be split into multiple functional components (i.e., local digital twin models), which are created in parallel on edge servers. These local digital twin models are essentially digital copies of different parts of the physical entity, which are built by collecting feature data from the associated sensor group. Finally, a complete global digital twin model can be formed by uploading the local digital twin models of all edge servers to the central cloud server for integration. According to the digital twin configuration specification, we define it as follows:

[0103]

[0104] In the formula, It represents the set of all local digital twins, and the symbol <·> represents the integration process of the local digital twin model into the global digital twin model.

[0105] The construction process of the digital twin of the state is divided into three steps: first, the cloud server splits the global digital twin model into multiple functional components (i.e., local digital twin models) and distributes them to different edge servers; second, sensors are dynamically associated with multiple edge servers to form an overlapping data acquisition network to support the parallel creation of different local models (because some models need to share data); finally, each edge server independently builds a local model based on the initial parameters (such as model structure, rendering configuration) sent by the cloud and the feature data collected by the associated sensors, and finally the cloud server integrates them uniformly to form a complete global digital twin model through parameter alignment and function coordination. In order to characterize the time-varying uncertainty in the evolution of digital twins, the overall running time of the system is divided into T time slices.

[0106] In order to characterize the system dynamics caused by the long-term evolution characteristics of digital twins, this paper adopts a time-sharing system model, in which each time slice t∈{1,2,…,T} corresponds to a digital twin evolution cycle. In this framework, the dynamic importance weight index is defined Among them I c (t)∈[0,1] represents the local digital twin model in time slice t The importance weight of each local digital twin model is determined by the specific application requirements of each local digital twin model.

[0107] Step 2: Construct the long-term average utility optimization problem of the three-party equipment.

[0108] Considering that performance gain (e.g., model quality of the global digital twin model) and potential cost (e.g., system energy consumption and latency) play an important role in the construction of the federated digital twin model, these factors are comprehensively considered in the construction of the long-term and short-term utility functions of the system. First, the model quality of the digital twin model is calculated:

[0109]

[0110] And the model quality of the local digital twin model:

[0111]

[0112] in, Indicates the construction of a local digital twin model The maximum model quality required, Represents each local digital twin model The model quality that can be achieved after multiple rounds of training using the accumulated data, It represents the impact of the accumulated feature data on the model quality, quantified as a logarithmic normalization function with parameter β. Represents the local digital twin model as of time slice t The amount of accumulated feature data collected, It represents the approximate value of the model accuracy calculated according to the logistic loss function. L, δ, γ are predefined hyperparameters, indicating that the loss function of the AI ​​model for training digital twins satisfies L-Lipschitz continuity and γ-strong convexity, and δ∈(0,2 / L) is the corresponding learning rate.

[0113] Next, the present invention calculates the sensor data acquisition delay and energy consumption

[0114]

[0115] Historical data migration delay and energy consumption

[0116]

[0117] in, Represents edge server To another edge server The transmission rate of the optical fiber link, P b' Represents edge server The transmission power;

[0118] Local digital twin model construction delay and energy consumption

[0119]

[0120] in, Represents edge server Calculate the number of CPU cycles required per byte of data (in seconds / byte), F b For edge servers CPU frequency (in cycles / second), ρ b Represents edge server The effective switching capacitance of

[0121] Local digital twin model parameter upload delay and energy consumption

[0122]

[0123] Among them, D c Local digital twin model Model size;

[0124] Global Digital Twin Model Integration Latency and energy consumption

[0125]

[0126] in, The number of CPU cycles required to integrate each byte of local digital twin model data, F CS is the CPU frequency of the cloud, CS is the access rate of a single processing unit in the cloud, H CS,c To integrate local digital twin models The average number of instructions, and are the hardware parameters of the cloud processing unit.

[0127] Based on these basic energy consumption and delay, the present invention further calculates the total delay τ of a single time slice of the system Total (t) and total energy consumption E Total (t):

[0128]

[0129] According to the physical quantities related to system benefits and costs, the present invention calculates the system short-term utility function

[0130]

[0131] Among them, ξ and k are weight coefficients, and the long-term utility function

[0132]

[0133] Among them, C Conf is the configuration cost coefficient, represents the total number of sensor selection changes of the edge server within time slice t, and ⊕ is the exclusive OR operator.

[0134] Based on system utility modeling, the present invention constructs an online optimization problem for maximizing the long-term performance of the system:

[0135]

[0136] τ Total (t)≤δ T ,

[0137]

[0138] in, represents the set of decision variables of the system. Then, the present invention transforms Two parallel sub-problems, including the cloud server side sub-problem

[0139]

[0140]

[0141] τ Total (t)≤δ T ,

[0142]

[0143] And the edge server side sub-problems:

[0144]

[0145] τ Total (t)≤δ T ,

[0146]

[0147] Step 3: Construct a dynamic two-layer hierarchical game to transform sub-problems and The game expansion is as follows:

[0148]

[0149] in, express The set of participants, namely the cloud server and all edge servers, represents the upper subgame in each time slice t, It represents the corresponding upper-level game strategy. represents the lower-level subgame in each time slice t, represents the corresponding lower-level game strategy, and They represent the long-term utility of the cloud server and all edge servers respectively.

[0150] In order to solve In the offloading of the local digital twin model construction task, a bilateral matching subgame is constructed:

[0151]

[0152] in, express The set of participants, i.e., all local digital twin models and edge servers, denote the preferences of the local digital twin model and the edge server for each other, φ t (i) represents the bidirectional matching mapping between the local digital twin model and the edge server. In the resource allocation for sensor selection and construction of local digital twin models for edge servers, overlapping coalition formation subgames are constructed:

[0153]

[0154] in, Express participation All sensors and edge servers, Δ represents all possible sensor clusters (i.e. alliances), represents the set of all alliances in time slice t, represents the set of resource allocation decisions, Indicates that all sensors selected by edge server b are members of the alliance Co b The utility function, U n (t) represents the contribution of sensor n to each alliance.

[0155] Step 4: Define the alliance preferences and alliance switching criteria of each party's devices, and use the distributed alliance selection and alliance formation method to obtain the stable alliance division and optimal alliance selection of the three-party devices in each time slice.

[0156] First, we define the dynamic hierarchical game Long-term equilibrium solution, that is, long-term optimal decision: strategy set yes The long-term equilibrium solution must satisfy Subgames and The equilibrium solution of , and satisfies:

[0157] and

[0158]

[0159] Next, define The equilibrium solution, that is, the stable matching: A matching is stable if and only if:

[0160] For any local digital twin model With edge servers There is no blocking pair (c,b) that satisfies and where φ t (c) and φ t(b) Represents the current matching objects of the local digital twin model c and the edge server b within time slice t.

[0161] Next, define The equilibrium solution, that is, the optimal response: is an optimal response if and only if:

[0162] and

[0163]

[0164] because Depends on And it can be solved by using the Gale-Shapley algorithm. The present invention is to solve Define overlapping alliance switching criteria:

[0165] Transfer criteria: Sensor n from Alliance Co a Transfer to Alliance Co b , that is, execute If and only if 1) the remaining subcarriers of edge server b are sufficient; 2) 3) as well as

[0166] Joining Guidelines: Sensors from Alliance Co a Overlap to join the allianceCo b , that is, execute If and only if 1) the remaining subcarriers of edge server b are sufficient and sensor n can continue to join new alliances; 2) 3) 4)

[0167] Departure Guidelines: Sensors from Alliance Co a Unilateral departure means execution If and only if:

[0168] 1)

[0169] 2)

[0170] Based on the above criteria, the present invention adopts a stable overlapping coalition formation (SOCF) algorithm to obtain stable overlapping coalition divisions in each time slot.

[0171] Specifically, given the alliance partition of the previous time slot, that is, exist Each alliance in In the example, each sensor first calculates its own utility and then decides whether to leave the current alliance and join another existing alliance according to the three alliance switching criteria. or overlap another existing in The process is repeated until the alliance division remains unchanged, and the final alliance division is obtained after iteration. according to Get the final sensor selection y b,n (t).

[0172] Step 5: Use an intelligent decision-making algorithm based on deep reinforcement learning to solve multi-stage sequential games Global equilibrium solution during the entire system runtime.

[0173] First, the present invention uses two Markov decision processes (MDPs) to describe resource allocation decisions. and the preference list p c (t),p b (t) is the decision process. The corresponding MDPs are expressed as and

[0174] The detailed explanation is as follows:

[0175] (1) State space The state in is defined as The state in is defined as and

[0178] (2) Action Space and The action in is its own strategic decision, namely

[0179] and

[0180] (3) Conditional transition probability Ξ R ,Ξ PL : and The conditional transition probability in is defined as

[0181] (4) Reward Set and The reward functions in are expressed as:

[0182]

[0183] as well as:

[0184]

[0185] like Figure 3 As shown, the present invention adopts a deep reinforcement learning algorithm based on proximal policy optimization (PPO) and actor-critic framework (AC) to solve these two MDPs, that is, to solve the game The equilibrium solution is as follows:

[0186] 1) Establishment Agent Each agent agt b Generate resource allocation In addition, the intelligent and Used to generate and The set of all agents is defined as For an intelligent agent Design a coefficient φ e The value network is used to estimate the state value in Refers to the long-term discounted reward of e, with a discount factor of η and an initial state of s 0 . and a coefficient θ e The policy network is used to quantize the initial policy. In addition, there is an experience replay buffer to store historical experience, including previous states, new states, actions, and rewards during deep reinforcement learning training.

[0187] 2) Secondly, since the formulated MDP itself is coupled, the present invention adopts a hierarchical training process, in which the interactions between all agents follow At each time slice t in a training step, this decision sequence presents two stages: in stage 1, the agent At the same time, according to the initial generate In Phase 2, the agent At the same time, according to the initial state generate Then call the SOCF algorithm to iteratively obtain a stable alliance division and update each agent agt b of until and until convergence; and in the second stage, and First, according to and generate and Then use the Gale-Shapley algorithm to find On top of this, the present invention calculates the rewards of all agents in time slice t, and then assigns each agent e, Tuples in Stored in the experience replay buffer.

[0188] 3) In order to achieve global optimization, the network parameters of each agent must be updated at a certain frequency. When the experience replay pool of all agents reaches the rated capacity, the strategy network and value network of all agents are updated. For each agent e in the , this update process includes i) calculating the available reward of e, i.e., the discounted reward where γ t is the discount factor; ii) calculate the advantage function of e

[0189] iii) Calculate the loss function of the actor network of e in is the actor network of e in state Select action , θ' is the original parameter of the actor network of e; and iv) calculate the critic network loss function of e The parameters θ and φ can then be updated via a stochastic gradient descent method (e.g., Adam Optimizer) to minimize their corresponding loss functions.

[0191] Figure 3 This paper describes an overview of the structure of the deep reinforcement learning method proposed in this paper, including the interactions of all agents (i.e., the actions taken by each agent, the observations from the environment and other agents, and the decision sequences between them), the detailed AC framework within each agent, and the update process of the environment and network parameters (i.e., the policy network θ and the value network φ). The action generation process of each agent and the updating process of the two types of networks are repeated in each training step to achieve long-term performance guarantee.

[0192] In the performance control experiment, this example considers an uplink communication system with a range of 1000m×1000m, with N=20 sensors randomly scattered, B=5 edge servers, and cloud servers to jointly build C=5 local digital twin models. The remaining parameters are set as L n =3,W∈[1,5]~MHz, β=200,ρ b =10 -16 ,L=8,δ=0.02,γ=2,F b =64MHz,F CS =3000MHz, d n,c (t)∈[200,600]kbits,D c ∈[1,5]Mbits,P b ,P n ∈[5,33]dBm, ξ=0.1,κ=15,C Conf =15,δ T =7.6s,H n,b,w (t)∈{0.2,0.4,0.6}, η=0.92.

[0193] Figure 4 The effectiveness of the proposed federated digital twin construction framework compared with other schemes was tested. Two benchmark schemes were set up in this experiment: the Centra scheme, in which sensors transmit all data directly to the cloud, and the Non-Overlap scheme, in which the edge server adopts a non-overlapping sensor selection strategy. Figure 4 (a) and Figure 4 (b) It can be seen that the proposed federated digital twin model construction framework is superior to the Centra and Non-Overlap frameworks in terms of cumulative utility and gain. This is because the proposed framework allows sensors to upload feature data in an overlapping manner, which enables more efficient data collection, thereby making the quality of the global digital twin model higher. In addition, although the Centra framework allows sensors to upload data directly to the cloud, the number of associated sensors is limited by the available subcarriers and strict delay constraints, resulting in lower cumulative gain. Figure 4(c) shows that the cumulative cost of the framework proposed by the present invention is lower than that of the Centra framework and only slightly higher than that of the Non-Overlap framework. This is because in the Centra framework, centralized data collection and digital twin creation introduce a lot of communication and computing delays, and in the Non-Overlap framework, although non-overlapping sensor selection reduces data transmission energy consumption, it actually only accounts for a small part of the total energy consumption compared to the relatively resource-intensive local digital twin model creation and global digital twin model integration, which is difficult to offset the cumulative loss.

[0194] In the construction of the digital twin of the intelligent transportation system, the federated architecture proposed in the present invention realizes dynamic virtual-reality mapping of the global traffic situation through distributed traffic data perception and edge-cloud collaborative computing. Specifically, traffic cameras and roadside units at urban intersections collect road images and electronic police data (including traffic flow monitoring, violation photography, traffic light status, etc.) in real time, and these data are uploaded to local edge servers (such as intelligent roadside units, regional traffic regulatory departments); the edge servers perform target detection, trajectory tracking, and semantic segmentation on the original traffic data based on lightweight AI models, extract structured and standardized information such as vehicle contours, pedestrian locations, and traffic flow, and integrate visual data such as traffic monitoring to build local traffic digital twin models for each intersection and section; then, each edge server uploads the core parameters of the local twin model to the cloud server (such as the traffic command center, provincial and municipal traffic police bureaus, etc.) instead of transmitting the original data stream, thereby reducing communication pressure; finally, the cloud server integrates multiple local digital twin model parameters, reconstructs the city-level global traffic twin model, dynamically integrates road condition information at different intersections and sections, and forms a virtual model of the global road network, thereby driving business decisions such as traffic light control and accident warning. During this process, the edge server continuously updates the local twin model based on real-time collected data, and performs global model integration in the cloud, forming a digital twin model evolution of "local perception-federated edge-cloud collaboration-global integration", which not only breaks through the data perception limitations caused by data islands in the traditional centralized construction architecture, but also realizes the full-factor, high-real-time physical entity-digital twin synchronization of the transportation system through edge-cloud computing power collaboration.

Claims

1. A method for constructing an edge-cloud collaborative federated digital twin model based on dynamic hierarchical game, wherein the method is to construct a digital twin model based on distributed perception data in an edge-cloud collaborative network, characterized in that: The federated digital twin model includes the following third aspects for data interaction: First, in each time slice, the central cloud server is responsible for building a global model and dividing the complete model into at least one local digital twin according to its function; Second, each local digital twin model construction task is assigned to an edge server, which uses the feature data provided by its associated sensors to create and update the local digital twin model; Third, after all local digital twin models are built, the edge server transmits them to the cloud server for global integration, forming a complete global digital twin model by connecting and coordinating the local models; For the local digital twin model and the global digital twin model, the method further includes: The task offloading in the global digital twin model and the sensor selection and resource allocation decision-making problems in the local digital twin model are respectively handled through bilateral matching game and overlapping coalition game. A system utility function including local and global digital twin model quality, transmission speed, computing energy consumption, latency, and configuration cost is constructed to characterize system performance. In order to maximize system performance, this method adopts the Gale-Shapley algorithm and overlapping coalition selection algorithm to solve the short-term optimal construction task offloading and sensor selection decisions respectively, and combines the deep reinforcement learning algorithm to generate the long-term optimal preference sequence and resource allocation decision, thereby extending the short-term optimal decision to the long-term optimal.

2. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 1, characterized in that: The method for constructing the system utility function includes the following process: (1) In each time slice, obtain and calculate the physical quantities related to the modeling object, including the instantaneous characteristic data collection quantity d c (t), the upper limit of the number of rounds calculated by the edge server And characteristic data upload rate It also includes physical quantities related to the calculation of system performance, including the model quality A of the global digital twin model. Global (t), model quality A of the local digital twin model c,b (t), the total delay of a single time slice of the system τ Total (t), total system energy consumption E Total (t), sensor data acquisition delay and energy consumption Historical data migration delay and energy consumption Local digital twin model construction delay and energy consumption Local digital twin model parameter upload delay and energy consumption Global Digital Twin Model Integration Latency and energy consumption (2) Constructing the short-term utility function of the digital twin model and the long-term utility function U Sys , considering the model quality benefits, resource overhead costs and configuration losses during system operation; The short-term utility function It is expressed as: Among them, ξ and κ are weight coefficients; The long-term utility function U Sys It is expressed as: Among them, C Conf is the configuration cost coefficient, represents the total number of sensor selection changes of the edge server in time slice t, ⊕ is the XOR operator; (3) Constructing the decision variable set of the system where x c,b (t) represents the task offloading decision in the local digital twin model, y b,n (t) represents the sensor selection of each edge server, z b,n,w (t) represents subcarrier allocation, T b (t) represents the allocation of the number of calculation rounds for each edge server, and the mathematical expression of the online optimization problem for maximizing system performance is: t Total (t)≤δ T , In the formula, and denotes the set of all edge servers, sensors, and subcarriers, respectively, |W| denotes the cardinality of W, and δ T Indicates the maximum threshold of delay within a time slice. Indicates the minimum communication rate to maintain stable communication between the sensor and the edge server; (4) The original question Transformed into two parallel sub-problems, namely the cloud server side sub-problem And the edge server side issue Mathematically expressed as: question and Problems The constraints are the same as those in step (3); (5) Construct a dynamic two-layer hierarchical game to transform sub-problems and The game expansion is as follows: in, express The set of participants, represents the upper subgame in each time slice t, It represents the corresponding upper-level game strategy. represents the lower-level subgame in each time slice t, represents the corresponding lower-level game strategy, and denote the long-term utilities of the cloud server and all edge servers, respectively; question The solution is modeled using a bilateral matching game, which is expressed as follows: in, express The set of participants, denote the preferences of the local digital twin model and the edge server for each other, φ t (i) represents the bidirectional matching mapping between the local digital twin model and the edge server; question The solution is to use overlapping coalition formation game to model it, and its expression is as follows: in, Express participation All sensors and edge servers, Δ represents all possible sensor clusters, represents the set of all alliances in time slice t, represents the set of resource allocation decisions, Indicates that all sensors selected by edge server b are members of the alliance Co b The utility function, U n (t) represents the contribution of sensor n to each alliance; (6) A method for solving short-term game equilibrium solutions based on the Gale-Shapley algorithm and the overlapping coalition selection algorithm, by solving the subgame in each time slice. and The balanced solution is to achieve the optimal task offloading decision of the local digital twin model and the optimal sensor selection of the edge server in each time slice; (7) Intelligent decision-making algorithm based on deep reinforcement learning to solve hierarchical games The global equilibrium solution in the entire system running time 0≤t≤T achieves long-term optimization of all decisions. The deep reinforcement learning algorithm involves decision variables including preference p c (t),p b (t) and resource allocation 3. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 1 or 2, characterized in that: In the constructed edge-cloud collaborative network, if the edge server selects a sensor to collect data, it needs to allocate subcarriers for uplink transmission; Sensors with the same subcarrier will generate co-channel interference when uploading collected data, and the local digital twin model has a time-varying importance weight, so it is expressed as Among them I c (t)∈[0,1] represents the importance of the local digital twin model c in time slice t; This method takes time-varying uncertainty into account. The overall operation time of the system is divided into T time slices, and the frequency bandwidth of each subcarrier is W.

4. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 2, characterized in that: In step (1): Instantaneous characteristic data collection amount d c The calculation method of (t) is: Among them, d n,c (t) represents the local digital twin model that each sensor n can obtain in time slice t The amount of feature data; Model quality of the global digital twin model A Global The calculation method of (t) is: Model quality of local digital twin model A c,b The calculation method of (t) is: in, represents the maximum model quality of the local digital twin model, represents the model quality that each local digital twin model can achieve after multiple rounds of training using the accumulated collected data, It represents the impact of the accumulated feature data on the model quality, quantified as a logarithmic normalization function with parameter β. Represents the local digital twin model as of time slice t The amount of accumulated feature data collected, It represents the approximate value of the model accuracy calculated according to the logistic loss function. L, δ, γ are predefined hyperparameters, indicating that the loss function of the AI ​​model for training digital twins satisfies L-Lipschitz continuity and γ-strong convexity, and δ∈(0,2 / L) is the corresponding learning rate; The upper limit of edge server calculation rounds The calculation method is: Characteristic data upload rate The calculation method is: Where W represents the bandwidth of each subcarrier, H n,b,w (t) is the instantaneous channel gain from wireless sensor n to edge server b on subcarrier w, represents the co-channel interference at edge server b, P n represents the transmission power of wireless sensor n, is the additive white Gaussian noise at the edge server b; Total delay of a single time slice of the system τ Total (t) and total energy consumption E Total The calculation methods of (t) are: Sensor data acquisition delay and energy consumption The calculation methods are: Historical data migration delay and energy consumption The calculation methods are: in, Represents edge server To another edge server The transmission rate of the optical fiber link, P b' Represents edge server The transmission power; Local digital twin model delay and energy consumption The calculation methods are: in, Represents edge server Calculate the number of CPU cycles required per byte of data, F b For edge servers CPU frequency, ρ b Represents edge server The effective switching capacitance of Local digital twin model parameter upload delay and energy consumption The calculation methods are: Among them, D c Local digital twin model Model size; Global Digital Twin Model Integration Latency and energy consumption The calculation methods are: in, The number of CPU cycles required to integrate each byte of local digital twin model data, F CS is the CPU frequency of the cloud, CS is the access rate of a single processing unit in the cloud, H CS,c To integrate local digital twin models The average number of instructions, and are the hardware parameters of the cloud processing unit.

5. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 2, characterized in that: The hierarchical game Evolves in each time slot, thereby achieving dynamic strategy adjustment, specifically: in each time slice t, Making short-term decisions as a leader c,b (t), and on this basis, As a follower, after observing the decision x c,b (t) and then determine the short-term decision y b,n (t),z b,n,w (t) and T b (t), and the process repeats from one time slice to another, thus integrating the two short-term subgames into In the long-term hierarchical game framework.

6. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 2, characterized in that: In step (6), the sensors of each party in the same time slice independently calculate the alliance selection of each edge server, and then solve the alliance selection in each time slice by overlapping alliance switching criteria. Equilibrium and stable alliance division The overlapping alliance switching criteria include: Transfer criteria: Sensor n is transferred from alliance Co if and only if all the following conditions are met. a Transfer to Alliance Co b ,implement 1) The remaining subcarriers of edge server b are sufficient; 2) 3) 4) Joining criteria: Sensor n joins the alliance Co only if all the following conditions are met. a Overlap to join the alliance Co b ,implement 1) The remaining subcarriers of edge server b are sufficient, and sensor n can continue to join new alliances; 2) 3) 4) Leaving criterion: Sensor n leaves the alliance Co if and only if all conditions are met. a Unilateral departure, execution 1) 2) 7. The method for constructing an edge-cloud collaborative federated digital twin model according to claim 2, characterized in that: Layered Game The deep reinforcement learning algorithm for the global equilibrium solution during the entire system runtime is based on the proximal policy optimization algorithm and the actor-critic framework; the importance weights, historical data volume, historical decisions, and coalition status are comprehensively considered through the state space of the reinforcement learning process.

Images