Digital twin network trusted scheduling method and device

Abstract

The invention discloses a digital twin network trusted scheduling method and device, and relates to the technical field of network scheduling, and the method comprises the steps: constructing a digital twin network system; establishing a diffusion block chain mechanism to guarantee credible scheduling; performing model construction based on a preset constraint condition, a preset total delay function and a preset block chain throughput function to obtain an optimization model for digital twin network scheduling; performing model reconstruction on the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling; and optimizing the target optimization model by adopting a preset multi-agent deep reinforcement learning model based on observation state information of each terminal device in the digital twin network system acquired in real time to obtain a target parameter value corresponding to each to-be-optimized parameter. According to the method, the task execution speed of the terminal equipment can be improved and the resource utilization efficiency can be improved while network scheduling is ensured to be safe and credible.

Description

Technical Field

[0001] The present invention relates to the field of network scheduling technology, and in particular to a digital twin network trusted scheduling method, device, storage medium and electronic device. Background Art

[0002] Thanks to the advancement of new-generation information technologies such as 5G, big data, and artificial intelligence, the implementation of digital twins has gradually become possible and has been widely used in scenarios such as end-edge collaborative computing. However, most terminal devices are still limited by manufacturing costs and internal space, and their computing and cache resources are still too limited to handle a large number of complex tasks and cannot fully meet people's growing needs. In order to meet this challenge, deploying edge servers with wide-range communication coverage, high computing power, and large-capacity cache capabilities to establish digital twin networks to collaboratively process computing tasks generated by terminal devices has become a common solution.

[0003] In addition, the dynamic and open nature of the digital twin network makes it impossible to guarantee the trust between the edge server and the terminal device. The task scheduling delay is not fully considered during the task scheduling process, resulting in low task processing efficiency and high cost. Summary of the invention

[0004] In view of this, the present invention provides a digital twin network trusted scheduling method, device, storage medium and electronic device, the main purpose of which is to solve the current problems of low efficiency and high cost in the network scheduling process of the digital twin network system.

[0005] To solve the above problems, the present application provides a digital twin network trusted scheduling method, including:

[0006] Build a digital twin network system;

[0007] Based on the digital twin network system, a diffusion blockchain mechanism is established to enable all edge servers and terminal devices to join the blockchain to ensure the security and reliability of network scheduling;

[0008] A model is constructed based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0009] Reconstructing the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0010] Based on the observation status information of each terminal device in the digital twin network system collected in real time, the target optimization model is optimized using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0011] To solve the above problems, the present application provides a digital twin network trusted scheduling device, including:

[0012] System building modules for building digital twin network systems;

[0013] A diffusion blockchain mechanism building module is used to establish a diffusion blockchain mechanism based on the digital twin network system, so that each edge server and terminal device joins the blockchain to ensure the security and reliability of network scheduling;

[0014] A model building module, used to build a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0015] A model reconstruction module, used to reconstruct the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0016] The optimization module is used to optimize the target optimization model based on the observation status information of each terminal device in the digital twin network system collected in real time using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0017] To solve the above problems, the present application provides a storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned digital twin network trusted scheduling method.

[0018] To solve the above problems, the present application provides an electronic device, comprising at least a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program on the memory, the steps of the above-mentioned digital twin network trusted scheduling method are implemented.

[0019] Beneficial effects of this application: This application is aimed at the end-to-end collaborative scheduling of tasks and resources and blockchain performance optimization in highly dynamic and open digital twin network scenarios. It fully considers constraints such as task migration, communication bandwidth, computing frequency, cache capacity, task deadline, and blockchain runtime latency, and formulates a joint optimization problem to jointly optimize task migration ratio, communication bandwidth allocation, computing frequency allocation, cache decision, block size, and block generation interval to minimize task trusted processing latency and maximize blockchain transaction throughput. For traditional high-overhead blockchain solutions, a digital twin-assisted lightweight diffusion delegated Byzantine fault-tolerant blockchain solution is proposed. By migrating the identity authentication process to the digital twin layer, the switching authentication overhead caused by mobility is reduced, and the blockchain consensus process is optimized to reduce the communication overhead in the consensus process. In view of the modeling difficulties and algorithm state space explosion problems caused by the complex coupling of multi-dimensional resources and blockchain parameters, it is converted into a multi-agent Markov decision process problem, and it is proposed to use a multi-agent deep reinforcement learning model to seek the optimal scheduling strategy to improve the overall performance of the digital twin network.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0022] Figure 1 A schematic diagram of a process of a digital twin network trusted scheduling method provided in an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of a scenario of a digital twin network system based on edge computing assisted by a digital twin in an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of a lightweight blockchain with diffused delegated Byzantine fault tolerance provided by an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of the algorithm structure of a dual actor-critic algorithm based on maximum entropy according to an embodiment of the present application is shown;

[0026] Figure 5A structural block diagram of a digital twin network trusted scheduling method device according to another embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0028] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0029] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0030] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0031] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0032] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0033] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0034] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0035] The present application embodiment provides a digital twin network trusted scheduling method, such as Figure 1 As shown, including:

[0036] Step S101: constructing a digital twin network system;

[0037] In the specific implementation process of this step, the digital twin network system includes multiple edge servers and multiple terminal devices. The edge servers are powered by the power grid and connected to each other by wires. The multiple edge servers include edge computing servers, base stations, and blockchain node servers; the edge computing servers provide edge computing and content caching services; the base station servers provide communication services; the blockchain node servers provide security for communication, computing and caching services; the terminal devices generate tasks and perform local calculations on the tasks, and support migrating the tasks to the edge servers through wireless channels for edge computing; the digital twins are deployed on the edge servers of the digital twin network system, and are characterized as virtualized models established for edge servers and terminal devices in the network, which are used to evaluate the operating status and resource status of the digital twin network system. It can support the training of deep reinforcement learning methods to carry out end-to-end collaborative scheduling of resources and tasks. It can also be used to execute the consensus process between blockchain nodes.

[0038] Step S102: Based on the digital twin network system, a diffusion blockchain mechanism is established so that each edge server and each terminal device joins the blockchain to ensure the security and reliability of network scheduling;

[0039] In the specific implementation process of this step, in the blockchain consensus stage, each edge server and each terminal device are added to the blockchain system, so that the edge computing process is based on the diffusion blockchain mechanism to achieve secure and reliable network scheduling.

[0040] Step S103: constructing a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0041] In the specific implementation process of this step, the blockchain consensus stage includes the migration and calculation stage, the pre-preparation stage, the preparation stage, the maintenance stage and the block generation interval stage. The preset constraints include: task migration ratio constraints, cache constraints, bandwidth ratio constraints, first computing resource constraints, second computing resource constraints, deadline constraints and blockchain stability constraints; the online transaction stability of the blockchain is fully considered. Specifically, based on the preset total delay function and the preset blockchain throughput function, an optimization model for digital twin network scheduling is constructed with the goal of minimizing the task trusted processing delay and maximizing the blockchain throughput. The optimization model for digital twin network scheduling carries various parameters to be optimized. Each of the parameters to be optimized includes: task migration ratio, bandwidth allocation ratio, computing resource allocation ratio, cache decision, and blockchain variable decision. The present application adopts a diffused Byzantine fault-tolerant blockchain construction method, which reduces the switching authentication overhead caused by the movement of terminal devices by migrating the identity authentication process to the digital twin layer, while optimizing the blockchain consensus process and reducing the communication overhead in the consensus process.

[0042] Step S104: reconstructing the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0043] In the specific implementation process of this step, a function is constructed based on the preset security parameters and the stability constraints to obtain a blockchain stability penalty function; a function is constructed based on the cache decision parameters of content popularity estimation, the data volume parameters of cache content, the maximum cache capacity parameters of the edge server, the storage resource estimation deviation parameters, the content cache target decision parameters and the cache constraints to obtain a cache penalty function; a function is constructed based on the blockchain stability penalty function and the cache penalty function to obtain a target penalty function; a function is constructed based on the preset task trusted processing delay weight coefficient, the blockchain transaction throughput weight coefficient, the preset total delay function and the preset blockchain throughput function to obtain a reward function; a model is constructed based on the reward function, the agent set, the system state set, the action set and the state transition probability of the digital twin network system to obtain the preset multi-agent Markov decision process model; the optimization model of digital twin network scheduling is transformed based on the preset multi-agent Markov decision process model and the reward function to obtain a target optimization model for digital twin network scheduling.

[0044] Step S105: Based on the observation status information of each terminal device in the digital twin network system collected in real time, the target optimization model is optimized using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0045] During the specific implementation of this step, the observation state information of each terminal device in the digital twin network system obtained in real time is predicted using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​that meet each observation state information; based on each target parameter value, the target digital twin network scheduling optimization model is used to calculate to obtain a target reward value; the target parameter value is executed to obtain the target observation state information of each terminal device in the next state; each observation state information, each target parameter value, the target reward value and the target observation state information are entered into a preset experience database; this lays the foundation for the subsequent offline update of the preset multi-agent deep reinforcement learning model in specific applications.

[0046] This application is aimed at the end-to-end collaborative scheduling of tasks and resources and blockchain performance optimization in highly dynamic and open digital twin network scenarios. It fully considers constraints such as task migration, communication bandwidth, computing frequency, cache, task deadline, task processing delay, and blockchain stability, and formulates a joint optimization problem to jointly optimize the task migration ratio, communication bandwidth allocation, computing frequency allocation, cache decision, block size, and block generation interval to minimize the task trusted processing delay and maximize the blockchain transaction throughput. For traditional high-overhead blockchain solutions, a lightweight diffuse delegated Byzantine fault-tolerant blockchain solution assisted by digital twins is proposed. By migrating the identity authentication process to the digital twin layer, the switching authentication overhead caused by mobility is reduced, and the blockchain consensus process is optimized to reduce the communication overhead in the consensus process. In view of the modeling difficulties and algorithm state space explosion problems caused by the complex coupling of multi-dimensional resources and blockchain parameters, it is converted into a multi-agent Markov decision process problem, and a deep reinforcement learning algorithm is proposed to seek the optimal scheduling strategy to improve the overall performance of the digital twin network.

[0047] Another embodiment of the present application provides another digital twin network trusted scheduling method, including:

[0048] Step S201: constructing a digital twin network system;

[0049] In the specific implementation process of this step, Figure 2The digital twin network system based on edge computing assisted by the digital twin of the present application is shown as a scene diagram. The digital twin network system includes multiple edge servers and multiple terminal devices. The edge servers are powered by the power grid and are connected to each other by wires. The multiple edge servers include edge computing servers, base stations, and blockchain node servers; the edge computing servers provide edge computing and content caching services; the base station servers provide communication services; the blockchain node servers provide security for communication, computing and caching services; the terminal devices generate tasks and perform local calculations on the tasks, and support migrating the tasks to the edge servers through wireless channels for edge computing; the digital twins are deployed on the edge servers and are characterized as virtualized models established for the edge servers and terminal devices in the network, which are used to evaluate their operating status and resource status. It can support the training of deep reinforcement learning methods to carry out end-to-end collaborative scheduling of the network. It can also be used to execute the consensus process between blockchain nodes.

[0050] Step S202: Constructing preset constraints corresponding to the task scheduling phase of the digital twin network system, the resource scheduling phase of the digital twin network system, and the blockchain consensus phase respectively;

[0051] In the specific implementation process of this step, the blockchain consensus phase includes the migration and calculation phase, the pre-preparation phase, the preparation phase, the maintenance phase, and the block generation interval phase. Figure 3 The schematic diagram of the lightweight blockchain with diffuse delegation and Byzantine fault tolerance of the present application is shown, and the preset constraints include: task migration ratio constraint, cache constraint, bandwidth ratio constraint, first computing resource constraint, second computing resource constraint, deadline constraint and blockchain stability constraint. Specifically, based on the task ratio of each terminal device migrated to each edge server, the constraint is constructed to obtain the task migration ratio constraint; the mathematical expression of the task migration ratio constraint can be shown as the following formula (1):

[0052]

[0053] Among them, u m,n It indicates the proportion of tasks migrated from terminal device m to edge server n. When n=0, it means that this part of the tasks is calculated locally.

[0054] Constraints are constructed based on the cache decision parameters of content popularity estimation, the data volume parameters of cached content, the maximum cache capacity parameters of the edge server, the storage resource estimation deviation parameters, and the content cache target decision parameters to obtain cache constraints. The mathematical expression of the cache constraints can be expressed as the following formula (2):

[0055]

[0056] Among them, w k,m is the cache decision parameter for content popularity estimation; when w k,m = 1, content k is popular enough for terminal device m, and it is recommended to be cached by edge server n; the final algorithm content caching target decision parameter for content k is v n,k ; is the data volume parameter of cache content k, S n is the maximum cache capacity parameter of edge server n, ΔS n It is the storage resource estimation deviation parameter, that is, the cached content size cannot exceed the maximum storage capacity limit.

[0057] Based on the bandwidth ratio parameter allocated by the edge server to each of the terminal devices, a constraint condition is constructed to obtain a bandwidth ratio constraint condition; the mathematical expression of the bandwidth ratio constraint condition can be expressed as the following formula (3):

[0058]

[0059] Among them, b m,n It is the bandwidth ratio parameter allocated by edge server n to terminal device m.

[0060] Based on the edge computing resource parameters estimated by the preset digital twin layer and allocated to each terminal device by the edge server, the computing resource estimation deviation parameters of the digital twin layer, and the maximum computing frequency parameters of the edge server, the constraint conditions are constructed to obtain the first computing resource constraint condition and the second computing resource constraint condition; the mathematical expression of the first computing resource constraint condition can be expressed as the following formula (4):

[0061]

[0062] The mathematical expression of the second computing resource constraint condition can be expressed as the following formula (5):

[0063]

[0064] Among them, f m，n represents the edge computing resource parameter allocated by edge server n to terminal device m estimated by the digital twin layer; Δf m,n represents the computational resource estimation bias parameter of the digital twin layer; Indicates the maximum computing frequency parameter of edge server n.

[0065] Based on the deadline parameters of the tasks executed by each of the terminal devices, constraints are constructed to obtain task deadline constraints. The mathematical expression of the task deadline constraints can be expressed as the following formula (6):

[0066]

[0067] in, It represents the deadline parameter of the task executed by the terminal device m, that is, the longest task processing delay that the terminal device m can accept;

[0068] Based on the blockchain runtime delay parameters and block generation interval parameters, the constraint conditions are constructed to obtain the blockchain stability constraint conditions. The mathematical expression of the stability constraint conditions can be expressed as the following formula (7):

[0069] C7:T LTF ≤ωT BI (7);

[0070] Among them, T LTF is the blockchain runtime delay parameter, T BI is the block generation interval parameter, ω is an integer representing a number of intervals, that is, a block should be published and verified within multiple consecutive block generation intervals.

[0071] The blockchain consists of full nodes acted by edge servers and lightweight nodes acted by terminal devices: full nodes are responsible for recording transaction information during communication, calculation and caching of blocks, and participate in the decentralized consensus process; lightweight nodes do not participate in the blockchain consensus process, but can use blockchain to verify the credibility of edge servers when moving between different edge servers.

[0072] Step S203: constructing a preset total delay function of the digital twin network system;

[0073] In the specific implementation process of this step, a function is constructed based on the pre-preparation delay parameter, the preparation delay parameter and the holding delay parameter to obtain a consensus delay function; the mathematical expression of the consensus delay function can be expressed as the following formula (8):

[0074] T BC =T p-prep +T prep +T pers (8);

[0075] Among them, T p-prep To prepare the delay parameters, T prep To prepare the delay parameters, T pers To maintain the delay parameters;

[0076] The mathematical expression of the pre-preparation delay parameter can be expressed as follows:

[0077]

[0078] Among them, the pre-preparation delay parameter T p-prep and transaction data volume parameter D tr 、Block data volume parameter D block , verify the required calculation frequency parameter C ver and the transmission rate parameter R between the edge servers rsu Related.

[0079] The mathematical expression of the preparation delay parameter can be expressed as follows:

[0080]

[0081] Among them, C ver Indicates the calculation frequency required for validity verification.

[0082] The mathematical expression of the hold delay can be expressed as follows:

[0083]

[0084] Based on the consensus delay function and the block generation interval delay parameter, a function is constructed to obtain a blockchain runtime delay function; the mathematical expression of the blockchain runtime delay function can be expressed as the following formula (12):

[0085] T LTF =T BC +T BI (12);

[0086] Based on the preset result download delay function, the preset task calculation delay function, and the preset task migration delay function, a function is constructed to obtain an edge computing delay function; the mathematical expression of the edge computing delay function can be shown as follows:

[0087]

[0088] The task migration delay function The mathematical expression of can be expressed as follows:

[0089]

[0090] Among them, u m,n Indicates the migration ratio, represents the task size, R m,n Indicates the transmission rate.

[0091] The result download delay function The mathematical expression of can be expressed as follows:

[0092]

[0093] in, Indicates the amount of result data and R m,n Indicates the transmission rate.

[0094] The preset task calculates the delay function The mathematical expression of can be expressed as follows:

[0095]

[0096] in, is the edge computing delay deviation, To estimate the edge computing latency. m Indicates the required calculation frequency of the task.

[0097] A function is constructed based on the estimated local calculation delay parameter and the local calculation delay deviation parameter to obtain a local calculation delay function; the mathematical expression of the local calculation delay function can be expressed as the following formula (17):

[0098]

[0099] in, To estimate the local computation latency, Calculate the delay deviation locally.

[0100] Based on the consensus delay function, the blockchain operation delay function, the edge computing delay function and the local computing delay function, a function is constructed to obtain the preset total delay function. The mathematical expression of the preset total delay function can be expressed as the following formula (18):

[0101]

[0102] in, is the edge computing delay of task m, which is determined by the cache decision and the edge computing delay of all edge servers n. When edge server n decides to cache the content k requested by the terminal device, that is, v n,k When =1, we only need to download the result from the edge server n. Otherwise, we need to consider the delay of task migration, calculation and result download at the same time.

[0103] In the process of constructing the total delay function in this application, the blockchain consensus mechanism mainly includes the following steps:

[0104] Step 1: Pre-preparation phase, first, the terminal device offloads the task to the target edge server, and the edge server records the transaction information of task offloading, calculation and cache. Then, all transactions are uploaded to a randomly selected delegated full node. For a transaction, the full node first verifies its signature. If valid, the full node will execute the smart contract, and all transactions will be packaged to generate a new block.

[0105] Step 2: In the preparation phase, using the generated new block, the delegated full node broadcasts the signed block as a new block proposal to other primary full nodes participating in edge computing for verification. The block proposal is transmitted together with the pre-prepare message, which contains the ID, signature, and hash result of the new block of the delegated full node. When the primary full nodes receive the pre-prepare message and the new block, they first verify the signature and MAC of the block, then verify the block through the smart contract, and finally return the vote for the block proposal to the delegated full node. If the new block proposal is accepted by the majority of the primary full nodes, that is, the number of votes is not less than the threshold, the blockchain consensus will enter the next stage.

[0106] Step 3: In the maintenance phase, after the primary full node votes, the new block will be added to the chain for update. Then, the delegated full node will spread this block to all other primary full nodes, including secondary full nodes that do not participate in edge computing.

[0107] The edge computing process is mainly for tasks generated by a single terminal device, which can be selected to be not migrated, partially migrated, or fully migrated to one or more edge servers for computing.

[0108] Step S204: constructing the preset blockchain throughput function;

[0109] In the specific implementation process of this step, a function is constructed based on the transaction quantity parameter, the block data volume parameter and the transaction data volume parameter to obtain a new block generation quantity function; the mathematical expression of the new block generation quantity function can be expressed as the following formula (19):

[0110]

[0111] Among them, L all Represents the number of transactions, expressed by formula (20):

[0112]

[0113] Based on the new block generation quantity function, block data volume parameter, transaction data volume parameter, block generation interval parameter, and preset transmission rate parameter between edge servers, a function is constructed to obtain the preset blockchain throughput function. The mathematical expression of the preset blockchain throughput function can be expressed as the following formula (21):

[0114]

[0115] in, The throughput of the blockchain is represented by the number of transactions contained in the unit block, T BI Indicates the block generation interval, L block Indicates the number of blocks generated, R rsu Indicates the transmission rate between edge servers.

[0116] Step S205: Based on the digital twin network system, a diffusion blockchain mechanism is established so that each edge server and each terminal device joins the blockchain to ensure the security and reliability of network scheduling;

[0117] In the specific implementation process of this step, in the blockchain consensus stage, each terminal device and each edge server are added to the blockchain system, so that the edge computing process is based on the diffusion blockchain mechanism to achieve secure and reliable scheduling within the network.

[0118] Step S206: constructing a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0119] In the specific implementation process of this step, the mathematical expression of the optimization model of the digital twin network scheduling can be expressed as follows:

[0120]

[0121] represents minimizing the trusted processing delay of tasks and maximizing the throughput of blockchain, T represents the total trusted processing delay of the entire digital twin network task, H represents the throughput of blockchain, is a set of parameters to be optimized, which respectively represent the task migration ratio, bandwidth allocation ratio, computing resource allocation ratio, cache decision, and blockchain variable decision.

[0122] Step S207: constructing a function based on the preset security parameters and the stability constraint conditions to obtain a blockchain stability penalty function;

[0123] In the specific implementation process of this step, the mathematical expression of the blockchain stability penalty function can be expressed as the following formula (23):

[0124]

[0125] Among them, ξ is a negative value given according to safety requirements.

[0126] Step S208: constructing a function based on the cache decision parameter of the content popularity estimate, the data volume parameter of the cache content, the maximum cache capacity parameter of the edge server, the storage resource estimation deviation parameter, the content cache target decision parameter and the cache constraint condition to obtain a cache penalty function;

[0127] In the specific implementation of this step, the mathematical expression of the cache penalty function can be expressed as follows:

[0128]

[0129] Step S209: constructing a function based on the blockchain stability penalty function and the cache penalty function to obtain a target penalty function;

[0130] In the specific implementation process of this step, the mathematical expression of the target penalty function can be expressed as follows:

[0131]

[0132] Step S210: constructing a function based on the preset task trusted processing delay weight coefficient, the blockchain transaction throughput weight coefficient, the preset total delay function and the preset blockchain throughput function to obtain a reward function;

[0133] In the specific implementation of this step, the mathematical expression of the reward function can be expressed as follows:

[0134]

[0135] Among them, α and β are weight coefficients that balance the task trusted processing delay and blockchain transaction throughput according to the specific task.

[0136] Step S211: constructing a model based on the reward function, the agent set, the system state set, the action set and the state transition probability of the digital twin network system to obtain the preset multi-agent Markov decision process model;

[0137] In the specific implementation process of this step, the preset multi-agent Markov decision process model consists of a tuple Definition, where is a collection of agents, is the set of observed system states, is a collection of actions. is the state transition probability, is the reward function. The agent set is represented as Responsible for observing the state space and learning better action strategies to maximize the reward. The observation space includes data size, required computing resource frequency, task deadline, communication bandwidth, cache storage capacity and estimated deviation, computing resources and their estimated deviation, and signal-to-noise ratio. The set of observed system states can be expressed as the following formula (27):

[0138]

[0139] in, C(t)={C1,...,C M , C ver , C val}, indicating the data size of cache contents, tasks, and transactions; Indicates the task deadline. Indicates the maximum computing frequency of the terminal device; ΔF end (t)={Δf1,Δf2,...,Δf M}and Represents the computational resource estimation bias of the digital twin. γ n (t) = {γ 1，n , γ 2，n , ..., γ M，n} represents the signal-to-noise ratio.

[0140] In addition, the mathematical expression of the total state space of all agents can be expressed as follows:

[0141] O(t)={O1(t),O2(t),...,O N (t)} (28);

[0142] The action space is the action performed by the agent at time t. The mathematical expression of the action space can be expressed as follows:

[0143]

[0144] Among them, u n (t) = {u 1，n ,u 2，n , ..., u M，n} represents the task migration ratio, b n (t) = {b 1，n , b 2，n , ..., b M，n} represents the bandwidth allocation ratio, f n (t) = {f 1，n , f 2，n , ..., f M，n} represents the computing resource allocation ratio, v n (t) = {v 1，n , v 2，n , ..., v K，n} is the cache decision. The action space set of all agents is defined as A(t) = {A1(t), A2(t), ..., A N (t)}.

[0145] The state transition probability describes the probability that O(t) is converted to O(t+1) when the current agent performs operation A(t). The transition probability from the current state to the next state can be expressed as follows:

[0146] pr(O(t+1)|O(t),A(t))=∫ O(t+1) f(O(t), A(t), s′)ds′ (30); where f() is the probability density function of state transition.

[0147] Step S212: performing model transformation on the optimization model of the digital twin network scheduling based on the preset multi-agent Markov decision process model and the reward function to obtain a target optimization model of the digital twin network scheduling;

[0148] In the specific implementation process of this step, the mathematical expression of the target optimization model of digital twin network scheduling can be expressed as follows:

[0149]

[0150] It means maximizing the long-term cumulative reward under the condition of satisfying the constraints, so as to minimize the task trusted processing delay and maximize the blockchain throughput.

[0151] Step S213: construct a preset multi-agent deep reinforcement learning model;

[0152] In the specific implementation process of this step, Figure 4The figure shows the algorithm structure diagram of the present application for training the preset multi-agent deep reinforcement learning model based on the maximum entropy dual actor-critic algorithm. A first initial critic network model, a second initial critic network model and an initial actor network model are constructed respectively; based on historical experience data, a preset first loss function, a preset second loss function, a preset third loss function and a preset entropy regularization loss function, calculations are performed to obtain various model parameters, each of which includes a first model parameter corresponding to the first initial critic network model, a second model parameter corresponding to the second initial critic network model, a third model parameter corresponding to the initial actor network model and a regularization coefficient corresponding to the preset entropy regularization loss function; specifically, based on historical experience data, the first loss function of the first initial critic network model is used for calculations to obtain the first model parameter σ1 when the loss value of the first loss function is the smallest; wherein the mathematical expression of the first loss function can be expressed as the following formula (32):

[0153]

[0154] Where ρ is the discount factor, is the first state value function, It is the first initial critic network model; is the expected value calculation function; the mathematical expression of the first state value function can be expressed as follows:

[0155]

[0156] Based on the historical experience data, the second loss function of the second initial critic network model is used for calculation and processing to obtain the second model parameter σ2 when the loss value of the second loss function is minimum; wherein the mathematical expression of the second loss function can be expressed as the following formula (34):

[0157]

[0158] Where ρ is the discount factor, is the second state value function. The mathematical expression of the second state value function can be expressed as follows:

[0159]

[0160] Based on the first model parameter and the second model parameter, a target model parameter is obtained by screening; specifically, the first model parameter and the second model parameter are compared to determine the smaller model parameter as the target model parameter. Based on the target model parameter and the historical experience data, a third loss function of the initial actor network model is used for calculation and processing to obtain a third model parameter θ when the loss value of the third loss function is the smallest; the mathematical expression of the third loss function can be shown as the following formula (36):

[0161]

[0162] Based on the historical experience data, a preset entropy regularization loss function is used for calculation and processing to obtain a regularization coefficient when the loss value of the preset entropy regularization loss function is the smallest. The mathematical expression of the preset entropy regularization loss function can be shown as the following formula (37):

[0163]

[0164] in, is the policy entropy of maximum entropy reinforcement learning, and E0 is the target entropy;

[0165] Based on the first model parameter, the second model parameter, the third model parameter and the regularization coefficient, the model is updated to obtain the first current critic network model, the second current critic network model, the current actor model and the current entropy regularization loss function; specifically, based on the first model parameter, the first initial critic network model is updated to obtain the first current critic network model; based on the second model parameter, the second initial critic network model is updated to obtain the second current critic network model; based on the third model parameter, the initial actor network model is updated to obtain the current actor model; based on the regularization coefficient, the preset entropy regularization loss function is updated to obtain the current entropy regularization loss function. Specifically, the following soft update function can be used to update each model, and the mathematical expression of the soft update function can be shown as follows:

[0166]

[0167] The first current critic network model, the second current critic network model, the current actor model and the current entropy regularization loss function are iteratively updated in a loop until the reward function value obtained by optimizing the target optimization model of the digital twin network scheduling using the updated current actor model to perform state prediction on each of the historical experience data meets the preset conditions, and the updated current actor network model is determined as the target actor network model.

[0168] Step S214: Based on the observation status information of each terminal device in the digital twin network system collected in real time, a preset multi-agent deep reinforcement learning model is used to optimize the optimization model of the target digital twin network scheduling to obtain the target parameter values ​​corresponding to each parameter to be optimized.

[0169] During the specific implementation of this step, the observation state information of each terminal device in the digital twin network system obtained in real time is predicted using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​that meet each observation state information; based on each target parameter value, the target digital twin network scheduling optimization model is used to calculate to obtain a target reward value; the target parameter value is executed to obtain the target observation state information of each terminal device in the next state; each observation state information, each target parameter value, the target reward value and the target observation state information are entered into a preset experience database; this lays the foundation for the subsequent offline update of the preset multi-agent deep reinforcement learning model in specific applications.

[0170] Another embodiment of the present application provides a digital twin network trusted scheduling device, such as Figure 5 As shown, including:

[0171] System building module 1, used to build a digital twin network system;

[0172] The diffusion blockchain mechanism construction module 2 is used to establish a diffusion blockchain mechanism based on the digital twin network system, so that each edge server and each terminal device joins the blockchain to ensure the security and reliability of network scheduling;

[0173] A model building module 3 is used to build a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0174] A model reconstruction module 4 is used to reconstruct the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0175] The optimization module 5 is used to optimize the target optimization model based on the observation status information of each terminal device in the digital twin network system collected in real time using a preset multi-agent deep reinforcement learning model to obtain the target parameter value corresponding to each parameter to be optimized.

[0176] In the specific implementation process, the diffusion blockchain mechanism construction module 2 is specifically used for: in the blockchain consensus stage, each edge server and each terminal device are added to the blockchain system, so that the edge computing process is based on the diffusion blockchain mechanism to achieve secure and reliable network scheduling. In the specific implementation process, the device also includes: a constraint construction module, and the constraint construction module is specifically used for: constructing preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, specifically including: constructing constraints based on the proportion of tasks migrated from each terminal device to each edge server to obtain task migration proportion constraints; constructing constraints based on cache decision parameters estimated by content popularity, data volume parameters of cached content, maximum cache capacity parameters of edge servers, storage resource estimation deviation parameters, and content cache target decision parameters to obtain cache constraints; constructing constraints based on bandwidth ratio parameters allocated to each terminal device by the edge server. The constraint conditions are constructed to obtain the bandwidth allocation ratio constraint conditions; the constraint conditions are constructed based on the edge computing resource parameters allocated to each terminal device by the edge server estimated in the preset digital twin layer, the computing resource estimation deviation parameters of the digital twin layer, the maximum computing frequency parameters of the edge server, the computing resource parameters estimated by the digital twin layer for the terminal device itself, the computing resource estimation deviation parameters of the digital twin layer and the maximum computing frequency parameters of the terminal device, to obtain the first computing resource constraint condition and the second computing resource constraint condition; the constraint conditions are constructed based on the deadline parameters of the tasks executed by each of the terminal devices to obtain the task deadline constraint condition; the constraint conditions are constructed based on the blockchain runtime delay parameters and the block generation interval parameters to obtain the blockchain stability constraint condition.

[0177] In the specific implementation process, the device also includes: a preset total delay function construction module, and the preset total delay function construction module is specifically used to: construct a function based on the pre-preparation delay parameter, the preparation delay parameter and the holding delay parameter to obtain a consensus delay function; construct a function based on the consensus delay function and the block generation interval delay parameter to obtain a blockchain operation delay function; construct a function based on the preset task calculation delay function, the preset task migration delay function, and the preset result download delay function to obtain an edge computing delay function; construct a function based on the estimated local computing delay parameter, the local computing delay deviation parameter and the local task computing delay function to obtain a local computing delay function; construct a function based on the consensus delay function, the blockchain operation delay function, the edge computing delay function and the local computing delay function to obtain the preset total delay function.

[0178] In the specific implementation process, the device also includes: a preset blockchain throughput function construction module, and the preset blockchain throughput function construction module is specifically used to: construct a function based on a transaction quantity parameter, a block data volume parameter, and a transaction data volume parameter to obtain a new block generation quantity function; construct a function based on the new block generation quantity function, the block data volume parameter, the transaction data volume parameter, the block generation interval parameter, and a preset transmission rate parameter between edge servers to obtain the preset blockchain throughput function.

[0179] In the specific implementation process, the model construction module 3 is specifically used to: construct a function based on the preset security parameters and the stability constraints to obtain a blockchain stability penalty function; construct a function based on the cache decision parameters estimated by content popularity, the data volume parameters of the cache content, the maximum cache capacity parameters of the edge server, the storage resource estimation deviation parameters, the content cache target decision parameters and the cache constraints to obtain a cache penalty function; construct a function based on the blockchain stability penalty function and the cache penalty function to obtain a target penalty function; construct a function based on the preset task trusted processing delay weight coefficient, the blockchain transaction throughput weight coefficient, the preset total delay function and the preset blockchain throughput function to obtain a reward function; construct a model based on the reward function, the agent set, the system state set, the action set and the state transition probability of the digital twin network system to obtain the preset multi-agent Markov decision process model; based on the preset multi-agent Markov decision process model and the reward function, the optimization model of the digital twin network scheduling is transformed to obtain the target optimization model of the digital twin network scheduling.

[0180] In the specific implementation process, the device also includes: a preset multi-agent deep reinforcement learning model construction module, and the preset multi-agent deep reinforcement learning model construction module is specifically used to: respectively construct a first initial critic network model, a second initial critic network model and an initial actor network model; based on historical experience data, a preset first loss function, a preset second loss function, a preset third loss function and a preset entropy regularization loss function, calculate and process to obtain each model parameter, each of the model parameters includes a first model parameter corresponding to the first initial critic network model, a second model parameter corresponding to the second initial critic network model, a third model parameter corresponding to the initial actor network model and a corresponding to the preset entropy regularization loss function. regularization coefficient; updating the model based on the first model parameter, the second model parameter, the third model parameter and the regularization coefficient to obtain a first current critic network model, a second current critic network model, a current actor model and a current entropy regularization loss function; iteratively updating the first current critic network model, the second current critic network model, the current actor model and the current entropy regularization loss function until the reward function value obtained by optimizing the target optimization model using each state parameter value obtained by state prediction of each historical experience data using the updated current actor model meets the preset conditions, and determining the updated current actor network model as the preset multi-agent deep reinforcement learning model.

[0181] In the specific implementation process, the preset multi-agent deep reinforcement learning model construction module is also used to: based on the historical experience data, use the first loss function of the first initial critic network model for calculation and processing, and obtain the first model parameters when the loss value of the first loss function is minimized; based on the historical experience data, use the second loss function of the second initial critic network model for calculation and processing, and obtain the second model parameters when the loss value of the second loss function is minimized; based on the first model parameters and the second model parameters, perform screening to obtain the target model parameters; based on the target model parameters and the historical experience data, use the third loss function of the initial actor network model for calculation and processing, and obtain the third model parameters when the loss value of the third loss function is minimized; based on the historical experience data, use the preset entropy regularization loss function for calculation and processing, and obtain the regularization coefficient when the loss value of the preset entropy regularization loss function is minimized.

[0182] This application is aimed at the end-to-end collaborative scheduling of tasks and resources and blockchain performance optimization in highly dynamic and open digital twin network scenarios. It fully considers constraints such as task migration, communication bandwidth, computing frequency, cache capacity, task deadline, and blockchain runtime latency, and formulates a joint optimization problem to jointly optimize the task migration ratio, communication bandwidth allocation, computing frequency allocation, cache decision, block size, and block generation interval to minimize the task trusted processing latency and maximize the blockchain transaction throughput. For traditional high-overhead blockchain solutions, a lightweight diffuse delegated Byzantine fault-tolerant blockchain solution assisted by digital twins is proposed. By migrating the identity authentication process to the digital twin layer, the switching authentication overhead caused by mobility is reduced, and the blockchain consensus process is optimized to reduce the communication overhead during the consensus process. In view of the modeling difficulties and algorithm state space explosion problems caused by the complex coupling of multi-dimensional resources and blockchain parameters, it is transformed into a multi-agent Markov decision process problem, and it is proposed to use the target actor network model to seek the optimal scheduling strategy to improve the overall performance of the digital twin network.

[0183] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0184] Step 1: Build a digital twin network system;

[0185] Step 2: Based on the digital twin network system, a diffusion blockchain mechanism is established to enable each edge server and each terminal device to join the blockchain to ensure the security and reliability of network scheduling;

[0186] Step 3: construct a model based on the preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, the preset total delay function of the digital twin network system, and the preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0187] Step 4: reconstruct the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0188] Step 5: Based on the observation status information of each terminal device in the digital twin network system collected in real time, the target optimization model is optimized using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0189] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0190] The specific implementation process of the above method steps can be found in any of the above-mentioned embodiments of the trusted scheduling method for the digital twin network, and this embodiment will not be repeated here.

[0191] This application is aimed at the end-to-end collaborative scheduling of tasks and resources and blockchain performance optimization in highly dynamic and open digital twin network scenarios. It fully considers constraints such as task migration, communication bandwidth, computing frequency, cache capacity, task deadline, and blockchain runtime latency, and formulates a joint optimization problem to jointly optimize the task migration ratio, communication bandwidth allocation, computing frequency allocation, cache decision, block size, and block generation interval to minimize the task trusted processing latency and maximize the blockchain transaction throughput. For traditional high-overhead blockchain solutions, a lightweight diffuse delegated Byzantine fault-tolerant blockchain solution assisted by digital twins is proposed. By migrating the identity authentication process to the digital twin layer, the switching authentication overhead caused by mobility is reduced, and the blockchain consensus process is optimized to reduce the communication overhead during the consensus process. In view of the modeling difficulties and algorithm state space explosion problems caused by the complex coupling of multi-dimensional resources and blockchain parameters, it is transformed into a multi-agent Markov decision process problem, and it is proposed to use the target actor network model to seek the optimal scheduling strategy to improve the overall performance of the digital twin network.

[0192] Another embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the following method steps when executing the computer program in the memory:

[0193] Step 1: Build a digital twin network system;

[0194] Step 2: Based on the digital twin network system, a diffusion blockchain mechanism is established to enable each edge server and each terminal device to join the blockchain to ensure the security and reliability of network scheduling;

[0195] Step 3: construct a model based on the preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, the preset total delay function of the digital twin network system, and the preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized;

[0196] Step 4: reconstruct the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling;

[0197] Step 5: Based on the observation status information of each terminal device in the digital twin network system collected in real time, the target optimization model is optimized using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0198] The specific implementation process of the above method steps can be found in any of the above-mentioned embodiments of the trusted scheduling method for the digital twin network, and this embodiment will not be repeated here.

[0199] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A digital twin network trusted scheduling method, characterized in that: include: Build a digital twin network system; Based on the digital twin network system, a diffusion blockchain mechanism is established to enable each edge server and each terminal device to join the blockchain to ensure the security and reliability of network scheduling; A model is constructed based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized; Reconstructing the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling; Based on the observation status information of each terminal device in the digital twin network system collected in real time, the target optimization model is optimized using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

2. The method according to claim 1, characterized in that In the blockchain consensus stage, each edge server and each terminal device are added to the blockchain system, so that the edge computing process is based on the diffusion blockchain mechanism to achieve secure and reliable network scheduling.

3. The method according to claim 1, characterized in that Before constructing a model based on the preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, the preset total delay function of the digital twin network system, and the preset blockchain throughput function, the method further includes: constructing preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, specifically including: Based on the task ratio of each terminal device migrated to each edge server, a constraint condition is constructed to obtain a task migration ratio constraint condition; Based on the cache decision parameters of content popularity estimation, the data volume parameters of cached content, the maximum cache capacity parameters of edge servers, the storage resource estimation deviation parameters and the content cache target decision parameters, the constraints are constructed to obtain the cache constraints; Constructing a constraint condition based on a bandwidth ratio parameter allocated by the edge server to each of the terminal devices to obtain a bandwidth allocation ratio constraint condition; Based on the edge computing resource parameters allocated to each terminal device by the edge server estimated in the preset digital twin layer, the computing resource estimation deviation parameters of the digital twin layer, the maximum computing frequency parameters of the edge server, the computing resource parameters allocated to the terminal device itself estimated by the digital twin layer, the computing resource estimation deviation parameters of the digital twin layer, and the maximum computing frequency parameters of the terminal device, constraints are constructed to obtain a first computing resource constraint condition and a second computing resource constraint condition; Constructing constraint conditions based on deadline parameters of tasks executed by each of the terminal devices to obtain task deadline constraint conditions; Constraints are constructed based on blockchain runtime delay parameters and block generation interval parameters to obtain blockchain stability constraints.

4. The method according to claim 1, characterized in that Before constructing a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, the preset total delay function of the digital twin network system, and the preset blockchain throughput function, the method further includes: constructing the preset total delay function of the digital twin network system and constructing the preset blockchain throughput function, specifically including: Construct a function based on the pre-preparation delay parameter, the preparation delay parameter, and the hold delay parameter to obtain a consensus delay function; A function is constructed based on the consensus delay function and the block generation interval delay parameter to obtain a blockchain operation delay function; Based on the preset task calculation delay function, the preset task migration delay function, and the preset result download delay function, a function is constructed to obtain an edge computing delay function; Based on the estimated local computing delay parameter, the local computing delay deviation parameter and the local task computing delay function, a function is constructed to obtain a local computing delay function; A function is constructed based on the consensus delay function, the blockchain operation delay function, the edge computing delay function, and the local computing delay function to obtain the preset total delay function; Based on the transaction quantity parameter, block data volume parameter and transaction data volume parameter, a function is constructed to obtain a new block generation quantity function; A function is constructed based on the new block generation quantity function, the block data volume parameter, the transaction data volume parameter, the block generation interval parameter, and the transmission rate parameter between the preset edge servers to obtain the preset blockchain throughput function.

5. The method according to claim 1, characterized in that The optimization model is reconstructed based on the preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling, which specifically includes: A function is constructed based on the preset security parameters and the stability constraint conditions to obtain a blockchain stability penalty function; A cache penalty function is obtained by constructing a function based on cache decision parameters estimated by content popularity, data volume parameters of cached content, maximum cache capacity parameters of edge servers, storage resource estimation deviation parameters, content cache target decision parameters, and cache constraints. Constructing a function based on the blockchain stability penalty function and the cache penalty function to obtain a target penalty function; A function is constructed based on a preset task trusted processing delay weight coefficient, a blockchain transaction throughput weight coefficient, the preset total delay function, and the preset blockchain throughput function to obtain a reward function; A model is constructed based on the reward function, the agent set, the system state set, the action set and the state transition probability of the digital twin network system to obtain the preset multi-agent Markov decision process model; Based on the preset multi-agent Markov decision process model and the reward function, the optimization model of the digital twin network scheduling is transformed to obtain the target optimization model of the digital twin network scheduling.

6. The method according to claim 1, characterized in that Before optimizing the target optimization model using a preset multi-agent deep reinforcement learning model based on the observation state information of each terminal device in the digital twin network system collected in real time, the method further includes: constructing the preset multi-agent deep reinforcement learning model, specifically including: Constructing a first initial critic network model, a second initial critic network model and an initial actor network model respectively; Performing calculations based on historical experience data, a preset first loss function, a preset second loss function, a preset third loss function, and a preset entropy regularization loss function to obtain various model parameters, each of which includes a first model parameter corresponding to the first initial critic network model, a second model parameter corresponding to the second initial critic network model, a third model parameter corresponding to the initial actor network model, and a regularization coefficient corresponding to the preset entropy regularization loss function; Based on the first model parameter, the second model parameter, the third model parameter and the regularization coefficient, a model is updated to obtain a first current critic network model, a second current critic network model, a current actor model and a current entropy regularization loss function; The first current critic network model, the second current critic network model, the current actor model and the current entropy regularization loss function are iteratively updated in a loop until a reward function value obtained by optimizing the target optimization model using each action parameter value obtained by state prediction of each historical experience data using the updated current actor model meets a preset condition, and the updated network models are determined as the preset multi-agent deep reinforcement learning model.

7. The method according to claim 6, characterized in that The calculation process based on the historical experience data, the preset first loss function, the preset second loss function, the preset third loss function and the preset entropy regularization loss function is performed to obtain various model parameters, specifically including: Based on historical experience data, a first loss function of a first initial critic network model is used for calculation processing to obtain a first model parameter when the loss value of the first loss function is minimum; Based on the historical experience data, a second loss function of the second initial critic network model is used for calculation processing to obtain a second model parameter when the loss value of the second loss function is minimum; Perform screening based on the first model parameter and the second model parameter to obtain target model parameters; Based on the target model parameters and the historical experience data, a third loss function of the initial actor network model is used to perform calculation processing to obtain a third model parameter when the loss value of the third loss function is the smallest; Based on the historical experience data, a preset entropy regularization loss function is used for calculation and processing to obtain a regularization coefficient when the loss value of the preset entropy regularization loss function is the smallest.

8. A network resource scheduling device, characterized in that: include: System building modules for building digital twin network systems; A diffusion blockchain mechanism building module is used to establish a diffusion blockchain mechanism based on the digital twin network system, so that each edge server and terminal device can join the blockchain to ensure the security and reliability of network scheduling; A model building module, used to build a model based on preset constraints corresponding to the task scheduling stage of the digital twin network system, the resource scheduling stage of the digital twin network system, and the blockchain consensus stage, a preset total delay function of the digital twin network system, and a preset blockchain throughput function, to obtain an optimization model for digital twin network scheduling, wherein the optimization model carries various parameters to be optimized; A model reconstruction module, used to reconstruct the optimization model based on a preset multi-agent Markov decision process model to obtain a target optimization model for digital twin network scheduling; The optimization module is used to optimize the target optimization model based on the observation status information of each terminal device in the digital twin network system collected in real time using a preset multi-agent deep reinforcement learning model to obtain target parameter values ​​corresponding to each parameter to be optimized.

9. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the digital twin network trusted scheduling method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It at least includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program on the memory, the steps of the digital twin network trusted scheduling method described in any one of claims 1 to 7 are implemented.

Images