A digital twin-enabled industrial Internet of Things resource allocation system and method

Abstract

The present invention belongs to the field of communication technology and relates to a digital twin-enabled industrial Internet of Things resource allocation system and method; the method comprises: constructing a digital twin-enabled industrial Internet of Things resource allocation system; jointly considering task offloading device association decision, offloading decision, resource allocation, task proportion allocation in the digital twin layer and proposing a target optimization problem of minimizing system cost; modeling the target optimization problem as a multi-agent Markov decision problem; adopting a collaborative computing and resource allocation CCRA algorithm to solve the Markov decision problem, output an offloading strategy, and complete resource allocation optimization; the system constructed by the present invention is conducive to realizing unified scheduling and management of global resources; reducing the computing load of edge servers, and realizing effective processing of large-scale computing-intensive and delay-sensitive tasks; the method considers the deviation between the true value and the digital twin estimated value, and summarizes the law of system change due to the deviation.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and more specifically, relates to a digital twin-enabled industrial Internet of Things resource allocation system and method. Background Art

[0002] Digital twin is an emerging digital mapping technology that provides an excellent digital solution for smart manufacturing. A large number of studies have demonstrated the effectiveness of digital twins in facilitating task offloading decisions. By creating a digital representation of the physical system, digital twins are able to optimize resource allocation, thereby improving network performance. Currently, many teams are committed to studying resource allocation enabled by digital twins. Hou et al. proposed a hierarchical task offloading method enabled by digital twins that can simultaneously meet the requirements of delay-tolerant and delay-sensitive tasks. In addition, they utilized federated learning techniques to optimize the decision model. However, the gap between the digital domain and the real world is not incorporated into their model. Therefore, this oversight may lead to a significant deviation between the system's predicted and actual values, which will have an adverse impact on the overall performance.

[0003] In addition, at present, the application of digital twins in the industrial Internet is limited to the end and edge, and the computing power of the cloud is not fully utilized. Further investigation and research are needed to fill this gap. Considering the dynamic characteristics of the end-edge-cloud system in the industrial Internet of Things, we need an efficient solution that can integrate digital twins and end-edge-cloud systems. At the same time, the industrial Internet of Things terminals will generate huge amounts of data during operation, but their computing and storage capabilities are limited. How to effectively handle large-scale computing-intensive and delay-sensitive tasks has become an urgent problem to be solved. Summary of the invention

[0004] In view of the problems existing in the above-mentioned prior art, the present invention provides an industrial Internet of Things resource allocation system and method enabled by a digital twin.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A digital twin-enabled industrial Internet of Things resource allocation system, the system includes an object entity layer and a digital twin layer; the physical entity layer includes industrial Internet of Things terminal devices, edge servers, and cloud servers, and on this basis, a digital twin layer is created, which includes digital twins of various object entities. These digital twins interact with their physical counterparts in real time to achieve seamless information exchange and decision-making; the physical entity layer is equipped with an acquisition platform and an execution platform; the acquisition platform includes a data acquisition module and a data sending module; the data acquisition module is used to collect the current real-time status information and task offloading requirements of all industrial Internet of Things terminal devices, edge servers, and cloud servers in the physical entity layer The data transmission module is used to transmit the real-time status information and task offloading requirement information of the physical entity layer transmitted by the data acquisition module to the digital twin layer; the execution platform includes a decision receiving module and a decision execution module; the decision receiving module is used to receive the optimal offloading decision instruction from the demand decision module of the digital twin layer, and send the optimal offloading decision instruction to the decision execution module; the decision execution module is used to receive the optimal offloading decision instruction from the decision receiving module, and complete the task offloading of the physical entity layer according to the optimal offloading decision instruction; the digital twin layer Equipped with a synchronization platform and a decision-making platform; the synchronization platform includes a data storage module, a data simulation module, and a demand analysis module; the data storage module is used to store real-time status information and task offloading demand information of physical physical layer devices from the physical entity layer; the data simulation module is used to receive the real-time status information of the physical physical layer devices from the data storage module, and perform data modeling simulation to construct a real-time digital twin corresponding to the physical physical layer devices; the demand analysis module is used to receive the real-time task offloading demand information of the physical physical layer devices stored in the data storage module, and perform corresponding task offloading demand analysis based on the real-time digital twin constructed by the data simulation module; the decision platform includes a task offloading ratio decision module, a device association decision module, an offloading decision module, and a decision instruction issuing module; the task offloading ratio decision module, the device association decision module, and the offloading decision module are used to load the deep reinforcement learning model online after the demand analysis module completes the corresponding matching task offloading demand analysis; make the optimal offloading decision based on the demand analysis of the demand analysis module, including the task offloading ratio decision, the device association decision, and the offloading decision, and send the optimal offloading decision to the decision instruction issuing module; the decision instruction issuing module is used to generate the optimal offloading decision instruction, and issue the optimal offloading decision instruction to the physical entity layer.

[0007] A digital twin-enabled industrial Internet of Things resource allocation method, the method specifically comprises the following steps:

[0008] Step 1, build a digital twin-enabled industrial Internet of Things resource allocation system, the system includes a two-layer architecture of an object physical layer and a digital twin layer; the object physical layer includes industrial Internet of Things terminal devices, edge servers, cloud servers, acquisition platforms and execution platforms; the object physical layer uploads real-time resource information to the digital twin layer through the acquisition platform, and executes digital twin layer decisions through the execution platform. The digital twin layer includes a synchronization platform and a decision platform. The digital twin layer is responsible for calling intelligent algorithms, considering system costs, achieving global optimal decisions for task offloading requirements, and sending them to the object physical layer;

[0009] Step 2: Based on the constructed digital twin-enabled industrial IoT resource allocation system, the task offloading device association decision, offloading decision, resource allocation, and task ratio allocation are jointly considered in the digital twin layer and the target optimization problem of minimizing the system cost is proposed;

[0010] Step 3, model the target optimization problem as a multi-agent Markov decision problem;

[0011] Step 4: Use the collaborative computing and resource allocation CCRA algorithm to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization.

[0012] Further, in step 1, the object entity layer includes M industrial IoT terminal devices, K edge servers and a cloud server c; the M industrial IoT terminal devices and the K edge servers are respectively used as a collection and express;

[0013] The construction of digital twins requires obtaining real-time global status information, including the operating status and operating requirements of end devices, and the available transmission and computing resources on edge servers and cloud servers; the digital twin layer includes a one-to-one digital mapping of M industrial IoT terminal devices, K edge servers, and one cloud server c in the physical entity layer. It is expressed as:

[0014] Furthermore, in step 2, the decision method of jointly considering task offloading device association decision, offloading decision, resource allocation, and task ratio allocation in the digital twin layer is as follows:

[0015] Step S2-1-1, define a set S = {S1, S2, ... S m} represents the set of all tasks, S m It is a real-time computing task generated by the industrial IoT mobile terminal device m, which is represented by S m ={D m ,C m}, D m Represents task S mThe amount of data, C m Indicates execution of this task S m The total number of CPU cycles required;

[0016] Step S2-1-2: Since the computing power of the IIoT mobile terminal device m is limited to perform computing tasks, it is necessary to offload part of the tasks to the selectable edge server k or cloud server c in a certain proportion; the task is divided into two disjoint parts in a certain proportion, and the task processed locally on the IIoT terminal device m is defined as S m,l ={D m,l ,C m,l}, where D m,l Indicates the amount of task data processed locally, C m,l represents the total number of CPU cycles required to execute this task locally; the part of the task that needs to be offloaded to the edge server k or cloud server c is represented by γ m D m Indicates that, where γ m Defined as the task offloading ratio factor, satisfying 0≤γ m ≤1 and D m,l =(1-γ m )D m ;

[0017] Step S2-1-3: If the edge server k has sufficient computing resources to process the task part that the industrial IoT terminal device m needs to offload, the edge server k can choose to receive the computing task from the industrial IoT terminal device m. At this time, the edge-to-edge association decision variable ξ is defined. m,k =1, otherwise ξ m,k =0; Each industrial IoT terminal device m can only be associated with one edge server k at a time for task offloading, that is,

[0018]

[0019] Step S2-1-4: If the computing resources of the current edge server k are insufficient for processing, the edge server k can be used as a relay to continue to offload the task to the cloud server c, and define the end-edge-cloud offloading decision variable ξ m,c =1, otherwise ξ m,c =0; then the two unloading methods can be expressed as:

[0020] End-to-side unloading: m,k =1,ξ m,c =0;

[0021] Device-Edge-Cloud Offloading: m,k =1,ξ m,c =1.

[0022] Furthermore, in step 2, the objective optimization problem of minimizing system cost is proposed, and the problem is stated as follows:

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] Where: γ m is the task offloading ratio factor; m is the bandwidth allocation ratio; the end-edge association decision variable ξ m,k , edge-cloud offloading decision variables ξ m,c ; is the system cost; Delay for unloading; To unload energy consumption; A collection of M industrial IoT terminal devices; is a set of industrial IoT terminal devices associated with edge server k;

[0029] The maximum tolerable time for generating a task for the industrial Internet of Things terminal device m. If this time value is exceeded, it means that the task has failed. Constraint C1 indicates that the total delay to complete the task must be within the tolerable range. Constraint C2 describes the task splitting ratio requirement. Constraint C3 indicates that the industrial Internet of Things terminal device m can only be associated with one edge server for task offloading. Constraint C4 indicates that only one offloading method can be selected. Constraint C5 describes the bandwidth splitting ratio requirement.

[0030] Furthermore, in step 2, the objective optimization problem of minimizing system cost is proposed, and the specific steps are as follows:

[0031] Step S2-2-1, constructing a transmission model: The transmission process is usually divided into uplink transmission and downlink transmission. The uplink transmission delay mainly includes the wireless transmission delay from the industrial IoT terminal device m to the edge server k and the wired transmission delay from the edge server k to the cloud server c. The return data transmitted by the downlink is generally small, and the downlink transmission delay is usually considered to be negligible. The specific steps are as follows;

[0032] 1) End-to-edge offloading: Multiple IIoT terminal devices m share the same channel resources and are offloaded to edge server k at the same time. The set of IIoT terminal devices m associated with edge server k is set to is a set of M industrial IoT terminal devices; when industrial IoT terminal device m produces task D m,k =γ m D m When offloading to edge server k, the bandwidth allocated by edge server k to industrial IoT terminal device m is B m , γ m Defined as the task offloading ratio factor; D m Represents task S m The amount of data;

[0033] B m =λ m B (2)

[0034] 0≤λ m ≤1 (3)

[0035] Where B is the channel bandwidth of edge server k, λ m Allocate a ratio for bandwidth;

[0036] The maximum transmission rate of uplink wireless transmission from industrial IoT terminal device m to edge server k is:

[0037]

[0038] in, is the transmission power of the industrial IoT terminal device m, g m,k is the channel gain, σ k 2 is the power of the white noise, then the task uplink wireless transmission delay of the industrial IoT terminal device m is:

[0039]

[0040] Then, the transmission energy consumption of the industrial IoT terminal device m is:

[0041]

[0042] 2) End-Edge-Cloud Offloading: At this time, the uplink transmission delay includes wireless transmission delay and wired transmission delay. The wired transmission delay can be simply expressed as:

[0043]

[0044] Where ρ is the transmission delay per unit distance, d k,c is the limited transmission distance between edge server k and cloud server c;

[0045] Therefore, the transmission delay required for users to offload tasks to the cloud server is:

[0046]

[0047] Then, the sending energy consumption of edge server k is:

[0048]

[0049] in, is the transmit power of edge server k;

[0050] The total transmission energy consumption is:

[0051]

[0052] Among them, ξ m,c Offloading decision variables for edge-cloud;

[0053] Step S2-2-2, constructing a calculation model: Considering the estimation deviation of the calculation process brought about by the introduction of digital twins, the specific steps are as follows:

[0054] 1) Local computing: The estimated computing delay of local processing of the industrial IoT terminal device m is:

[0055]

[0056] Among them, γ m is the task offloading scale factor; C m Indicates execution of this task S m The total number of CPU cycles required; Calculate frequency estimates for the CPUs available to the IIoT end device m in the digital twin;

[0057] Assume that the real computing resources of the industrial IoT terminal device m in the physical entity and the user device in the digital twin have been obtained in advance. The estimated computing resource difference is defined as Δf m,l , then the delay error is calculated as:

[0058]

[0059] Then the local actual calculation delay is:

[0060]

[0061] Then the energy consumption of local processing computing tasks is:

[0062]

[0063] Among them, q m is the effective switching capacitance of the local computing chip;

[0064] 2), End-to-Edge Offloading: Edge Server Computation Delay; The estimated computation delay at edge server k is:

[0065]

[0066] in, Edge Servers in Digital Twins Assign to user device The CPU calculates the frequency estimate; similarly, the difference is Δf m,k , then the delay error is calculated as:

[0067]

[0068] Then the actual computation delay of edge server k is:

[0069]

[0070] Then the energy consumption of edge server k in processing computing tasks is:

[0071]

[0072] Among them, q k represents the energy consumption of edge server k processing one CPU cycle;

[0073] 3), End-Edge-Cloud Offloading: Cloud Server Computation Delay; The estimated computation delay at cloud server c is:

[0074]

[0075] in, Cloud server for digital twins Distributed to Industrial IoT end devices The CPU calculates the frequency estimate; similarly, the difference is Δf m,c , then the delay error is calculated as:

[0076]

[0077] Then the actual computing delay of cloud server c is:

[0078]

[0079] Cloud servers do not need to consider computing energy consumption;

[0080] The total computational energy consumption is:

[0081]

[0082] Step S2-2-3, offloading delay calculation. Due to the parallelism of calculation and transmission, the offloading delay depends on the maximum value of the sum of the calculation and transmission delays of each part of the task:

[0083]

[0084]

[0085] in, is the local offloading delay of the industrial IoT terminal device m, is the offloading delay of edge server k, is the offloading delay of cloud server c, Delay for unloading;

[0086] The offloading energy consumption is the sum of the total transmission energy consumption and the total computation energy consumption:

[0087]

[0088] The system cost is:

[0089]

[0090] Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively, satisfying:

[0091]

[0092] Step S2-2-4, propose an optimization problem and complete the problem optimization under the premise of minimizing the task system cost.

[0093] Furthermore, in step 3, the target optimization problem is modeled as a multi-agent Markov decision problem. The specific steps are as follows:

[0094] Step S3-1, the digital twin DT-UE of each industrial IoT terminal device is used as an intelligent agent; the Markov decision process of M intelligent agents can be used as a tuple express, Describes the state of the environment. Thanks to the digital twin layer, information exchange can be achieved between various subjects, and each agent's observation of the environment constitutes the system state space; is the set of actions performed by all agents; is the set of reward functions for each agent, defined as the benefit of performing an action in the current state; It is expressed as the probability of transitioning from the current state to the next completely new state;

[0095] Step S3-2, state space In a digital twin, the environment is fully observed, so the state of the environment consists of the observations of each agent.

[0096] s(t)={o1(t),...,o m (t),...,o M (t)}

[0097] Among them m (t) represents the observation of agent m at time t;

[0098] Step S3-3, action space The decisions made by the agents include the associated target, the offloading target, the bandwidth occupancy ratio, and the task offloading ratio. The actions of the m agents are described as:

[0099] a m (t) = {γ m (t),λ m (t),ξ m,k (t),ξ m,c (t)}

[0100] where γ m (t) represents the task division ratio of agent m at time t, λ m (t) represents the bandwidth allocation ratio of agent m at time t, ξ m,k (t) represents the associated decision variable of agent m at time t, ξ m,c (t) represents the unloading decision variable of agent m at time t;

[0101] Step S3-4, Reward Space According to the current system state, the agent takes action and obtains reward; m agents should collaboratively minimize the total system cost while completing the computational task; therefore, the cost of each DT-UE is The negative value of is used as part of the reward; if a task is not completed within the specified time, there will be a penalty η in the reward function; therefore, the reward of the mth agent is expressed as:

[0102]

[0103] Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively; To uninstall the delay, To unload energy consumption.

[0104] Furthermore, in step 4, the collaborative computing and resource allocation CCRA algorithm is used to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization. The specific steps are as follows:

[0105] Step S4-1-1: Deploy the DDPG algorithm on the digital twin DT-UE of each industrial IoT terminal device. The algorithm includes a weight Actor Module and one with weight Critic module Where o is the observation of all agents, a is the action of all agents, and the Actor module converts the local observation value o m As input, then output action a m (t) = μ m (o m (t)); The Critic module obtains the observations and actions of all agents during the training phase to give a Q value Q m (s(t), a(t)), evaluates the output of the Actor module;

[0106] Step S4-1-2, using the method of converting discrete action space into probability distribution, for the continuous value γ m (t),λ m (t), discrete value ξ m,k (t),ξ m,c (t) constitutes the hybrid action space, and the Actor module of DT-UE is defined as where μ m is the strategy of the agent DT-UE, and the output action a of DT-UE m (t) = {γ m (t),λ m (t),ξ m,k (t),ξ m,c (t)}, where ξ m,k (t),ξ m,c (t) are the probability distributions of two discrete actions respectively;

[0107] Step S4-1-3, using the experience replay mechanism in the MADDPG algorithm, at each moment, all agents can replay the current experience tuple Store to playback buffer Where a(t) is the action in the current time slot, is the reward of the current time slot, s(t) is the state of the current time slot, and s(t+1) is the state of the next time slot; for each DT-UE, in the playback buffer A batch of size (a j ,r j ,s j ,s' j ) of small batch samples, where a j is the randomly sampled action at the current moment, r jis the reward randomly sampled at the current moment, s j is the state randomly sampled at the current moment, s' j The state of random sampling at the next moment. Random sampling can break the correlation between sample data and reduce training oscillation;

[0108] Step S4-1-4, s in the sampled experience sample j Input to the Actor module, and use the policy gradient strategy to update the weight of the Actor module.

[0109]

[0110] in, is the expected return gradient, N b is the size of the mini-batch samples, Actor module weights, which can be updated by minimizing the loss;

[0111]

[0112] In addition, the Critic module updates the parameters by minimizing the loss of the target Q value:

[0113]

[0114] The updated target y m (j) is calculated as,

[0115]

[0116] Where ε is the discount factor.

[0117] Furthermore, the specific process of DT-assisted CCRA algorithm training is as follows:

[0118] Step S4-2-1, first initialize the parameters of the four neural networks and the playback buffer; in each episode, each DT-UE will obtain the observation value of the environment o m (t), and then input to the Actor module, the output includes bandwidth allocation ratio, task division ratio, and probability distribution of associated targets and offload targets; the current sampling probability and action are described as:

[0119]

[0120] Step S4-2-2, after the DT-UE action is completed, the environment will switch to the next state s(t+1), and all DT-UEs will receive corresponding rewards and new observations o(t); agents that fail to complete the task will receive penalties; if the replay buffer is not full, the experience of this stage will be stored directly, otherwise the earliest batch of experience will be replaced;

[0121] Step S4-2-3, then, DT-UE will use small batch samples to sample in the playback buffer to train the parameters of the neural network; the parameters of the Actor module and the Critic module are updated by minimizing the loss function; and the parameters of the two target networks are soft updated, which are,

[0122]

[0123]

[0124] Where τ is the update rate;

[0125] Step S4-2-4, after the training is completed, the Critic module is no longer needed, and the decision of DT-UE will act on the real environment. The mth agent makes decisions in the digital world based on local observations and transmits the action instruction a to the user device in the physical world in real time; then the changes in the physical world will be fed back to DT-UE in real time and enter the next state.

[0126] The present invention has the following technical effects: 1. The present invention introduces digital twin technology to perceive the status information and task offloading demand information of the physical entity layer in real time, which is conducive to the unified scheduling and management of global resources; constructing a two-layer end-edge-cloud collaborative computing architecture containing a physical entity layer and a digital twin layer can effectively realize the empowerment of digital twin technology, and apply deep reinforcement learning algorithms to perform flexible and intelligent scheduling; the decision-making platform of the digital twin layer considers a variety of offloading modes, and can achieve optimal resource allocation through the deep reinforcement learning algorithm deployed in the digital twin layer; fully combine digital twin technology with the collaborative offloading mechanism of "industrial Internet of Things terminal equipment-edge server-cloud server" to reduce the computing load of the edge server and achieve effective processing of large-scale computing-intensive and delay-sensitive tasks.

[0127] 2. Aiming at complex industrial IoT application scenarios, the present invention proposes a digital twin-enabled industrial IoT resource allocation method. The method involves creating digital twins for various entities such as devices, edge servers, and cloud servers; these digital twins interact with their physical counterparts in real time, taking into account the computing power deviations in the digital environment to achieve seamless information exchange and decision-making; taking into account the deviations between the true value and the digital twin estimate, and summarizing the laws of system changes due to the deviations.

[0128] 3. The present invention is based on the CCRA algorithm to solve complex optimization problems in collaborative computing and resource allocation. The agent is deployed on the digital twin to obtain the state information of the cloud edge for training. Compared with other solutions, the present invention transforms the discrete action of device association into continuous probability, solving the problem of mixed action space. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 It is an architecture diagram of the industrial Internet of Things resource allocation system enabled by digital twins in the present invention;

[0130] Figure 2 It is a physical entity layer architecture diagram in the present invention;

[0131] Figure 3 It is the digital twin layer architecture diagram in the present invention;

[0132] Figure 4 It is the digital twin-enabled resource allocation algorithm framework in the present invention;

[0133] Figure 5 It is a flow chart of the industrial Internet of Things resource allocation method enabled by digital twins in the present invention;

[0134] Figure 6 is a schematic diagram of the correlation between the system cost and the number of user devices in the present invention;

[0135] Figure 7 It is a curve diagram of the system cost under different bandwidths and deviations in the present invention. DETAILED DESCRIPTION

[0136] The contents and drawings of the present invention will be described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, involving detailed implementation plans and operation processes, but the protection scope of the present invention is not limited to the following specific embodiments. The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0137] A digital twin-enabled industrial Internet of Things resource allocation method, the method specifically comprises the following steps:

[0138] Step 1: Build a digital twin-enabled industrial IoT resource allocation system, such as Figure 1-3As shown, the system includes a two-layer architecture of an object physical layer and a digital twin layer; the object physical layer includes industrial Internet of Things terminal devices, edge servers, cloud servers, acquisition platforms and execution platforms; the object physical layer uploads real-time resource information to the digital twin layer through the acquisition platform, and executes digital twin layer decisions through the execution platform. The digital twin layer includes a synchronization platform and a decision platform. The digital twin layer is responsible for calling intelligent algorithms, considering system costs, achieving global optimal decisions for task offloading requirements, and sending them to the object physical layer;

[0139] Step 2: Based on the constructed digital twin-enabled industrial IoT resource allocation system, the task offloading device association decision, offloading decision, resource allocation, and task ratio allocation are jointly considered in the digital twin layer and the target optimization problem of minimizing the system cost is proposed;

[0140] Step 3, model the target optimization problem as a multi-agent Markov decision problem;

[0141] Step 4: Use the collaborative computing and resource allocation CCRA algorithm to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization.

[0142] In step 1, the object entity layer includes M industrial IoT terminal devices, K edge servers and a cloud server c; M industrial IoT terminal devices and K edge servers are respectively represented by the set and express;

[0143] The construction of digital twins requires obtaining real-time global status information, including the operating status and operating requirements of end devices, and the available transmission and computing resources on edge servers and cloud servers; the digital twin layer includes a one-to-one digital mapping of M industrial IoT terminal devices, K edge servers, and one cloud server c in the physical entity layer. It is expressed as:

[0144] In step 2, the decision method of task offloading device association decision, offloading decision, resource allocation, and task ratio allocation is jointly considered in the digital twin layer. The specific steps are as follows:

[0145] Step S2-1-1, define a set S = {S1, S2, ... S m} represents the set of all tasks, S m It is a real-time computing task generated by the industrial IoT mobile terminal device m, which is represented by S m ={D m ,C m}, D m Represents task S m The amount of data, C m Indicates execution of this task Sm The total number of CPU cycles required;

[0146] Step S2-1-2: Since the computing power of the IIoT mobile terminal device m is limited to perform computing tasks, it is necessary to offload part of the tasks to the selectable edge server k or cloud server c in a certain proportion; the task is divided into two disjoint parts in a certain proportion, and the task processed locally on the IIoT terminal device m is defined as S m,l ={D m,l ,C m,l}, where D m,l Indicates the amount of task data processed locally, C m,l represents the total number of CPU cycles required to execute this task locally; the part of the task that needs to be offloaded to the edge server k or cloud server c is represented by γ m D m Indicates that, where γ m Defined as the task offloading ratio factor, satisfying 0≤γ m ≤1 and D m,l =(1-γ m )D m ;

[0147] Step S2-1-3: If the edge server k has sufficient computing resources to process the task part that the industrial IoT terminal device m needs to offload, the edge server k can choose to receive the computing task from the industrial IoT terminal device m. At this time, the edge-to-edge association decision variable ξ is defined. m,k =1, otherwise ξ m,k =0; Each industrial IoT terminal device m can only be associated with one edge server k at a time for task offloading, that is,

[0148]

[0149] Step S2-1-4: If the computing resources of the current edge server k are insufficient for processing, the edge server k can be used as a relay to continue to offload the task to the cloud server c, and define the end-edge-cloud offloading decision variable ξ m,c =1, otherwise ξ m,c =0; then the two unloading methods can be expressed as:

[0150] End-to-side unloading: m,k =1,ξ m,c =0;

[0151] Device-Edge-Cloud Offloading: m,k =1,ξ m,c =1.

[0152] In step 2, the objective optimization problem of minimizing system cost is proposed. The specific steps are as follows:

[0153] Step S2-2-1, constructing a transmission model: The transmission process is usually divided into uplink transmission and downlink transmission. The uplink transmission delay mainly includes the wireless transmission delay from the industrial IoT terminal device m to the edge server k and the wired transmission delay from the edge server k to the cloud server c. The return data transmitted by the downlink is generally small, and the downlink transmission delay is usually considered to be negligible. The specific steps are as follows;

[0154] 1) End-to-edge offloading: Multiple IIoT terminal devices m share the same channel resources and are offloaded to edge server k at the same time. The set of IIoT terminal devices m associated with edge server k is set to is a set of M industrial IoT terminal devices; when industrial IoT terminal device m produces task D m,k =γ m D m When offloading to edge server k, the bandwidth allocated by edge server k to industrial IoT terminal device m is B m , γ m Defined as the task offloading ratio factor; D m Represents task S m The amount of data;

[0155] B m =λ m B (2)

[0156] 0≤λ m ≤1 (3)

[0157] Where B is the channel bandwidth of edge server k, λ m Allocate a ratio for bandwidth;

[0158] The maximum transmission rate of uplink wireless transmission from industrial IoT terminal device m to edge server k is:

[0159]

[0160] in, is the transmission power of the industrial IoT terminal device m, g m,k is the channel gain, σ k 2 is the power of the white noise, then the task uplink wireless transmission delay of the industrial IoT terminal device m is:

[0161]

[0162] Then, the transmission energy consumption of the industrial IoT terminal device m is:

[0163]

[0164] 2) End-Edge-Cloud Offloading: At this time, the uplink transmission delay includes wireless transmission delay and wired transmission delay. The wired transmission delay can be simply expressed as:

[0165]

[0166] Where ρ is the transmission delay per unit distance, d k,c is the limited transmission distance between edge server k and cloud server c;

[0167] Therefore, the transmission delay required for users to offload tasks to the cloud server is:

[0168]

[0169] Then, the sending energy consumption of edge server k is:

[0170]

[0171] in, is the transmit power of edge server k;

[0172] The total transmission energy consumption is:

[0173]

[0174] Among them, ξ m,c Offloading decision variables for edge-cloud;

[0175] Step S2-2-2, constructing a calculation model: Considering the estimation deviation of the calculation process brought about by the introduction of digital twins, the specific steps are as follows:

[0176] 1) Local computing: The estimated computing delay of local processing of the industrial IoT terminal device m is:

[0177]

[0178] Among them, γ m is the task offloading scale factor; C m Indicates execution of this task S m The total number of CPU cycles required; Calculate frequency estimates for the CPUs available to the IIoT end device m in the digital twin;

[0179] Assume that the real computing resources of the industrial IoT terminal device m in the physical entity and the user device in the digital twin have been obtained in advance. The estimated computing resource difference is defined as Δf m,l , then the delay error is calculated as:

[0180]

[0181] Then the local actual calculation delay is:

[0182]

[0183] Then the energy consumption of local processing computing tasks is:

[0184]

[0185] Among them, q m is the effective switching capacitance of the local computing chip;

[0186] 2), End-to-Edge Offloading: Edge Server Computation Delay; The estimated computation delay at edge server k is:

[0187]

[0188] in, Edge Servers in Digital Twins The CPU assigned to user device m calculates the frequency estimate; similarly, the difference is Δf m,k , then the delay error is calculated as:

[0189]

[0190] Then the actual computation delay of edge server k is:

[0191]

[0192] Then the energy consumption of edge server k in processing computing tasks is:

[0193]

[0194] Among them, q k represents the energy consumption of edge server k processing one CPU cycle;

[0195] 3), End-Edge-Cloud Offloading: Cloud Server Computation Delay; The estimated computation delay at cloud server c is:

[0196]

[0197] in, Cloud server for digital twins Distributed to Industrial IoT end devices The CPU calculates the frequency estimate; similarly, the difference is Δf m,c , then the delay error is calculated as:

[0198]

[0199] Then the actual computing delay of cloud server c is:

[0200]

[0201] Cloud servers do not need to consider computing energy consumption;

[0202] The total computational energy consumption is:

[0203]

[0204] Step S2-2-3, offloading delay calculation. Due to the parallelism of calculation and transmission, the offloading delay depends on the maximum value of the sum of the calculation and transmission delays of each part of the task:

[0205]

[0206] in, is the local offloading delay of the industrial IoT terminal device m, is the offloading delay of edge server k, is the offloading delay of cloud server c, Delay for unloading;

[0207] The offloading energy consumption is the sum of the total transmission energy consumption and the total computation energy consumption:

[0208]

[0209] The system cost is:

[0210]

[0211] Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively, satisfying:

[0212]

[0213] Step S2-2-4, propose an optimization problem, and complete the problem optimization under the premise of minimizing the task system cost. The problem is stated as follows:

[0214]

[0215] in: The maximum tolerable time for generating a task for the industrial Internet of Things terminal device m. If this time value is exceeded, it means that the task has failed. Constraint C1 indicates that the total delay to complete the task must be within the tolerable range. Constraint C2 describes the task splitting ratio requirement. Constraint C3 indicates that the industrial Internet of Things terminal device m can only be associated with one edge server for task offloading. Constraint C4 indicates that only one offloading method can be selected. Constraint C5 describes the bandwidth splitting ratio requirement.

[0216] In step 3, if Figure 4 As shown in Figure 2, the target optimization problem is modeled as a multi-agent Markov decision problem. The specific steps are as follows:

[0217] Step S3-1, the digital twin DT-UE of each industrial IoT terminal device is used as an intelligent agent; the Markov decision process of M intelligent agents can be used as a tuple express, Describes the state of the environment. Thanks to the digital twin layer, information exchange can be achieved between various subjects, and each agent's observation of the environment constitutes the system state space; is the set of actions performed by all agents; is the set of reward functions for each agent, defined as the benefit of performing an action in the current state; It is expressed as the probability of transitioning from the current state to the next completely new state;

[0218] Step S3-2, state space In a digital twin, the environment is fully observed, so the state of the environment consists of the observations of each agent.

[0219] s(t)={o1(t),...,o m (t),...,o M (t)} (29)

[0220] Among them m (t) represents the observation of agent m at time t;

[0221] Step S3-3, action space The decisions made by the agents include the associated target, the offloading target, the bandwidth occupancy ratio, and the task offloading ratio. The actions of the m agents are described as:

[0222] a m (t) = {γ m (t),λ m (t),ξ m,k (t),ξ m,c (t)} (30)

[0223] where γ m (t) represents the task division ratio of agent m at time t, λ m (t) represents the bandwidth allocation ratio of agent m at time t, ξ m,k (t) represents the associated decision variable of agent m at time t, ξ m,c (t) represents the unloading decision variable of agent m at time t;

[0224] Step S3-4, Reward Space According to the current system state, the agent takes action and obtains reward; m agents should collaboratively minimize the total system cost while completing the computational task; therefore, the cost of each DT-UE is The negative value of is used as part of the reward; if a task is not completed within the specified time, there will be a penalty η in the reward function; therefore, the reward of the mth agent is expressed as:

[0225]

[0226] Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively; To uninstall the delay, To unload energy consumption;.

[0227] In step 4, the collaborative computing and resource allocation CCRA algorithm is used to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization. The specific steps are as follows:

[0228] Step S4-1-1: Deploy the DDPG algorithm on the digital twin DT-UE of each industrial IoT terminal device. The algorithm includes a weight Actor module and one with weight Critic module Where o is the observation of all agents, a is the action of all agents, and the Actor module converts the local observation value o m As input, then output action a m (t) = μ m (o m (t)); The Critic module obtains the observations and actions of all agents during the training phase to give a Q value Q m (s(t), a(t)), evaluates the output of the Actor module;

[0229] Step S4-1-2, using the method of converting discrete action space into probability distribution, for the continuous value γ m (t),λ m (t), discrete value ξ m,k (t),ξ m,c (t) constitutes the hybrid action space, and the Actor module of DT-UE is defined as where μ m is the strategy of the agent DT-UE, and the output action a of DT-UE m (t) = {γ m (t),λ m (t),ξ m,k (t),ξ m,c(t)}, where ξ m,k (t),ξ m,c (t) are the probability distributions of two discrete actions respectively;

[0230] Step S4-1-3, using the experience replay mechanism in the MADDPG algorithm, at each moment, all agents can replay the current experience tuple Store to playback buffer Where a(t) is the action in the current time slot, is the reward of the current time slot, s(t) is the state of the current time slot, and s(t+1) is the state of the next time slot; for each DT-UE, in the playback buffer A batch of size (a j ,r j ,s j ,s' j ) of small batch samples, where a j is the randomly sampled action at the current moment, r j is the reward randomly sampled at the current moment, s j is the state randomly sampled at the current moment, s' j The state of random sampling at the next moment. Random sampling can break the correlation between sample data and reduce training oscillation;

[0231] Step S4-1-4, s in the sampled experience sample j Input to the Actor module, and use the policy gradient strategy to update the weight of the Actor module.

[0232]

[0233] in, is the expected return gradient, N b is the size of the mini-batch samples, Actor module weights, which can be updated by minimizing the loss;

[0234]

[0235] In addition, the Critic module updates the parameters by minimizing the loss of the target Q value:

[0236]

[0237] The updated target y m (j) is calculated as,

[0238]

[0239] Where ε is the discount factor;

[0240] The specific process of using DT-assisted CCRA algorithm training is as follows:

[0241] Step S4-2-1, first initialize the parameters of the four neural networks and the playback buffer; in each episode, each DT-UE will obtain the observation value of the environment o m (t), and then input to the Actor module, the output includes bandwidth allocation ratio, task division ratio, and probability distribution of associated targets and offload targets; the current sampling probability and action are described as:

[0242]

[0243] Step S4-2-2, after the DT-UE action is completed, the environment will switch to the next state s(t+1), and all DT-UEs will receive corresponding rewards and new observations o(t); agents that fail to complete the task will receive penalties; if the replay buffer is not full, the experience of this stage will be stored directly, otherwise the earliest batch of experience will be replaced;

[0244] Step S4-2-3, then, DT-UE will use small batch samples to sample in the playback buffer to train the parameters of the neural network; the parameters of the Actor module and the Critic module are updated by minimizing the loss function; and the parameters of the two target networks are soft updated, which are,

[0245]

[0246]

[0247] Where τ is the update rate;

[0248] Step S4-2-4, after the training is completed, the Critic module is no longer needed, and the decision of DT-UE will act on the real environment. The mth agent makes decisions in the digital world based on local observations and transmits the action instruction a to the user device in the physical world in real time; then the changes in the physical world will be fed back to DT-UE in real time and enter the next state.

[0249] The present invention considers the industrial Internet of Things scenario based on edge cloud. All industrial Internet of Things terminal devices and edge servers are evenly distributed throughout the area, and it is assumed that a cloud server and multiple edge servers provide computing services for multiple user devices. In the system, the bandwidth of each edge server is set to B = 20MHz, and the transmission rate between the edge server and the cloud server is set to ν = 100Mbps. For the industrial Internet of Things terminal device m, the transmission power Channel gain g m,k=-20dB, Gaussian white noise power The locally calculated effective switch capacitance is set to q m =10 -27 , the CPU computing frequency of the industrial IoT terminal device m is estimated to be The CPU computing frequency of edge server k is estimated to be The energy consumption of the CPU processing one cycle of edge server k is set to e k =10 -9 For cloud servers, the estimated CPU computing frequency is set to In this invention, both latency and energy consumption are considered to be important factors in the Internet of Things, so the weight ω1=ω2=0.5. The simulation environment and algorithm are executed on Ubuntu 20.04.5, using Python 3.8.13 and PyTorch 1.12.1. In the algorithm framework proposed in this invention, the Actor and Critic modules include two hidden layers, both with a size of 400.

[0250] To verify the benefits of the CCRA-based algorithm enabled by the edge-cloud collaboration and digital twin, the proposed solution is compared with some baseline solutions as shown below:

[0251] 1) Doubledeep Q-Learning (DDQN): A DDQN-based offloading method is implemented for the collaborative edge-cloud system. Bandwidth allocation and task split ratio are discretized into 32 levels. The settings of algorithm parameters, such as discount coefficient and replay buffer size, are the same as those of the present invention.

[0252] 2) No Digital Twin (No DT): In this scheme, digital twins are not used. Therefore, the user device will randomly assign some tasks to the edge server or cloud server without considering the system state. In addition, due to the lack of necessary information exchange, the user devices will use greedy strategies to compete with each other for the limited system bandwidth.

[0253] 3) Local computing: All tasks are executed locally on the user's device, and no algorithm is required to output decisions on task segmentation and resource allocation.

[0254] 4) Edge End only: In this scheme, we exclude the participation of cloud servers. Instead, all tasks within the system are collaboratively computed by user devices and edge servers using a MADDPG-based algorithm. It is worth noting that the discrete action space mainly focuses on device association, while the continuous action space includes bandwidth allocation and task segmentation.

[0255] 5) Cloud Computing: The edge servers in this solution only act as relay nodes and lack computing power. All tasks are performed by user devices and a cloud server.

[0256] like Figure 6 As shown in the figure, the correlation between system cost and the number of industrial IoT terminal devices is described, while the number of edge servers is fixed to 4. Due to the increase in computing tasks, more computing and communication resources need to be consumed. The solution proposed by the present invention is the most cost-effective solution, with an average cost of 33.7% lower than that of other benchmark solutions when the number of devices is 28. In contrast, due to limited computing resources, local computing has the highest system cost. In contrast, "EE Only" reduces communication and computing delays compared to local computing and cloud computing. However, when the number of user devices exceeds the threshold of 20, the system cost of "EE Only" exceeds that of cloud computing. This may be due to the fact that some edge servers have undertaken too many computing tasks, resulting in longer processing delays, which in turn pushes up the overall system cost.

[0257] like Figure 7 As shown in Figure 2, the changes in system cost under different bandwidths and deviations are compared, where the error exists in all elements in the system, including cloud servers and edge servers. When the digital twin error is constant, as the edge server bandwidth increases, the corresponding system cost will decrease. This is because the industrial IoT terminal devices can obtain a larger bandwidth and choose edge servers with better service quality to offload tasks. On the other hand, when the bandwidth of the edge server is constant, the corresponding system cost is positively correlated with the digital twin deviation.

[0258] The above description of the disclosed embodiments enables professionals in the field to use the present invention. At the same time, the above embodiments are only for illustrating the technical ideas of the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical ideas proposed by the present invention fall within the protection scope of the present invention.

Claims

1. A digital twin-enabled industrial Internet of Things resource allocation system, characterized in that: The system includes an object physical layer and a digital twin layer; The physical entity layer includes industrial IoT terminal devices, edge servers, and cloud servers. On this basis, a digital twin layer is created, which includes digital twins of various physical entities. These digital twins interact with their physical counterparts in real time to achieve seamless information exchange and decision-making. The physical entity layer is equipped with a collection platform and an execution platform; the collection platform includes a data collection module and a data sending module; The data acquisition module is used to collect the current real-time status information and task offloading demand information of all industrial Internet of Things terminal devices, edge servers, and cloud servers at the physical entity layer, and transmit the collected real-time status information and task offloading demand information to the data sending module; The data sending module is used to send the real-time status information and task offloading requirement information of the physical entity layer equipment transmitted by the data acquisition module to the digital twin layer; The execution platform includes a decision receiving module and a decision execution module; the decision receiving module is used to receive the optimal unloading decision instruction from the digital twin layer demand decision module, and send the optimal unloading decision instruction to the decision execution module; The decision execution module is used to receive the optimal unloading decision instruction from the decision receiving module, and complete the task unloading of the physical entity layer according to the optimal unloading decision instruction; The digital twin layer is equipped with a synchronization platform and a decision-making platform; the synchronization platform includes a data storage module, a data simulation module, and a demand analysis module; The data storage module is used to store real-time status information and task offloading requirement information of physical layer devices from the physical layer; The data simulation module is used to receive real-time status information of the physical layer device from the data storage module, and perform data modeling simulation to build a real-time digital twin corresponding to the physical layer device; The demand analysis module is used to receive the real-time task offloading demand information of the physical entity layer device stored in the data storage module, and perform corresponding matching task offloading demand analysis based on the real-time digital twin constructed by the data simulation module; The decision platform includes a task offloading ratio decision module, a device association decision module, an offloading decision module, and a decision instruction issuing module; The task offloading ratio decision module, the device association decision module, and the offloading decision module are used to load the deep reinforcement learning model online after the demand analysis module completes the corresponding matching task offloading demand analysis; Make the optimal offloading decision based on the demand analysis of the demand analysis module, including task offloading ratio decision, device association decision, and offloading decision, and send the optimal offloading decision to the decision instruction issuing module; The decision instruction issuing module is used to generate an optimal offloading decision instruction and issue the optimal offloading decision instruction to the physical entity layer; The specific steps of the system to implement the digital twin-enabled industrial Internet of Things resource allocation method are as follows: Step 1, build a digital twin-enabled industrial Internet of Things resource allocation system, the system includes a two-layer architecture of an object physical layer and a digital twin layer; the object physical layer includes industrial Internet of Things terminal devices, edge servers, cloud servers, acquisition platforms and execution platforms; the object physical layer uploads real-time resource information to the digital twin layer through the acquisition platform, and executes digital twin layer decisions through the execution platform. The digital twin layer includes a synchronization platform and a decision platform. The digital twin layer is responsible for calling intelligent algorithms, considering system costs, achieving global optimal decisions for task offloading requirements, and sending them to the object physical layer; Step 2: Based on the constructed digital twin-enabled industrial IoT resource allocation system, the task offloading device association decision, offloading decision, resource allocation, and task ratio allocation are jointly considered in the digital twin layer and the target optimization problem of minimizing the system cost is proposed; Step 3, model the target optimization problem as a multi-agent Markov decision problem; Step 4: Use the collaborative computing and resource allocation CCRA algorithm to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization.

2. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 1, characterized in that: In step 1, the object entity layer includes M industrial IoT terminal devices, K edge servers and a cloud server c; M industrial IoT terminal devices and K edge servers are respectively represented by the set and express; The construction of digital twins requires obtaining real-time global status information, including the operating status and operating requirements of end devices, and the available transmission and computing resources on edge servers and cloud servers; the digital twin layer includes a one-to-one digital mapping of M industrial IoT terminal devices, K edge servers, and one cloud server c in the physical entity layer. It is expressed as:

3. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 1, characterized in that: In step 2, the decision method of task offloading device association decision, offloading decision, resource allocation, and task ratio allocation is jointly considered in the digital twin layer. The specific steps are as follows: Step S2-1-1, define a set S = {S1, S2, ... Sm} to represent the set of all tasks, S m It is a real-time computing task generated by the industrial IoT mobile terminal device m, which is represented by S m ={D m ,C m }, D m Represents task S m The amount of data, C m Indicates execution of this task S m The total number of CPU cycles required; Step S2-1-2: Since the computing power of the industrial IoT mobile terminal device m to perform computing tasks is limited, it is necessary to offload part of the tasks to the selectable edge server k or cloud server c in a certain proportion; The task is divided into two disjoint parts according to a certain ratio, and the task processed locally on the IIoT terminal device m is defined as S m,l ={D m,l ,C m,l }, where D m,l Indicates the amount of task data processed locally, C m,l represents the total number of CPU cycles required to execute this task locally; the part of the task that needs to be offloaded to the edge server k or cloud server c is represented by γ m D m Indicates that, where γ m Defined as the task offloading ratio factor, satisfying 0≤γ m ≤1 and D m,l =(1-γ m )D m ; Step S2-1-3: If the edge server k has sufficient computing resources to process the task part that the industrial IoT terminal device m needs to offload, the edge server k can choose to receive the computing task from the industrial IoT terminal device m. At this time, the edge-to-edge association decision variable ξ is defined. m,k =1, otherwise ξ m,k =0; Each industrial IoT terminal device m can only be associated with one edge server k at a time for task offloading, that is, Step S2-1-4: If the computing resources of the current edge server k are insufficient for processing, the edge server k can be used as a relay to continue to offload the task to the cloud server c, and define the end-edge-cloud offloading decision variable ξ m,c =1, otherwise ξ m,c =0; then the two unloading methods can be expressed as: End-to-side unloading: m,k =1,ξ m,c =0; Device-Edge-Cloud Offloading: m,k =1,ξ m,c =1.

4. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 1, characterized in that: In step 2, the objective optimization problem of minimizing system cost is proposed, and the problem is stated as follows: Where: γ m is the task offloading scale factor; m is the bandwidth allocation ratio; the end-edge association decision variable ξ m,k , edge-cloud offloading decision variables ξ m,c ; is the system cost; Delay for unloading; To unload energy consumption; A collection of M industrial IoT terminal devices; is a set of industrial IoT terminal devices associated with edge server k; The maximum tolerable time for generating a task for the industrial Internet of Things terminal device m. If this time value is exceeded, it means that the task has failed. Constraint C1 indicates that the total delay to complete the task must be within the tolerable range. Constraint C2 describes the task splitting ratio requirement. Constraint C3 indicates that the industrial Internet of Things terminal device m can only be associated with one edge server for task offloading. Constraint C4 indicates that only one offloading method can be selected. Constraint C5 describes the bandwidth splitting ratio requirement.

5. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 4, characterized in that: In step 2, the objective optimization problem of minimizing system cost is proposed. The specific steps are as follows: Step S2-2-1, constructing a transmission model: The transmission process is usually divided into uplink transmission and downlink transmission. The uplink transmission delay mainly includes the wireless transmission delay from the industrial IoT terminal device m to the edge server k and the wired transmission delay from the edge server k to the cloud server c. The return data transmitted by the downlink is generally small, and the downlink transmission delay is usually considered to be negligible. The specific steps are as follows; 1) End-to-edge offloading: Multiple IIoT terminal devices m share the same channel resources and are offloaded to edge server k at the same time. The set of IIoT terminal devices m associated with edge server k is set to is a set of M industrial IoT terminal devices; when industrial IoT terminal device m produces task D m,k =γ m D m When offloading to edge server k, the bandwidth allocated by edge server k to industrial IoT terminal device m is B m , γ m Defined as the task offloading ratio factor; D m Represents task S m The amount of data; B m =λ m B (2) 0≤λ m ≤1 (3) Where B is the channel bandwidth of edge server k, λ m Allocate a ratio for bandwidth; The maximum transmission rate of uplink wireless transmission from industrial IoT terminal device m to edge server k is: in, is the transmission power of the industrial IoT terminal device m, g m,k is the channel gain, σ k 2 is the power of the white noise, then the task uplink wireless transmission delay of the industrial IoT terminal device m is: Then, the transmission energy consumption of the industrial IoT terminal device m is: 2) End-Edge-Cloud Offloading: At this time, the uplink transmission delay includes wireless transmission delay and wired transmission delay. The wired transmission delay can be simply expressed as: Where ρ is the transmission delay per unit distance, d k,c is the limited transmission distance between edge server k and cloud server c; Therefore, the transmission delay required for users to offload tasks to the cloud server is: Then, the sending energy consumption of edge server k is: in, is the transmit power of edge server k; The total transmission energy consumption is: Among them, ξ m,c Offloading decision variables for edge-cloud; Step S2-2-2, constructing a calculation model: Considering the estimation deviation of the calculation process brought about by the introduction of digital twins, the specific steps are as follows: 1) Local computing: The estimated computing delay of local processing of the industrial IoT terminal device m is: Among them, γ m is the task offloading scale factor; C m Indicates execution of this task S m The total number of CPU cycles required; Calculate frequency estimates for the CPUs available to the IIoT end device m in the digital twin; Assume that the difference between the real computing resources of the industrial IoT terminal device m in the physical entity and the estimated computing resources of the user device m in the digital twin has been obtained in advance, and the difference is defined as Δf m,l , then the delay error is calculated as: Then the local actual calculation delay is: Then the energy consumption of local processing computing tasks is: Among them, q m is the effective switching capacitance of the local computing chip; 2), End-to-Edge Offloading: Edge Server Computation Delay; The estimated computation delay at edge server k is: in, Edge Servers in Digital Twins Assign to user device The CPU calculates the frequency estimate; similarly, the difference is Δf m,k , then the delay error is calculated as: Then the actual computation delay of edge server k is: Then the energy consumption of edge server k in processing computing tasks is: Among them, q k represents the energy consumption of edge server k processing one CPU cycle; 3), End-Edge-Cloud Offloading: Cloud Server Computation Delay; The estimated computation delay at cloud server c is: in, Cloud server for digital twins Distributed to Industrial IoT end devices The CPU calculates the frequency estimate; similarly, the difference is Δf m,c , then the delay error is calculated as: Then the actual computing delay of cloud server c is: Cloud servers do not need to consider computing energy consumption; The total computational energy consumption is: Step S2-2-3, offloading delay calculation. Due to the parallelism of calculation and transmission, the offloading delay depends on the maximum value of the sum of the calculation and transmission delays of each part of the task: in, is the local offloading delay of the industrial IoT terminal device m, is the offloading delay of edge server k, is the offloading delay of cloud server c, Delay for unloading; The offloading energy consumption is the sum of the total transmission energy consumption and the total computation energy consumption: The system cost is: Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively, satisfying: Step S2-2-4, propose an optimization problem and complete the problem optimization under the premise of minimizing the task system cost.

6. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 1, characterized in that: In step 3, the target optimization problem is modeled as a multi-agent Markov decision problem. The specific steps are as follows: Step S3-1, the digital twin DT-UE of each industrial IoT terminal device is used as an intelligent agent; the Markov decision process of M intelligent agents can be used as a tuple express, Describes the state of the environment. Thanks to the digital twin layer, information exchange can be achieved between various subjects, and each agent's observation of the environment constitutes the system state space; is the set of actions performed by all agents; is the set of reward functions for each agent, defined as the benefit of performing an action in the current state; It is expressed as the probability of transitioning from the current state to the next completely new state; Step S3-2, state space In a digital twin, the environment is fully observed, so the state of the environment consists of the observations of each agent. s(t)={o1(t),...,o m (t),...,o M (t)} Among them m (t) represents the observation of agent m at time t; Step S3-3, action space The decisions made by the agents include the associated target, the offloading target, the bandwidth occupancy ratio, and the task offloading ratio. The actions of the m agents are described as: a m (t)={γ m (t),λ m (t),ξ m,k (t),ξ m,c (t)} where γ m (t) represents the task division ratio of agent m at time t, λ m (t) represents the bandwidth allocation ratio of agent m at time t, ξ m,k (t) represents the associated decision variable of agent m at time t, ξ m,c (t) represents the unloading decision variable of agent m at time t; Step S3-4, Reward Space Based on the current system state, the agent takes actions and receives rewards; The m agents should collaboratively minimize the total system cost while completing the computational tasks; therefore, the cost of each DT-UE is The negative value of is used as part of the reward; if a task is not completed within the specified time, there will be a penalty η in the reward function; therefore, the reward of the mth agent is expressed as: Among them, ω1 and ω2 are delay weight factor and energy consumption weight factor respectively; To uninstall the delay, To unload energy consumption.

7. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 1, characterized in that: In step 4, the collaborative computing and resource allocation CCRA algorithm is used to solve the Markov decision problem, output the offloading strategy, and complete the resource allocation optimization. The specific steps are as follows: Step S4-1-1: Deploy the DDPG algorithm on the digital twin DT-UE of each industrial IoT terminal device. The algorithm includes a weight Actor Module and one with weight Critic module Where o is the observation of all agents, a is the action of all agents, and the Actor module converts the local observation value o m As input, then output action a m (t) = μ m (o m (t)); The Critic module obtains the observations and actions of all agents during the training phase to give a Q value Q m (s(t), a(t)), evaluates the output of the Actor module; Step S4-1-2, using the method of converting discrete action space into probability distribution, for the continuous value γ m (t),λ m (t), discrete value ξ m,k (t),ξ m,c (t) constitutes the hybrid action space, and the Actor module of DT-UE is defined as where μ m is the strategy of the agent DT-UE, and the output action a of DT-UE m (t) = {γ m (t),λ m (t),ξ m,k (t),ξ m,c (t)}, where ξ m,k (t),ξ m,c (t) are the probability distributions of two discrete actions respectively; Step S4-1-3, using the experience replay mechanism in the MADDPG algorithm, at each moment, all agents can replay the current experience tuple Store to playback buffer Where a(t) is the action in the current time slot, is the reward of the current time slot, s(t) is the state of the current time slot, and s(t+1) is the state of the next time slot; for each DT-UE, in the playback buffer A batch of size (a j ,r j ,s j ,s' j ) of small batch samples, where a j is the randomly sampled action at the current moment, r j is the reward randomly sampled at the current moment, s j is the state randomly sampled at the current moment, s' j The state of random sampling at the next moment. Random sampling can break the correlation between sample data and reduce training oscillation; Step S4-1-4, s in the sampled experience sample j Input to the Actor module, and use the policy gradient strategy to update the weight of the Actor module. in, is the expected return gradient, N b is the size of the mini-batch samples, Actor module weights, which can be updated by minimizing the loss; In addition, the Critic module updates the parameters by minimizing the loss of the target Q value: The updated target y m (j) is calculated as, Where ε is the discount factor.

8. The digital twin-enabled industrial Internet of Things resource allocation system according to claim 7, characterized in that: The specific process of DT-assisted CCRA algorithm training is as follows: Step S4-2-1, first initialize the parameters of the four neural networks and the playback buffer; in each episode, each DT-UE will obtain the observation value of the environment o m (t), and then input to the Actor module, the output includes bandwidth allocation ratio, task division ratio, and probability distribution of associated targets and offload targets; the current sampling probability and action are described as: Step S4-2-2, after the DT-UE action is completed, the environment will switch to the next state s(t+1), and all DT-UEs will receive corresponding rewards and new observations o(t); agents that fail to complete the task will receive penalties; if the replay buffer is not full, the experience of this stage will be stored directly, otherwise the earliest batch of experience will be replaced; Step S4-2-3, then, DT-UE will use small batch samples to sample in the playback buffer to train the parameters of the neural network; the parameters of the Actor module and the Critic module are updated by minimizing the loss function; and the parameters of the two target networks are soft updated, which are, Where τ is the update rate; Step S4-2-4, after the training is completed, the Critic module is no longer needed, and the decision of DT-UE will act on the real environment. The mth agent makes decisions in the digital world based on local observations and transmits the action instruction a to the user device in the physical world in real time; then the changes in the physical world will be fed back to DT-UE in real time and enter the next state.

Images