A queue-aware cloud-edge collaborative reasoning method for digital twin synchronization

Abstract

This invention discloses a queue-aware cloud-edge collaborative reasoning method for digital twin synchronization, comprising the following steps: S1, establishing an inference task model based on the processing requirements of sensor data collection; S2, establishing a wireless channel model based on the wireless communication processes between the sensor device and the edge server, and between the edge server and the cloud server; S3, establishing a task queue model based on the processing process of each task in the edge server queue buffer; S4, establishing a collaborative reasoning model based on the collaborative reasoning process of the inference task on the edge server and the cloud server; S5, constructing an optimization objective function and a Markov model based on the low-latency requirements of digital twin synchronization; S6, using the MAPPO algorithm to establish sensor device and edge server agents, respectively, training the two agents based on the Markov model, and updating the current strategy. This invention can reduce the waiting delay of priority tasks and improve the efficiency of digital twin synchronization.

Description

Technical Field

[0001] The present invention relates to the technical field of the combination of edge computing and artificial intelligence, and in particular to a queue-aware cloud-edge collaborative reasoning method for digital twin synchronization. Background Art

[0002] A digital twin (DT) is a virtual model of a physical entity in a digital space. Maintaining the consistency of this virtual model requires DT synchronization through real-time data collection and deep neural network (DNN) analysis. However, this process typically requires significant overhead, and traditional approaches relying on inference on cloud servers (CS) or edge servers (ES) cannot meet the low-latency requirements of real-world projects. Therefore, in a cloud-edge network, the DNN model is divided into intermediate layers and deployed on CS and ES to achieve collaborative inference acceleration.

[0003] At the same time, due to the different priorities in the processing of different sensory data, prioritizing high-priority tasks can cause low-priority tasks to wait for extended periods, increasing the risk of DT synchronization failure. Previous research on collaborative reasoning has lacked in-depth understanding of task queue priority processing methods. For example, it has ignored the risk of edge server overload and failed to fully consider task priority and latency tolerance, resulting in limitations in handling multi-priority tasks and mixed queues. Summary of the Invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a queue-aware cloud-edge collaborative reasoning method for digital twin synchronization, which considers the differences in task priorities and the real-time status of task queues in the collaborative reasoning process of multi-priority tasks, dynamically optimizes DNN segmentation and task scheduling, reasonably allocates communication resources and computing resources, and reduces DT synchronization delay.

[0005] Technical solution: A queue-aware cloud-edge collaborative reasoning method for digital twin synchronization, including the following steps:

[0006] S1, based on the processing requirements of the data collected by the sensor equipment, establish an inference task model and assign the inference task priority, delay tolerance and computational complexity;

[0007] S2, based on the wireless communication process between the sensor device and the edge server, and the edge server and the cloud server, establishes a wireless channel model and divides the channel into independent sub-channels;

[0008] S3: Based on the processing of each task in the edge server queue buffer, a task queue model is established, and tasks of different priorities are optimized according to the M / M / 1 non-preemptive priority model.

[0009] S4: Based on the collaborative reasoning process of the reasoning task on the edge server and the cloud server, a collaborative reasoning model is established, and the collaborative reasoning latency is analyzed according to the processing flow of the reasoning task;

[0010] S5: Build an optimization objective function based on the low-latency requirement for digital twin synchronization. By dividing the optimization problem into time slots, we transform it into a partially observable Markov decision problem and construct a Markov model.

[0011] S6, use the MAPPO algorithm to establish the sensor device agent and the edge server agent respectively, train the two agents based on the Markov model and update the current strategy.

[0012] Furthermore, the specific steps for establishing the reasoning task model include:

[0013] In S11, the process of continuously collecting physical object state data by the sensor device and handing it over to the DNN model for reasoning is considered to be a Poisson process, where the task arrival rate is set to The total number of tasks generated within time T is ;

[0014] S12, define the sensor device set as , where N represents the total number of sensor devices; any sensor device SD n The kth reasoning task generated for:

[0015] ,

[0016] in, Indicates the data size; Indicates the digital twin synchronization priority. A larger value indicates a higher task priority. Indicates that the task tolerates delays. If the delay is exceeded, the task is considered failed.

[0017] Furthermore, the specific steps of establishing a wireless channel model include:

[0018] S21, define the edge server set as , where M represents the total number of edge servers; any edge server ES e The communication range of each sensor device covers multiple sensor devices, and each sensor device only communicates with one edge server. ; Define a set Indicates SD within the communication range n serial number, Indicates ES e SD within communication range n The total number, of which and , represents the empty set;

[0019] S22, SD n With ES e By subchannel set To communicate, Indicates ES e With SD n The total number of communication sub-channels, each communication selection A subchannel in express: Indicates SD n Use subchannel c to transmit the sensed data to the associated ES e To achieve digital twin synchronization; Indicates that the subchannel is not used; according to Shannon's formula, SD n to ES e Uplink wireless link transmission rate Expressed as:

[0020] ,

[0021] in, represents the bandwidth of subchannel c, represents the power of the additive white Gaussian noise of subchannel c, Indicates SD n The transmission power, represents the uplink channel gain;

[0022] Therefore, SD n The inference task is divided into two parts through the sub-channel c. Transfer to ES e Transmission delay Expressed as: ,

[0023] in, for The initial data volume;

[0024] S23, define ES e The set of subchannels communicating with the cloud server is , among which, ES e Using subchannels Communicate with the cloud server without mutual interference; therefore, ES e Uplink wireless link transmission rate to the cloud server Expressed as:

[0025] ,

[0026] in, Indicates ES e Corresponding subchannel bandwidth, Indicates ES e Corresponding subchannel The power of additive white Gaussian noise, Indicates ES e The transmission power, Indicates the uplink channel gain.

[0027] Furthermore, the specific steps to establish the task queue model include:

[0028] S31, define ES e The task queue is ,in Indicates SD n The kth reasoning task In ES e The position in the task queue, Indicates ES e service rate; and ES e Following the M / M / 1 non-preemptive priority channel model, high-priority tasks will interrupt low-priority tasks, but ongoing reasoning tasks will not be interrupted;

[0029] S32, when low priority reasoning task While waiting at position p in the queue, if q higher priority tasks enter the queue, In ES e The waiting delay x of the task queue is measured by the distribution function:

[0030] ,

[0031] in, represents the distribution function of the waiting delay x; e represents a natural constant, and q is the number of high-priority tasks entering the queue within the time slot;

[0032] S33, define the probability of digital twin synchronization failure as , during the waiting process, the system The probability of maintaining synchronization continuity, then the minimum waiting delay is .

[0033] Furthermore, the specific steps of establishing a collaborative reasoning model include:

[0034] S41, reasoning task DNN segmentation point Use split point identifier Represents, where L indicates the total number of feasible segmentation points of DNN, Indicates reasoning tasks The split point is , Representing reasoning tasks The split point is not ;

[0035] If the split point is Indicates reasoning tasks All are processed in the cloud server, and the split point is Indicates reasoning tasks All in ES e processing; in other cases, the first half of the DNN is processed in ES e Processing, then transferring the intermediate data to the cloud server to complete the second half of the reasoning;

[0036] S42, reasoning task In ES e After completing part of the inference, the transmission delay of the intermediate data to the cloud server Expressed as:

[0037] ,

[0038] in, The split point is When the reasoning task The amount of intermediate data transmitted to the cloud server; For ES e The transmission rate between the server and the cloud;

[0039] S43, based on the definition of the M / M / 1 queueing system, reasoning task In ES e Delay in stay Expressed as:

[0040] ,

[0041] in, For ES e The service rate of the task queue, Characterization of ES e computing power, For SD p The task arrival rate;

[0042] ,

[0043] ,

[0044] in, Indicates ES e The computational complexity of all inference tasks at Representing reasoning tasks Computational complexity, Indicates the computational complexity of the corresponding segmentation point; For SD p The total number of inference tasks completed by the transfer;

[0045] S44, reasoning task The total delay is:

[0046] ,

[0047] in, Indicates SD n To ES e Transmission delay.

[0048] Furthermore, the specific steps of constructing the Markov model include:

[0049] S51, through reasonable channel allocation and determination of the inference task splitting point, with minimizing the maximum cumulative delay of digital twin synchronization as the optimization goal, the optimization problem P1 is expressed as:

[0050] ,

[0051] Among them, constraint C1 ensures that each sensor device uses only one channel, constraint C2 ensures that each reasoning task has only one split point, and constraint C3 ensures that the reasoning task is completed within the delay tolerance limit; For SD n The total number of inference tasks generated;

[0052] S52, defines the time slot set for the digital twin synchronization process , where T is the total length of the time slot, and the interval between each time slot is a fixed time ; By dividing the time slots, the optimization problem P1 is transformed into the optimization problem P2 that conforms to the Markov model:

[0053] ,

[0054] in, Indicates the number of tasks completed in a time slot.

[0055] Furthermore, the sensor device agent model is as follows:

[0056] Defining SD n MDP state space corresponding to the agent , action space With the reward function ; where the state space , Indicates SD nWith ES e The channel gain between represents the number of sensing devices in the previous time slot channel c;

[0057] Action Space , For SD n a communication identifier transmitted using subchannel c;

[0058] Reward Function , Indicates SD within a single time slot n The number of tasks completed by the transfer, Indicates SD within a single time slot n The number of tasks that failed to synchronize, It's SD n Penalties for failure to synchronize digital twins;

[0059] The edge server agent model is as follows:

[0060] Defining ES e MDP state space corresponding to the agent , action space With the reward function ; where the state space , Indicates ES in the time slot e Task queue status, Indicates SD in the time slot n Task status, Indicates ES in the time slot e Channel gain with the cloud server;

[0061] Action Space , express Split point identifier;

[0062] Reward Function , Indicates ES in a single time slot e The number of tasks completed by the transfer, Indicates ES in a single time slot e The number of tasks that failed to synchronize, It is ES e Penalties for failure to synchronize digital twins;

[0063] The specific steps for training an agent include:

[0064] S621, set hyperparameters and initialize the agent model;

[0065] S622, at the beginning of each round, initialize environmental parameters, including: initializing the time slot to 0, initializing the task statistics to 0, clearing the experience pool, and resetting the communication distance between the sensor device and the edge server to a random number;

[0066] S623, obtain the current time slot SD n Environmental status ;SD n in accordance with Make decisions and determine current actions , and set the SD n With ES e communication channels;

[0067] S624, obtain the current time slot ES e Environmental status ;ES e in accordance with Make decisions with the current strategy and determine the current action , and set the SD n The reasoning task split point;

[0068] S625, update the environment state based on the time slot interval T0, realize the generation and processing of reasoning tasks, and count the number of tasks completed in the time slot and the number of failed tasks ;

[0069] S626, Intelligent Body SD n Based on the reward function , calculate the reward value of the current time slot ; Agent ES e Based on the reward function Calculate the reward value for the current time slot ;

[0070] S627, store the state, action, and reward of the current time slot into the experience pool. When the sample size in the experience pool reaches the preset number, use the saved samples to train the two agents separately and update the current strategy.

[0071] Compared with the prior art, the present invention has the following significant effects:

[0072] 1. Based on collaborative reasoning considering task priorities in cloud-edge networks, the present invention models the processing process of each task in the edge server queue buffer as a task queue model, and optimizes tasks of different priorities according to the M / M / 1 non-preemptive priority model to reduce the waiting delay of priority tasks and improve the synchronization efficiency of digital twins;

[0073] 2. The present invention models the digital twin synchronization process as an MDP, and further proposes a dual-type multi-agent deep reinforcement learning algorithm. Through the collaboration of sensing devices and edge servers, it dynamically adapts to task priorities and latency requirements, and autonomously determines channel allocation and DNN partitioning points. With minimizing the maximum cumulative delay of digital twin synchronization as the optimization goal, the current strategy is continuously updated through agent training to achieve the reduction of digital twin synchronization delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a schematic diagram of the digital twin synchronization scenario in the cloud-edge network of the present invention;

[0075] Figure 2 It is a flow chart of the queue-aware cloud-edge collaborative reasoning method constructed by the present invention;

[0076] Figure 3 It is a model diagram of the dual-type intelligent agent constructed by the present invention;

[0077] Figure 4 This is a simulation experiment diagram of the cumulative delay change of the sensor device under different DNN models according to an embodiment of the present invention;

[0078] Figure 5 Figure 1 is a simulation experiment showing the cumulative delay changes of sensor devices under different DNN models and with an increase in the number of sensor devices according to an embodiment of the present invention. (a) is the Vgg11 network model, and (b) is the ResNet18 network model.

[0079] Figure 6 This is a simulation experiment diagram of the cumulative delay change of the sensor device under different DNN models and the increase in the number of sub-channels in an embodiment of the present invention, where (a) is the Vgg11 network model and (b) is the MobileNetV2 network model. DETAILED DESCRIPTION

[0080] The present invention will be described in further detail below with reference to the accompanying drawings and specific implementations.

[0081] like Figure 1 As shown, the digital twin (DT) synchronization scenario of the present invention under the cloud-edge network is based on the network structure of sensor devices (SD), edge servers (ES) and cloud servers (CS).

[0082] The sensor device collection is , responsible for collecting state data of physical entities (such as temperature, pressure, images, etc.) and transmitting the data to the associated edge server via wireless channels. Each sensor device is bound to an edge server, and the appropriate channel must be selected during data transmission to reduce latency.

[0083] The edge server set is , responsible for receiving data transmitted by sensor devices, managing task queues, and dynamically adjusting the split points of the DNN model based on task priority and queue status. Due to the limited computing power of edge servers, they need to collaborate with cloud servers to complete inference tasks.

[0084] The cloud server has powerful computing capabilities, receives the intermediate inference results transmitted by the edge server, completes the remaining DNN inference, and feeds the results back to the digital twin system for synchronization.

[0085] like Figure 2 The figure shows a flowchart of the queue-aware cloud-edge collaborative reasoning method proposed in the present invention. The present invention aims to achieve efficient synchronization of digital twins under multi-priority reasoning tasks. The specific implementation steps are as follows:

[0086] Step 1: The processing requirements of the sensor data collected are modeled as inference tasks that follow a Poisson distribution. Based on the actual performance requirements, multiple attributes are assigned to the inference tasks, including priority, latency tolerance, and computational complexity. The specific steps are as follows:

[0087] Step 11: The sensor device continuously collects the state data of the physical object and sends it to the specified DNN model for inference. This process is considered a Poisson process, where the task arrival rate is set to , the total number of tasks generated in time T is .

[0088] Step 12: Define the sensor device set as , where N represents the total number of sensor devices. The sensor devices collect physical object state data and pass it to the specified DNN model for processing. This process is considered an inference task. Any sensor device SD n The kth reasoning task generated for:

[0089] ,

[0090] in, Indicates the data size; Indicates the digital twin synchronization priority. A larger value indicates a higher task priority. Indicates that the task tolerates delays. If the delay is exceeded, the task is considered failed.

[0091] Step 2: Model the wireless communication process between the sensor device and the edge server, and between the edge server and the cloud server as a wireless channel model, and divide the channel into independent sub-channels based on actual needs to improve transmission efficiency. The specific steps are as follows:

[0092] Step 21, define the edge server set as , where M represents the total number of edge servers. Any edge server ES e The communication range of each sensor device covers multiple sensor devices, and each sensor device only communicates with one edge server. ; Define a set Indicates ES e SD within communication range n serial number, Indicates ES e SD within communication range n The total number, of which and , Represents the empty set.

[0093] Step 22, define ES e With SD n The set of communication subchannels is , where these sub-channels are independent of each other, Indicates ES e With SD n The total number of sub-channels for communication. n With ES e Through the channel To communicate, you can select A subchannel in express: Indicates SD n Use subchannel c to transmit the sensed data to the associated ES e To achieve digital twin synchronization; , which means that the subchannel is not used. n There is mutual interference between them. According to Shannon’s formula, SD n to ES e Uplink wireless link transmission rate It can be expressed as:

[0094] ,

[0095] in, represents the bandwidth of subchannel c, represents the power of the additive white Gaussian noise of subchannel c, Indicates SD n The transmission power, Indicates the uplink channel gain.

[0096] Therefore, SD n The inference task is divided into two parts through the sub-channel c. Transfer to ES e Transmission delay It can be expressed as:

[0097] ,

[0098] in, For reasoning tasks The initial data volume.

[0099] Step 23, define ES e The set of subchannels communicating with the cloud server is , these sub-channels are independent of each other. The edge server and the cloud server use the channel ES e Using subchannels Communicate with the cloud server without mutual interference, so ES e Uplink wireless link transmission rate to the cloud server It can be expressed as:

[0100] ,

[0101] in, Indicates ES e Corresponding subchannel bandwidth, Represents ES e Corresponding subchannel The power of additive white Gaussian noise, Indicates ES e The transmission power, Indicates the uplink channel gain.

[0102] Step 3: Model the processing process of each task in the edge server queue buffer as a task queue model, and optimize tasks of different priorities based on the M / M / 1 non-preemptive priority model. The specific steps are as follows:

[0103] Step 31, define ES e The task queue is ,in Indicates SD n The kth reasoning task In ES e The position in the task queue, Indicates ES e Considering that reasoning tasks have different priorities, ES e Following the M / M / 1 non-preemptive priority channel model, high-priority tasks will interrupt low-priority tasks, but ongoing reasoning tasks will not be interrupted.

[0104] Step 32, define ES e Tasks in the queue The Cumulative Distribution Function (CDF) of the waiting time shows the impact of waiting time on the synchronization of digital twins. While waiting at position p in the queue, if q higher priority tasks enter the queue, In ES e The waiting delay x of the task queue can be expressed by CDF:

[0105] ,

[0106] in, represents the distribution function of the waiting delay x; e represents a natural constant, The position in the queue is p, and q is the number of high-priority tasks that enter the queue during the time slot.

[0107] Step 33: Define the probability of digital twin synchronization failure as , during the waiting process, the system The probability of maintaining synchronization continuity, further, the minimum value of the queuing waiting delay can be calculated as .

[0108] Step 4: Model the collaborative reasoning process of the reasoning task on the edge server and the cloud server as a collaborative reasoning model, and analyze the collaborative reasoning latency based on the processing flow of the reasoning task. The specific process is as follows:

[0109] Step 41: Reasoning Task DNN segmentation point Use split point identifier Represents, where L indicates the total number of feasible segmentation points of DNN, show The split point is , express The split point is not In particular, if the split point is show All are processed in the cloud server, and the split point is show All in ES e In other cases, the first half of DNN is processed in ES e The intermediate data is then transferred to the cloud server to complete the second half of the reasoning.

[0110] Step 42: Reasoning Task In ES e After completing part of the inference, the transmission delay of the intermediate data to the cloud server Expressed as:

[0111] ,

[0112] in, Indicates the split point is Time Reasoning Task The amount of intermediate data transmitted to the cloud server; Indicates ES e The transmission rate between the server and the cloud.

[0113] Step 43: Reasoning Task In ES e The process of queue queuing and task reasoning is required, and the delay caused by the process of queue queuing and task reasoning is collectively referred to as the stay delay;

[0114] The computational complexity is ,in Indicates the computational complexity of the corresponding split point.

[0115] ES e The computational complexity of all reasoning tasks can be expressed as ,in For SD p The task arrival rate, For SD p The total number of inference tasks that have completed transmission.

[0116] According to the definition of M / M / 1 queueing system, In ES e Delay in stay It can be expressed as:

[0117] ,

[0118] in, For ES e The service rate of the task queue, Characterization of ES e computing power.

[0119] Step 44: Reasoning Task Total delay Expressed as:

[0120] ,

[0121] By SD n To ES e Transmission delay , ES e Transmission delay to CS And in ES e Delay in stay composition.

[0122] Step 5: Based on step S4, an optimization objective function is constructed according to the low latency requirement of digital twin synchronization, and the optimization problem is converted into a partially observable Markov decision problem through time slot division to construct a Markov model.

[0123] Step 51: By reasonably allocating channels and determining the inference task splitting points, the optimization goal is to minimize the maximum cumulative delay of digital twin synchronization. The above optimization problem P1 can be expressed as:

[0124] ,

[0125] Among them, constraint C1 ensures that each sensor device uses only one channel, constraint C2 ensures that each reasoning task has only one split point, and constraint C3 ensures that the reasoning task is completed within the delay tolerance limit; For SD n The total number of inference tasks generated.

[0126] Step 52: Define the time slot set for the digital twin synchronization process , where T is the total length of the time slot, and the interval between each time slot is a fixed time T0. By dividing the time slots, the optimization problem P1 is transformed into the optimization problem P2 that conforms to the MDP (Markov Decision Process):

[0127] ,

[0128] in, Indicates the number of tasks completed in a time slot.

[0129] Step 6, such as Figure 3 As shown in the figure, a dual-type agent model of MAPPO (Multi-agent Proximal Policy Optimization) is used to establish agents for sensor devices and edge servers respectively, and to train the agents. The data transmission delay and inference delay are shortened through the strategy evaluation and optimization of the agents. 、 、 Represent the state, action and reward function of the Nth sensor device agent, respectively. 、 、 represent the state, action, and reward function of the Mth edge server agent, respectively.

[0130] Establishing a dual-type agent model based on the MAPPO algorithm includes the following steps:

[0131] Step 611, define SD n MDP state space corresponding to the agent , action space With the reward function . The state space , Indicates SD n With ES e The channel gain between represents the number of sensing devices in the previous time slot channel c; action space , For SD n Communication identifier transmitted using subchannel c; reward function , Indicates SD within a single time slot n The number of tasks completed by the transfer, Indicates SD within a single time slot n The number of tasks that failed to synchronize, It's SD n Penalty for failed digital twin synchronization.

[0132] Step 612, define ES e MDP state space corresponding to the agent , action space With the reward function . The state space , Indicates ES in the time slot e Task queue status, Indicates SD in the time slot n Task status, Indicates ES in the time slot e Channel gain with the cloud server; action space , express Split point identifier; reward function , Indicates ES in a single time slot e The number of tasks completed by the transfer, Indicates ES in a single time slot e The number of tasks that failed to synchronize, It is ES e Penalty for failed digital twin synchronization.

[0133] The specific steps for training an agent are as follows:

[0134] Step 621 sets hyperparameters and initializes the agent model. In the dual-agent model based on the MAPPO algorithm, the main parameters are set as follows: the actor learning rate is set to 1e-4, the critic learning rate is set to 5e-4, and the cumulative reward discount factor is set to 0.95. The experience pool capacity is set to 1024, and mini-batch training is used, with a sample size of 256 per batch. The number of training rounds is set to 10,000.

[0135] At the beginning of each round, step 622, the environment state parameters are initialized. The main parameters are set as follows: the time slot is initialized to 0, the task statistics are initialized to 0, the experience pool is cleared, and the communication distance between the sensor device and the edge server is reset to a random number.

[0136] Step 623: Get the current time slot sensor device status ;SD n Based on status Make decisions with the current strategy and determine the current action , and set the SD n With ES e communication channels.

[0137] Step 624: Get the edge server status of the current time slot ES e Based on status Make decisions with the current strategy and determine the current action , and set the SD n The reasoning task split point.

[0138] Step 625: Update the environment state based on the time slot interval T0, realize the generation and processing of reasoning tasks, and count the number of tasks completed in the time slot. and the number of failed tasks .

[0139] Step 626, Agent SD n Based on the reward function , calculate the reward value of the current time slot ; Agent ES e Based on the reward function Calculate the reward value for the current time slot .

[0140] In step 627, the state, action, and reward of the current time slot are stored in the experience pool. When the number of samples in the experience pool reaches a preset number, the two agents are trained using the stored samples to update the current strategy (the agents use the strategy to determine channel allocation and DNN partitioning).

[0141] To verify the superiority of the dual-type agent model based on the MAPPO algorithm in this invention, the maximum value of the cumulative digital twin synchronization delay of each sensor device is selected as the evaluation indicator and compared with the following three benchmark solutions:

[0142] F1, Edge only. There are no cloud servers in the system, and all DNN models are inferred at the edge servers.

[0143] F2, PPO (Proximal Policy Optimization): A centralized PPO algorithm is deployed on the cloud server, outputting both channel selection and model partitioning decisions. This method requires transmitting a large amount of data to the cloud server.

[0144] F3, Multi-Agent Deep Q-Networks (MADQN): Each sensor device is set as an agent and the DQN (Deep Q-Networks) algorithm is deployed to autonomously make channel selection and model partitioning decisions. However, this method fails to account for changes in queue status caused by queue interruptions by high-priority tasks.

[0145] like Figure 4 As shown in the figure, in order to verify the performance of the present invention under different DNN models, Vgg11, MobileNetV2 and ResNet18 are used as DNN models for experiments. Proposed represents the method proposed by the present invention. The number of sensor devices N is set to 15, the number of edge servers M is set to 3, and the total number of sub-channels of each ES is set to 15. Set to 3. Simulation results show that the proposed method can achieve the lowest latency, far exceeding the slow-converging MADQN. Compared with Edge-only, the average latency is reduced by 28%. Compared with centralized PPO, it shows lower digital twin synchronization latency. This proves the effectiveness of the proposed method in improving the efficiency of digital twin synchronization under mainstream models.

[0146] In order to verify the performance of the present invention under limited channel resources, experiments were conducted using Vgg11 and ResNet18 as DNN models. Figure 5 As shown in (a) and (b) in Figure 2. Among them, the number of sensing devices Set to 12, 15, 18, 21 and 24 respectively, the number of edge servers M is set to 3, and the total number of sub-channels of each edge server is Set to 3. Simulation results show that compared with the other three benchmark methods, the method proposed in this invention can still maintain good performance under the condition of tight channel resources, and as the number of sensing devices increases, the advantages of the method proposed in this invention become more obvious.

[0147] In order to verify the advantages of the present invention in channel allocation optimization, experiments were conducted using Vgg11 and MobileNetV2 as DNN models. Figure 6 As shown in (a) and (b) in the figure, the number of sensor devices N is set to 30, the number of edge servers M is set to 3, and the total number of sub-channels of each edge server is The settings are sequentially set to 4, 5, 6, 7 and 8. The simulation results show that as the number of channels increases, the interference between devices decreases, and all methods show a reduction in delay, but the method proposed in the present invention shows stronger adaptability.

[0148] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed in the present invention should be included in the protection scope recorded in the claims.

Claims

1. A queue-aware cloud-edge collaborative reasoning method for digital twin synchronization, characterized by: The steps include: S1, based on the processing requirements of the data collected by the sensor equipment, establish an inference task model and assign the inference task priority, delay tolerance and computational complexity; S2, based on the wireless communication process between the sensor device and the edge server, and the edge server and the cloud server, establishes a wireless channel model and divides the channel into independent sub-channels; S3: Based on the processing of each task in the edge server queue buffer, a task queue model is established, and tasks of different priorities are optimized according to the M / M / 1 non-preemptive priority model. S4: Based on the collaborative reasoning process of the reasoning task on the edge server and the cloud server, a collaborative reasoning model is established, and the collaborative reasoning latency is analyzed according to the processing flow of the reasoning task; S5: Build an optimization objective function based on the low-latency requirement for digital twin synchronization. By dividing the optimization problem into time slots, we transform it into a partially observable Markov decision problem and construct a Markov model. S6, using the MAPPO algorithm to establish the sensor device agent and the edge server agent respectively, and train the two agents based on the Markov model and update the current strategy; In step S4, the specific steps of establishing the collaborative reasoning model include: S41, Reasoning Task I n,k DNN segmentation point Use split point identifier Represents, where L indicates the total number of feasible segmentation points of DNN, Indicates that the reasoning task I n,k The split point is l, Represents reasoning task I n,k The split point is not l; where I n,k Indicates any sensor device SD n The kth reasoning task generated; If the split point is l0, it means that the reasoning task I n,k All processed in the cloud server, the split point is l L Indicates that the reasoning task I n,k All in ES e processing; in other cases, the first half of the DNN is processed in ES e Processing, then transferring the intermediate data to the cloud server to complete the second half of the reasoning; Among them, ES e represents any edge server; S42, Reasoning Task I n,k In ES e After completing part of the inference, the transmission delay of the intermediate data to the cloud server Expressed as: in, When the split point is l, the reasoning task I n,k The amount of intermediate data transmitted to the cloud server; R e For ES e The transmission rate between the server and the cloud; S43, based on the definition of the M / M / 1 queueing system, reasoning task I n,k In ES e Delay in stay Expressed as: in, For ES e The service rate of the task queue, F e Characterization of ES e The computing power of p For SD p The task arrival rate; Indicates SD within the communication range n Number, N e Indicates ES e SD within communication range n The total number, of which and represents the empty set; is the set of sensor devices, N represents the total number of sensor devices; is the set of edge servers, M represents the total number of edge servers; Among them, C e Indicates ES e The computational complexity of all reasoning tasks at C n,k Represents reasoning task I n,k Computational complexity, f n,k,l Indicates the computational complexity of the corresponding segmentation point; K p For SD p The total number of inference tasks completed by the transfer; S44, Reasoning Task I n,k The total delay is: in, Indicates SD n To ES e Transmission delay; In step S5, the specific steps of constructing the Markov model include: S51, through reasonable channel allocation and determination of the inference task splitting point, with minimizing the maximum cumulative delay of digital twin synchronization as the optimization goal, the optimization problem P1 is expressed as: Among them, constraint C1 ensures that each sensor device uses only one channel, constraint C2 ensures that each reasoning task has only one split point, and constraint C3 ensures that the reasoning task is completed within the delay tolerance limit; K n For SD n The total number of inference tasks generated; SD n With ES e By subchannel set To communicate, M e Indicates ES e With SD n The total number of communication sub-channels, each communication selection A subchannel in n,c ∈{0,1} means: α n,c =1 indicates SD n Use subchannel c to transmit the sensed data to the associated ES e To achieve digital twin synchronization; α n,c =0 means that the subchannel is not used; Indicates the task's tolerance for delay; S52, defines the time slot set for the digital twin synchronization process Where T is the total length of the time slot, and the interval between each time slot is a fixed time T0. By dividing the time slots, the optimization problem P1 is transformed into the optimization problem P2 that conforms to the Markov model: stC1, C2, C3 Where K(t) represents the number of tasks completed in a time slot; In step S6, the sensor device agent model is as follows: Defining SD n MDP state space corresponding to the agent Action Space With the reward function The state space h n,e (t) represents SD n With ES e The channel gain between e,c (t-1) represents the number of sensing devices in channel c in the previous time slot; Action Space α n,c (t) is SD n a communication identifier transmitted using subchannel c; Reward Function K n (t) represents the SD in a single time slot n The number of tasks completed by the transmission, F n (t) represents the SD in a single time slot n The number of tasks that failed to synchronize, η n It's SD n Penalties for failure to synchronize digital twins; The edge server agent model is as follows: Defining ES e MDP state space corresponding to the agent Action Space With the reward function The state space Q e (t) represents ES in the time slot e Task queue status, I n (t) represents the SD in the time slot n Task status, h e (t) represents ES in the time slot e Channel gain with the cloud server; action space Indicates I n,k Split point identifier; Reward Function K e (t) represents ES in a single time slot e The number of tasks completed by the transmission, F e (t) represents ES in a single time slot e The number of tasks that failed to synchronize, η e It is ES e Penalties for failure to synchronize digital twins; The specific steps for training an agent include: S621, set hyperparameters and initialize the agent model; S622, at the beginning of each round, initialize environmental parameters, including: initializing the time slot to 0, initializing the task statistics to 0, clearing the experience pool, and resetting the communication distance between the sensor device and the edge server to a random number; S623, obtain the current time slot SD n Environmental status SD n in accordance with Make decisions and determine current actions And set the SD in the current time slot n With ES e communication channels; S624, obtain the current time slot ES e Environmental status ES e in accordance with Make decisions with the current strategy and determine the current action And set the SD in the current time slot n The reasoning task split point; S625, update the environment state based on the time slot interval T0, realize the generation and processing of reasoning tasks, and count the number of tasks completed in the time slot K n (t) and the number of failed tasks F n (t); S626, Intelligent Body SD n Based on the reward function Calculate the reward value for the current time slot Agent ES e Based on the reward function Calculate the reward value for the current time slot S627, store the state, action, and reward of the current time slot into the experience pool. When the sample size in the experience pool reaches the preset number, use the saved samples to train the two agents separately and update the current strategy.

2. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 1 is characterized in that: The specific steps to build a reasoning task model include: In S11, the process of continuously collecting physical object state data by the sensor device and handing it over to the DNN model for reasoning is considered to be a Poisson process, where the task arrival rate is set to λ n The total number of tasks generated within time T is K n,T ; S12, any sensor device SD n The kth reasoning task I generated n,k for: Among them, D n,k Indicates the data size; γ n,k Indicates the digital twin synchronization priority. A larger value indicates a higher task priority. Indicates that the task tolerates delays. If the delay is exceeded, the task is considered failed.

3. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 1 is characterized in that: The specific steps to establish a wireless channel model include: S21, any edge server ES e The communication range of each sensor device covers multiple sensor devices, and each sensor device only communicates with one edge server. S22, according to Shannon's formula, SD n to ES e Uplink wireless link transmission rate R n,e Expressed as: Among them, B c represents the bandwidth of subchannel c, represents the power of the additive white Gaussian noise of subchannel c, p n Indicates SD n The transmission power, h n,e represents the uplink channel gain; Therefore, SD n The inference task I is divided into two parts through the sub-channel c. n,k Transfer to ES e Transmission delay Expressed as: in, For I n,k The initial data volume; S23, define ES e The set of subchannels communicating with the cloud server is Among them, ES e Use subchannel c e Communicate with the cloud server without mutual interference; therefore, ES e Uplink wireless link transmission rate to the cloud server R e Expressed as: Among them, B e Indicates ES e Corresponding subchannel c e bandwidth, Indicates ES e Corresponding subchannel c e The power of additive white Gaussian noise, P e Indicates ES e The transmission power, h e Indicates the uplink channel gain.

4. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 3 is characterized in that: The specific steps to establish a task queue model include: S31, define ES e The task queue is Q e ={pos n,k ,μ e }, where pos n,k Indicates SD n The kth reasoning task I n,k In ES e The position in the task queue; and ES e Following the M / M / 1 non-preemptive priority channel model, high-priority tasks will interrupt low-priority tasks, but ongoing reasoning tasks will not be interrupted; S32, when the low priority reasoning task I n,k When waiting at position p in the queue, if q higher priority tasks enter the queue, I n,k In ES e The waiting delay x of the task queue is measured by the distribution function: Among them, F n,k (x) represents the distribution function of the waiting delay x; e represents a natural constant, and q is the number of high-priority tasks entering the queue in the time slot; S33, define the probability of digital twin synchronization failure as ε. During the queuing process, the system maintains synchronization continuity with a probability of at least 1-ε. The minimum value of the queuing delay is min{x:F m,k (x)≥1-ε}.

Images