Queue awareness cloud edge collaborative reasoning method oriented to digital twinning synchronization

Abstract

The invention discloses a digital twinning synchronization-oriented queue sensing cloud edge collaborative reasoning method, which comprises the following steps of: S1, establishing a reasoning task model according to a processing requirement of data acquired by sensing equipment; s2, establishing a wireless channel model according to a wireless communication process between the sensing device and the edge server and a wireless communication process between the edge server and the cloud server; s3, establishing a task queue model according to the processing process of each task in the queue buffer area of the edge server; s4, establishing a collaborative reasoning model according to the collaborative reasoning process of the reasoning task on the edge server and the cloud server; s5, constructing an optimized objective function and a Markov model according to the low-delay demand of digital twin synchronization; and S6, a sensing device intelligent agent and an edge server intelligent agent are established by using an MAPPO algorithm, the two intelligent agents are trained based on a Markov model, and a current strategy is updated. According to the invention, the waiting time delay of priority tasks can be reduced, and the digital twinning synchronization efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of the combination of edge computing and artificial intelligence, and particularly relates to a queue-aware cloud-edge collaborative inference method for digital twin synchronization. Background Art

[0002] Digital Twin (DT) is a virtual model of a physical entity in the digital space. To maintain the virtual-real consistency of the virtual model, DT synchronization needs to be achieved through real-time data collection and Deep Neural Networks (DNN) analysis. However, the above process usually requires a large amount of overhead, and the traditional methods relying on cloud server (CS) inference or edge server (ES) inference cannot meet the low-latency requirements of actual projects. Therefore, in the cloud-edge network, the DNN model is divided into intermediate layers and deployed on the CS and ES respectively to achieve collaborative inference acceleration.

[0003] At the same time, since there are priorities in the processing of different sensing data, giving priority to high-priority tasks may lead to too long waiting delays for low-priority tasks, increasing the risk of DT synchronization failure. Previous research on collaborative inference lacks in-depth research on the task queue priority processing method. For example, it ignores the risk of edge server overload, does not fully consider task priorities and latency tolerance, and has limitations in processing multi-priority tasks and mixed queues. Summary of the Invention

[0004] Object of the Invention: The object of the present invention is to provide a queue-aware cloud-edge collaborative inference method for digital twin synchronization, which considers the differences in task priorities and the real-time status of task queues during the collaborative inference process of multi-priority tasks, dynamically optimizes DNN segmentation and task scheduling, rationally allocates communication resources and computing resources, and reduces DT synchronization latency.

[0005] Technical Solution: A queue-aware cloud-edge collaborative inference method for digital twin synchronization includes the following steps: S1, establish an inference task model according to the processing requirements of the data collected by the sensing device, and assign priorities, latency tolerance, and computational complexity to the inference tasks; S2, establish a wireless channel model according to the wireless communication processes between the sensing device and the edge server, and between the edge server and the cloud server, and divide the channel into independent sub-channels; S3, establish a task queue model according to the processing processes of the tasks in the edge server queue buffer, and optimize the tasks with different priorities according to the M / M / 1 non-preemptive priority model; S4. Establish a collaborative inference model based on the collaborative inference process of the inference task on the edge server and the cloud server, and analyze the collaborative inference delay according to the processing flow of the inference task. S5. Construct an optimization objective function according to the low-latency requirements of digital twin synchronization, and through time slot division, transform the optimization problem into a partially observable Markov decision problem, and construct a Markov model. S6. Use the MAPPO algorithm to establish a sensing device agent and an edge server agent respectively, and train and update the current policy for the two agents based on the Markov model.

[0006] Furthermore, the specific steps for establishing the inference task model include: S11. Consider the process that the sensing device continuously collects the physical object status data and submits it to the DNN model for inference as a Poisson process, where the task arrival rate is set to , and the total number of tasks generated within time T is . S12. Define the set of sensing devices as , where N represents the total number of sensing devices; the kth inference task n generated by any sensing device SD is: . Among them, represents the data size; represents the digital twin synchronization priority, and the larger the value, the higher the task priority; represents the tolerance delay of this task, and if the time is exceeded, the task is regarded as failed.

[0007] Furthermore, the specific steps for establishing the wireless channel model include: S21. Define the set of edge servers as , where M represents the total number of edge servers; the communication range of any edge server ES e covers multiple sensing devices, and each sensing device communicates with only one edge server, ; define the set to represent the SD n numbers within the communication range, represents the ES e the total number of SD n within the communication range, where and . represents the empty set; S22. SD n communicates with ES e through the sub-channel set . Denote ES e The total number of sub-channels communicating with SD n is selected for each communication. One sub-channel in is selected, and is denoted by the communication identifier as: It indicates that SD n uses sub-channel c to transmit the sensed data to the associated ES e to achieve digital twin synchronization; It means that this sub-channel is not used; According to Shannon's formula, the uplink wireless link transmission rate from SD n to ES e is denoted as: , where represents the bandwidth of sub-channel c, represents the power of additive white Gaussian noise of sub-channel c, represents the transmission power of SD n , represents the uplink channel gain; Therefore, the transmission delay of the inference task from SD n through sub-channel c to ES is e denoted as: , where is the initial data volume of ; S23, define the set of sub-channels for ES e to communicate with the cloud server as , where ES e uses sub-channel to communicate with the cloud server without mutual interference; Therefore, the uplink wireless link transmission rate from ES e to the cloud server is denoted as: , where represents the bandwidth of the corresponding sub-channel of ES e , represents the power of additive white Gaussian noise of the corresponding sub-channel of ES e , represents the transmission power of ES e , represents the uplink channel gain.

[0008] Furthermore, the specific steps to establish a task queue model include:​​ S31. Define ES e 's task queue as , where represents the k-th inference task of SD n , and is the position of the task in the ES e task queue, represents the service rate of ES e ; and ES e follows the M / M / 1 non-preemptive priority channel model, where high-priority tasks can cut in front of low-priority tasks, but ongoing inference tasks will not be interrupted; S32. When a low-priority inference task is waiting at position p in the queue, if q higher-priority tasks enter the queue, the waiting delay x of the task in the ES e task queue is measured by the distribution function: , where represents the distribution function of the waiting delay x; e represents the natural constant, and q is the number of high-priority tasks entering the queue within a time slot; S33. Define the digital twin synchronization failure probability as . During the queuing process, the system maintains the continuity of synchronization with at least probability, then the minimum value of the queuing waiting delay is .

[0009] Furthermore, the specific steps to establish the collaborative inference model include: S41. The DNN segmentation point of the inference task is represented by the segmentation point identifier , where L indicates the total number of feasible segmentation points of the DNN, indicates that the segmentation point of the inference task is , indicates that the segmentation point of the inference task is not ; If the segmentation point is it indicates that the entire inference task is processed on the cloud server, and the segmentation point indicates that the entire inference task is processed on the ES e ; in other cases, the first half of the DNN is processed on the ES e , and then the intermediate data is transmitted to the cloud server to complete the second half of the inference; S42. The inference task At ES e After partial inference is completed, the transmission delay of transmitting the intermediate data to the cloud server Is expressed as: , Wherein, Is the splitting point at When the inference task The amount of intermediate data transmitted to the cloud server; Is ES e The transmission rate between and the cloud server; S43. According to the definition of the M / M / 1 queuing system, the sojourn delay of the inference task At ES e Is expressed as: Is expressed as: , Wherein, Is the service rate of the ES e Task queue, Characterizes the ES e Computing power, Is the task arrival rate of SD p ; , , Wherein, Represents the computational complexity of all inference tasks at ES e , Represents the inference task Computational complexity, Represents the computational complexity of the corresponding splitting point; Is the total number of inference tasks completed by SD p Transmission; S44, the total delay of the inference task Is: , Wherein, Represents the transmission delay from SD n To ES e .

[0010] Furthermore, the specific steps for constructing the Markov model include: S51. Through reasonable channel allocation and determination of the inference task splitting point, with the goal of minimizing the maximum cumulative delay of digital twin synchronization, the optimization problem P1 is expressed as: , Among them, constraint C1 ensures that each sensing device uses only one channel, constraint C2 ensures that there is only one splitting point for each inference task, and constraint C3 ensures that the inference task is completed within the delay tolerance limit; For SD n The total number of inference tasks generated; S52, defining the time slot set of the digital twin synchronization process , where T is the total length of the time slot, and the interval time of each time slot is a fixed time ; Through time slot division, the optimization problem P1 is transformed into an optimization problem P2 that conforms to the Markov model: , Among them, represents the number of tasks completed within a time slot.

[0011] Furthermore, the sensing device agent model is as follows: Define SD n The MDP state space corresponding to the agent , action space and reward function ; Among them, the state space , represents SD n and ES e The channel gain between them, represents the number of sensing devices on channel c in the previous time slot; Action space , is the communication identifier used by SD n to transmit on sub-channel c; Reward function , represents the number of tasks completed by SD n within a single time slot, represents the number of tasks for which SD n synchronization fails within a single time slot, is the penalty for the digital twin synchronization failure of SD n ; The edge server agent model is as follows: Define ES e The MDP state space corresponding to the agent , action space and reward function ; Among them, the state space , represents the task queue status of ES e within the time slot, represents the task status of SD n within the time slot, Indicates the ES within a time slot e Channel gain between the cloud server and the ES Action space , Indicates Split point identifier Reward function , Indicates the number of tasks completed by the ES within a single time slot e , Indicates the number of tasks with synchronization failures by the ES within a single time slot e , Is the penalty for the digital twin synchronization failure of the ES e , The specific steps for training the agent include: S621, Set hyperparameters and initialize the agent model S622, At the start of each round, initialize the environmental parameters, including: Initialize the time slot to 0, initialize the task statistics to 0, empty the experience pool, and reset the communication distance between the sensing device and the edge server to a random number S623, Obtain the current time slot SD n Environmental state ; SD n Based on Make a decision to determine the current action , and set the communication channel between the current time slot SD n And the ES e , S624, Obtain the current time slot ES e Environmental state ; ES e Based on And the current policy to make a decision to determine the current action , and set the inference task split point of the current time slot SD n , S625, Update the environmental state at the time slot interval T 0 As the scale to generate and process the inference tasks, and count the number of tasks completed within the time slot And the number of failed tasks ; S626, The agent SD n Based on the reward function , Calculate the reward value of the current time slot ; The agent ES e Based on the reward function Calculate the reward value of the current time slot ; S627. Store the state, action, and reward of the current time slot in the experience pool. After the number of samples in the experience pool reaches the preset quantity, use the saved samples to train the two agents respectively and update the current policy.

[0012] Compared with the prior art, the present invention has the following remarkable effects: 1. Based on collaborative inference considering task priorities in a cloud-edge network, the present invention models the processing process of each task in the edge server queue buffer as a task queue model, and optimizes tasks with different priorities according to the M / M / 1 non-preemptive priority model to reduce the waiting delay of high-priority tasks and improve the digital twin synchronization efficiency. 2. The present invention models the digital twin synchronization process as an MDP, and further proposes a dual-type multi-agent deep reinforcement learning algorithm. Through the cooperation between the sensing device and the edge server, it dynamically adapts to task priorities and latency requirements, and autonomously determines channel allocation and DNN partitioning points. With the goal of minimizing the maximum cumulative delay of digital twin synchronization, by training the agents and continuously updating the current policy, it realizes reducing the digital twin synchronization delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of the digital twin synchronization scenario for the cloud-edge network according to the present invention; Figure 2 is a flowchart of the queue-aware cloud-edge collaborative inference method constructed by the present invention; Figure 3 is a model diagram of the dual-type agent constructed by the present invention; Figure 4 is a simulation experiment diagram of the cumulative latency change of the sensing device under different DNN models in the embodiment of the present invention; Figure 5 is a simulation experiment diagram of the cumulative latency change of the sensing device under different DNN models and with the increase in the number of sensing devices in the embodiment of the present invention, where (a) is the Vgg11 network model and (b) is the ResNet18 network model; Figure 6 is a simulation experiment diagram of the cumulative latency change of the sensing device under different DNN models and with the increase in the number of sub-channels in the embodiment of the present invention, where (a) is the Vgg11 network model and (b) is the MobileNetV2 network model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] The present invention will be further described in detail below with reference to the accompanying drawings of the specification and the specific embodiments.

[0015] As Figure 1As shown, this is the digital twin (DT) synchronization scenario of the present invention for the cloud-edge network, based on the network structure of sensor devices (SDs), edge servers (ESs), and cloud servers (CSs).

[0016] The set of sensor devices is , which is responsible for collecting the status data of physical entities (such as temperature, pressure, images, etc.) and transmitting the data to the associated edge server through a wireless channel. Each sensor device is bound to an edge server, and a suitable channel needs to be selected during data transmission to reduce latency.

[0017] The set of edge servers is , which is responsible for receiving the data transmitted by the sensor devices, managing the task queue, and dynamically adjusting the segmentation point of the DNN model according to the task priority and queue status. Due to the limited computing power of the edge server, it needs to cooperate with the cloud server to complete the inference task.

[0018] The cloud server has powerful computing capabilities. It receives the intermediate inference results transmitted by the edge server, completes the remaining DNN inferences, and feeds back the results to the digital twin system to achieve synchronization.

[0019] As Figure 2 shown, this is the flowchart of the queue-aware cloud-edge collaborative inference method proposed by the present invention. The present invention aims to achieve efficient synchronization of digital twins under multi-priority inference tasks. The specific implementation steps are as follows: Step 1, the processing requirements of the data collected by the sensor devices are modeled as inference tasks that follow a Poisson distribution, and multiple attributes including priority, latency tolerance, and computational complexity are assigned to the inference tasks according to the actual performance requirements. The specific steps are as follows: Step 11, the sensor devices continuously collect the status data of physical objects and hand it over to the specified DNN model for inference. This process is considered a Poisson process, where the task arrival rate is set to , and the total number of tasks generated within time T is .

[0020] Step 12, define the set of sensor devices as , where N represents the total number of sensor devices. The sensor devices collect the status data of physical objects and hand it over to the specified DNN model for processing, which is regarded as an inference task. The k-th inference task n generated by any sensor device SD is: , where represents the data size; represents the digital twin synchronization priority, and the larger the value, the higher the task priority; It indicates that the task tolerates time delay, and if the time is exceeded, the task is regarded as failed.

[0021] Step 2: Model the wireless communication processes between the sensing devices and the edge servers, and between the edge servers and the cloud servers as a wireless channel model, and divide the channel into independent sub-channels according to actual requirements to improve the transmission efficiency. The specific steps are as follows: Step 21: Define the set of edge servers as , where M represents the total number of edge servers. The communication range of any edge server ES e covers multiple sensing devices, and each sensing device communicates with only one edge server, ; define the set to represent the SD e numbers within the communication range of ES n , to represent the total number of SD e within the communication range of ES n , where and , represents the empty set.

[0022] Step 22: Define the set of sub-channels for communication between ES e and SD n as , where these sub-channels are independent of each other, represents the total number of sub-channels for communication between ES e and SD n . SD n communicates with ES e through the channel , and each communication can select a sub-channel from, and is represented by the communication identifier : indicates that SD n transmits the sensed data to the associated ES e to achieve digital twin synchronization; , which means that the sub-channel is not used. There is mutual interference between SD n using the same sub-channel. According to the Shannon formula, the uplink wireless link transmission rate n from SD to ES e can be expressed as: , where, represents the bandwidth of sub-channel c, represents the power of additive white Gaussian noise of sub-channel c, represents SD n ​The transmission power represents the uplink channel gain.

[0023] Therefore, SD n transmits the inference task to ES e The transmission delay can be expressed as: , where is the initial data volume of the inference task .

[0024] Step 23, define the set of subchannels for ES e to communicate with the cloud server as , and these subchannels are independent of each other. The edge server and the cloud server communicate through the channel . Among them, ES e uses the subchannel to communicate with the cloud server without mutual interference. Therefore, the uplink wireless link transmission rate of ES e to the cloud server can be expressed as: , where represents the bandwidth of the subchannel corresponding to ES e , respectively represent the power of the additive Gaussian white noise of the subchannel corresponding to ES e , , represents the transmission power of ES e ,

[0025] Step 3, model the processing process of each task in the edge server queue buffer as a task queue model, and optimize the tasks with different priorities according to the M / M / 1 non-preemptive priority model. The specific steps are as follows: Step 31, define the task queue of ES e as , where represents the position of the kth inference task of SD n in the task queue of ES , e and represents the service rate of ES e . Considering that the inference tasks have different priorities, ES eFollowing the M / M / 1 non-preemptive priority channel model, high-priority tasks will jump ahead of low-priority tasks, but ongoing inference tasks will not be interrupted.

[0026] Step 32, define ES e Tasks in the queue The waiting delay distribution function (Cumulative Distribution Function, CDF) represents the impact of waiting delay on digital twin synchronization. When a low-priority inference task is waiting at position p in the queue, if q higher-priority tasks enter the queue, In ES e The waiting delay x of the task queue can be represented by the CDF: , where, represents the distribution function of the waiting delay x; e represents the natural constant, The position in the queue is p, and q is the number of higher-priority tasks entering the queue within the time slot.

[0027] Step 33, define the digital twin synchronization failure probability as , during the queuing waiting process, the system maintains the continuity of synchronization with at least probability. Further, the minimum value of the queuing waiting delay can be calculated as .

[0028] Step 4, model the collaborative inference process of the inference task on the edge server and the cloud server as a collaborative inference model, and analyze the collaborative inference delay based on the processing flow of the inference task. The specific process is as follows: Step 41, the DNN splitting point of the inference task is represented by the splitting point identifier , where L indicates the total number of feasible splitting points of the DNN, indicates the splitting point of is denotes the splitting point of is not . Particularly, if the splitting point is it indicates that is all processed on the cloud server, and the splitting point is it indicates that e is all processed on ES e . In other cases, the first half of the DNN is processed on ES

[0029] Step 42, Inference Task At ES e After partial inference is completed, the transmission delay of transmitting the intermediate data to the cloud server Is expressed as: , Wherein, Indicates that the segmentation point is When the inference task The amount of intermediate data transmitted to the cloud server; Indicates ES e The transmission rate between and the cloud server.

[0030] Step 43, Inference Task At ES e It is necessary to experience queue queuing and task inference, and the delay generated during the queue queuing and task inference process is collectively referred to as the sojourn delay;

[0031] The computational complexity is , where Indicates the computational complexity corresponding to the segmentation point.

[0032] ES e The computational complexity of all inference tasks at can be expressed as , where Is the task arrival rate of SD p , Is the total number of inference tasks completed by SD p Transmission.

[0033] According to the definition of the M / M / 1 queuing system, At ES e The sojourn delay Can be expressed as: , Wherein, Is the service rate of the ES e Task queue, Characterizes ES e Computing power.

[0034] Step 44, Inference Task Total delay Is expressed as: , From SD n To ES e Transmission delay 、ES e To CS transmission delay And at ES eSojourn delay consists of

[0035] Step 5, based on Step S4, construct an optimization objective function according to the low-latency requirements of digital twin synchronization, and through time slot division, transform the optimization problem into a partially observable Markov decision problem and construct a Markov model.

[0036] Step 51, through reasonable channel allocation and determination of the inference task segmentation point, with the goal of minimizing the maximum cumulative delay of digital twin synchronization, the above optimization problem P1 can be expressed as: , where the constraint C1 ensures that each sensing device uses only one channel, the constraint C2 ensures that there is only one segmentation point for each inference task, and the constraint C3 ensures that the inference task is completed within the latency tolerance limit; is the total number of inference tasks generated by SD n

[0037] Step 52, define the time slot set of the digital twin synchronization process , where T is the total length of the time slot, and the interval time of each time slot is a fixed time T 0 . Through time slot division, transform the optimization problem P1 into an optimization problem P2 that conforms to the MDP (Markov Decision Process): , where, represents the number of tasks completed within a time slot.

[0038] Step 6, as Figure 3 shown, use the dual-type agent model of MAPPO (Multi-agent Proximal Policy Optimization) to establish agents for sensing devices and edge servers respectively, and conduct agent training. Through the policy evaluation and optimization of the agents, shorten the data transmission latency and inference latency, where , , represent the state, action, and reward function of the Nth sensing device agent respectively, , , represent the state, action, and reward function of the Mth edge server agent respectively.

[0039] Establish a dual-type agent model based on the MAPPO algorithm, which specifically includes the following steps: Step 611, define SD n ​MDP state space corresponding to the agent , action space and reward function . Among them, the state space , represents the channel gain between SD n and ES e ; represents the number of sensing devices of the previous time slot channel c; the action space , is the communication identifier used by SD n to transmit on sub-channel c; the reward function , represents the number of tasks completed by SD n in a single time slot, represents the number of tasks with synchronization failures of SD n in a single time slot, is the penalty for the digital twin synchronization failure of SD n .

[0040] Step 612, define the MDP state space e corresponding to the ES , action space and reward function . Among them, the state space , represents the task queue state of ES e in a time slot, represents the task state of SD n in a time slot, represents the channel gain between ES e and the cloud server in a time slot; the action space , represents the splitting point identifier; the reward function , represents the number of tasks completed by ES e in a single time slot, represents the number of tasks with synchronization failures of ES e in a single time slot, is the penalty for the digital twin synchronization failure of ES e .

[0041] The specific steps for training the agent are as follows: Step 621: Set hyperparameters and initialize the agent model. In the dual-type agent model based on the MAPPO algorithm, the main parameters are set as follows: the learning rate of the Actor in the agent is set to 1e-4, the learning rate of the Critic is set to 5e-4, and the discount factor of the cumulative reward is 0.95. The capacity of the experience pool is set to 1024, and mini-batch training is used, with the sample size of each batch being 256. The number of training rounds is set to 10000.

[0042] Step 622: At the beginning of each round, initialize the environmental state parameters. The main parameters are set as follows: the time slot is initialized to 0, the task statistic is initialized to 0, the experience pool is emptied, and the communication distance between the sensing device and the edge server is reset to a random number.

[0043] Step 623: Obtain the status of the sensing device at the current time slot ; SD n Make a decision based on the status and the current policy to determine the current action , and set the communication channel between the SD n and the ES e at the current time slot.

[0044] Step 624: Obtain the status of the edge server at the current time slot . ES e Make a decision based on the status and the current policy to determine the current action , and set the inference task splitting point of the SD n at the current time slot.

[0045] Step 625: Update the environmental state at the scale of the time slot interval T 0 to generate and process the inference tasks, and count the number of tasks completed within the time slot and the number of failed tasks .

[0046] Step 626: The agent SD n calculates the reward value of the current time slot according to the reward function ; the agent ES calculates the reward value of the current time slot according to the reward function e ; the agent ES calculates the reward value of the current time slot .

[0047] Step 627: Store the status, action, and reward of the current time slot in the experience pool. After the number of samples in the experience pool reaches the preset quantity, train the two agents respectively with the saved samples to update the current policy (the agent determines the channel allocation and DNN division through the policy).

[0048] To verify the superiority of the dual-type agent model based on the MAPPO algorithm in the present invention, the maximum value of the cumulative digital twin synchronization delay of each sensing device is selected as the evaluation index, and it is compared with the following three benchmark solutions: F1, Edge only (only edge processing). There is no cloud server in the system, and all DNN models are inferred at the edge server.

[0049] F2, PPO (Proximal Policy Optimization). A centralized PPO algorithm is deployed at the cloud server, which outputs channel selection and model partitioning decisions simultaneously. This method requires transmitting a large amount of data to the cloud server.

[0050] F3, Multi-Agent Deep Q-Networks (MADQN). Each sensing device is set as an agent, and the DQN (Deep Q-Networks) algorithm is deployed to autonomously make channel selection and model partitioning decisions. However, this method fails to consider the change in queue state information caused by high-priority task queuing.

[0051] As Figure 4 shown, to verify the performance of the present invention under different DNN models, Vgg11, MobileNetV2, and ResNet18 are used as DNN models for experiments respectively, and Proposed represents the method proposed in the present invention. Among them, the number of sensing devices N is set to 15, the number of edge servers M is set to 3, and the total number of sub-channels of each ES is set to 3. The simulation results show that the method proposed in the present invention can exhibit the lowest delay, far exceeding the slowly converging MADQN. Compared with Edge only, the average delay is reduced by 28%; compared with the centralized PPO, it shows a lower digital twin synchronization delay; thus proving the effectiveness of the present invention in improving the digital twin synchronization efficiency under mainstream models.

[0052] To verify the performance of the present invention under limited channel resources, Vgg11 and ResNet18 are used as DNN models for experiments respectively, as shown in (a) and (b) of Figure 5 . Among them, the number of sensing devices is set to 12, 15, 18, 21, and 24 in sequence, the number of edge servers M is set to 3, and the total number of sub-channels of each edge server is set to 3. The simulation results show that compared with the other three benchmark methods, the method proposed in the present invention can still maintain good performance under the condition of tight channel resources, and with the increase in the number of sensing devices, the advantages of the method proposed in the present invention become more obvious.

[0053] To verify the advantages of the present invention in channel allocation optimization, Vgg11 and MobileNetV2 are respectively used as DNN models for experiments, as shown in Figure 6 Figures (a) and (b) in. Among them, the number of sensing devices N is set to 30 in sequence, the number of edge servers M is set to 3, and the total number of sub-channels of each edge server is set to 4, 5, 6, 7, and 8 in sequence. The simulation results show that as the number of channels increases, the interference between devices decreases, and the delay of all methods shows a decrease, but the method proposed in the present invention shows stronger adaptability.

[0054] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. Any equivalent modification or change made by those of ordinary skill in the art according to the disclosed content of the present invention shall be included in the protection scope recorded in the claims.

Claims

1. A queue-aware cloud-edge collaborative reasoning method for digital twin synchronization, characterized in that: The steps include: S1, establish a reasoning task model based on the processing requirements of the data collected by the sensor equipment, and assign the reasoning task priority, delay tolerance and computational complexity; S2, according to the wireless communication process between the sensor device and the edge server, and the edge server and the cloud server, a wireless channel model is established, and the channel is divided into independent sub-channels; S3, according to the processing process of each task in the edge server queue buffer, a task queue model is established, and tasks of different priorities are optimized according to the M / M / 1 non-preemptive priority model; S4, establish a collaborative reasoning model based on the collaborative reasoning process of the reasoning task on the edge server and the cloud server, and analyze the collaborative reasoning delay based on the processing flow of the reasoning task; S5, builds the optimization objective function according to the low latency requirement of digital twin synchronization, and transforms the optimization problem into a partially observable Markov decision problem through time slot division to build a Markov model; S6, use the MAPPO algorithm to establish the sensor device agent and the edge server agent respectively, train the two agents separately based on the Markov model and update the current strategy.

2. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 1 is characterized in that: The specific steps to build a reasoning task model include: S11, the process of continuously collecting physical object state data by the sensor device and handing it over to the DNN model for reasoning is considered to be a Poisson process, where the task arrival rate is set to The total number of tasks generated within time T is ; S12, define the sensor device set as , where N represents the total number of sensor devices; any sensor device SD n The kth reasoning task generated for: ， in, Indicates the data size; Indicates the synchronization priority of the digital twin. The larger the value, the higher the task priority. Indicates that the task tolerates delays. If the delay is exceeded, the task is considered failed.

3. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 1 is characterized in that: The specific steps to establish a wireless channel model include: S21, define the edge server set as , where M represents the total number of edge servers; any edge server ES e The communication range of each sensor device covers multiple sensor devices, and each sensor device communicates with only one edge server. ; Define a set Indicates SD within the communication range n serial number, Indicates ES e SD within communication range n Total quantity, of which and , represents the empty set; S22, SD n With ES e By subchannel collection To communicate, Indicates ES e With SD n The total number of sub-channels for communication, each communication selects A subchannel in express: Indicates SD n Use subchannel c to transmit the sensed data to the associated ES e To achieve digital twin synchronization; Indicates that the subchannel is not used; according to Shannon's formula, SD n to ES e Uplink wireless link transmission rate It is expressed as: ， in, represents the bandwidth of subchannel c, represents the power of the additive white Gaussian noise of subchannel c, Indicates SD n The transmission power, represents the uplink channel gain; Therefore, SD n The reasoning task is divided into subchannels c Transfer to ES e The transmission delay It is expressed as: , in, for The initial data volume; S23, define ES e The set of subchannels communicating with the cloud server is , where ES e Using Subchannels Communicate with the cloud server without mutual interference; therefore, ES e Uplink wireless link transmission rate to the cloud server It is expressed as: ， in, Indicates ES e Corresponding subchannel bandwidth, Indicates ES e Corresponding subchannel The power of additive white Gaussian noise, Indicates ES e The transmission power, Indicates the uplink channel gain.

4. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 3 is characterized in that: The specific steps to establish a task queue model include: S31, define ES e The task queue is ,in Indicates SD n The kth reasoning task In ES e The position in the task queue, Indicates ES e service rate; and ES e Following the M / M / 1 non-preemptive priority channel model, high-priority tasks will interrupt low-priority tasks, but ongoing reasoning tasks will not be interrupted; S32, when low priority reasoning task While waiting at position p in the queue, if q higher priority tasks enter the queue, In ES e The waiting delay x of the task queue is measured by the distribution function: ， in, represents the distribution function of the waiting delay x; e represents a natural constant, q is the number of high-priority tasks entering the queue in the time slot; S33, define the probability of digital twin synchronization failure as , during the waiting process, the system at least The probability of maintaining synchronization continuity is, then the minimum value of the queuing waiting delay is .

5. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 3 is characterized in that: The specific steps to establish a collaborative reasoning model include: S41, reasoning task DNN segmentation point Use split point identifier Represents, where L indicates the total number of feasible segmentation points of DNN, Indicates reasoning task The split point is , Representing reasoning tasks The split point is not ; If the split point is Indicates reasoning task All are processed in the cloud server, and the split point is Indicates reasoning task All in ES e processing; in other cases, the first half of the DNN is processed in ES e Processing, then transferring the intermediate data to the cloud server to complete the second half of the reasoning; S42, reasoning task In ES e After completing part of the inference, the transmission delay of the intermediate data to the cloud server It is expressed as: ， in, The split point is When the reasoning task The amount of intermediate data transmitted to the cloud server; For ES e The transmission rate between the server and the cloud; S43, based on the definition of the M / M / 1 queueing system, reasoning task In ES e Delayed stay It is expressed as: ， in, For ES e The service rate of the task queue, Characterization of ES e The computing power of For SD p The task arrival rate; ， ， in, Indicates ES e The computational complexity of all reasoning tasks at Representing reasoning tasks Computational complexity, Indicates the computational complexity of the corresponding segmentation point; For SD p The total number of inference tasks that have been transferred; S44, reasoning task The total delay is: ， in, Indicates SD n To ES e Transmission delay.

6. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 5 is characterized in that: The specific steps to construct a Markov model include: S51, through reasonable channel allocation and determination of the reasoning task splitting point, the optimization goal is to minimize the maximum cumulative delay of digital twin synchronization, then the optimization problem P1 is expressed as: ， Among them, constraint C1 ensures that each sensor device uses only one channel, constraint C2 ensures that each reasoning task has only one split point, and constraint C3 ensures that the reasoning task is completed within the delay tolerance limit; For SD n The total number of inference tasks generated; S52, define the time slot set of the digital twin synchronization process , where T is the total length of the time slot, and the interval between each time slot is a fixed time ; Through time slot division, the optimization problem P1 is transformed into the optimization problem P2 that conforms to the Markov model: ， in, Indicates the number of tasks completed in a time slot.

7. The queue-aware cloud-edge collaborative reasoning method for digital twin synchronization according to claim 6 is characterized in that: The sensor device agent model is as follows: Defining SD n The MDP state space corresponding to the agent , Action Space With the reward function ; where the state space , Indicates SD n With ES e The channel gain between represents the number of sensing devices in the previous time slot channel c; Action Space , For SD n A communication identifier transmitted using subchannel c; Reward Function , Indicates SD in a single time slot n The number of tasks completed by the transfer, Indicates SD in a single time slot n The number of tasks that failed to synchronize, YesSD n Penalties for failed digital twin synchronization; The edge server agent model is as follows: Defining ES e The MDP state space corresponding to the agent , Action Space With the reward function ; where the state space , Indicates ES in the time slot e Task queue status, Indicates SD in the time slot n Task status, Indicates ES in the time slot e Channel gain with the cloud server; Action Space , express Split point identifier; Reward Function , Indicates ES in a single time slot e The number of tasks completed by the transfer, Indicates ES in a single time slot e The number of tasks that failed to synchronize, YesES e Penalties for failed digital twin synchronization; The specific steps for training an agent include: S621, setting hyper parameters and initializing the agent model; S622, at the beginning of each round, initialize environmental parameters; including: initializing the time slot to 0, initializing the task statistics to 0, clearing the experience pool, and resetting the communication distance between the sensor device and the edge server to a random number; S623, obtain current time slot SD n Environmental status ;SD n in accordance with Make decisions and determine the current action , and set the current time slot to SD n With ES e communication channels; S624, obtain the current time slot ES e Environmental status ;ES e in accordance with Make decisions with the current strategy to determine the current action , and set the current time slot to SD n The reasoning task split point; S625, update the environment state based on the time slot interval T0, realize the generation and processing of reasoning tasks, and count the number of tasks completed in the time slot Number of failed tasks ; S626, Smart Body SD n According to the reward function , calculate the reward value of the current time slot ; Agent ES e According to the reward function Calculate the reward value for the current time slot ; S627, store the state, action, and reward of the current time slot into the experience pool. When the sample size in the experience pool reaches the preset number, use the saved samples to train the two agents respectively and update the current strategy.

Images