Collaborative unloading method for interactive users in cloud-side-end network

Abstract

The invention provides a collaborative unloading method for interactive users in a cloud-edge-end network, and solves the unloading problem of interactive user tasks under multiple constraints. Aiming at computing-intensive and delay-sensitive task processing requirements caused by sharp increase of Internet of Things equipment and the problem of unloading complexity improvement caused by multi-user sharing of computing results in an interactive user scene, a three-layer collaborative unloading architecture of a cloud end, an edge server end and a user equipment end is constructed; in combination with service cache, communication resources and computing resource constraints, the delay and cost optimization problem of task unloading is modeled as a Markov decision process. According to the method, a deep reinforcement learning algorithm based on a Soft Actor-Critic (SAC) framework is designed, a discrete-continuous mixed action space is processed through a double-output-head mechanism, and the dynamic environment adaptability is improved by integrating transfer learning. According to the method, efficient unloading scheduling of interactive user tasks can be realized, the processing delay and cost are reduced, and the convergence speed and robustness of the system in a dynamic scene are improved.

Description

Technical Field

[0001] This invention relates to the fields of edge computing, intelligent scheduling, and deep learning technologies, specifically to a collaborative offloading method for interactive users in a cloud-edge-device network, applicable to computationally intensive and latency-sensitive application scenarios such as VR / AR interaction and streaming media transmission that require multiple users to share computation results. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) industry, a large number of IoT devices have been widely deployed, generating a massive number of computationally intensive and latency-sensitive tasks. In traditional cloud computing models, cloud servers are located far from mobile terminals, resulting in high transmission latency, making it difficult to meet the processing requirements of these tasks. The cloud-edge-device collaborative architecture, by deploying edge servers closer to terminal devices, combines the massive computing resources of the cloud with the low latency advantages of the edge, providing a more efficient solution for task processing.

[0003] However, in real-world scenarios such as VR / AR online interactive games and multi-user streaming collaboration, the results of computational tasks need to be transmitted to multiple interactive users. The presence of these users necessitates additional consideration of the spatial distribution of multiple user nodes during task unloading, significantly increasing the complexity of unloading decisions. Furthermore, edge servers have limited storage capacity and cannot cache all types of service data. Tasks that do not cache target service data must be migrated to other edge servers or the cloud for processing, further complicating the unloading decision-making process.

[0004] In existing technologies, research on cloud-edge-device collaborative offloading is mostly focused on single-user scenarios, without fully considering the multi-result distribution needs of interactive users. Although some studies involve task migration, they are mainly aimed at specific scenarios such as vehicle edge computing, and have not solved the problem of interactive user task offloading under the joint constraints of caching, communication and computing resources. In addition, traditional optimization algorithms and ordinary reinforcement learning algorithms are difficult to handle discrete-continuous mixed action spaces and have poor adaptability in dynamic environments (such as cache updates), resulting in poor performance in terms of latency and cost optimization of offloading schemes and slow convergence speed.

[0005] Therefore, there is an urgent need for a collaborative unloading method and system that can adapt to interactive user scenarios, jointly consider multiple resource constraints, and combine optimized performance with dynamic adaptability.

[0006] Currently, existing cloud-edge-device collaborative offloading technologies face three core pain points in interactive user scenarios:

[0007] (1) High complexity of multi-user result distribution: Traditional solutions do not consider the need for interactive users to share computation results, resulting in chaotic task distribution and result feedback logic between multi-user nodes, and loss of control over latency and cost;

[0008] (2) Poor adaptability to multiple resource constraints: Under the joint constraints of service cache capacity, communication resources and computing resources, existing algorithms have difficulty balancing the multi-dimensional decision-making of cache matching efficiency, resource allocation ratio and server selection at the same time;

[0009] (3) Insufficient adaptability to dynamic environments: In scenarios such as edge server cache data updates and user terminal movement, traditional algorithms need to retrain the model, resulting in slow convergence speed and large performance fluctuations. Summary of the Invention

[0010] The purpose of this invention is to provide a collaborative offloading method and system for interactive users in a cloud-edge-device network, which solves the problems of high offloading complexity, poor adaptability to multiple resource constraints, and insufficient adaptability to dynamic environments in existing technologies under interactive user scenarios. It achieves joint optimization of task processing latency and cost, and improves the robustness and convergence speed of the system in complex scenarios.

[0011] There is currently one centralized cloud server and a group of edge servers. , and a set of IoT mobile devices , A three-tier architecture. In this cloud-edge-device computing network, each edge server... and each mobile device Each has its corresponding physical location. ,in and They represent edge servers respectively The horizontal and vertical coordinates of physical location; similarly, mobile devices... The physical location can be represented as This is used to subsequently determine whether the device is within the communication coverage area of ​​the edge server.

[0012] The network model, latency model, and price model are modeled as follows.

[0013] (a) Network Model. Mobile Devices It can only access edge servers within its communication coverage area, and this association needs to be determined by physical location quantification.

[0014] . (1)

[0016] Filter by physical location distance The directly accessible edge server provides a foundation for selecting subsequent offloading servers, ensuring the stability of the communication link between the device and the server and avoiding a surge in transmission latency due to exceeding the coverage area.

[0017] In a cloud-edge-device network, edge servers The system's caching resources are limited and cannot store all types of caching services (such as machine learning model parameters, application components, etc.). To ensure reasonable allocation of caching resources, it is necessary to quantify the constraint that the total scale of caching services should not exceed the server's maximum cache capacity using a formula:

[0018] (2)

[0020] in, For edge servers In the time slot For caching services The caching decision is set to 1 for cached and 0 for uncached. For caching services The scale of data For edge servers Maximum storage capacity This represents the total number of types of cached services in the system.

[0021] (II) Latency Model. When the system makes decisions in each time slot, the total latency comprises two main components: edge server network latency and cloud processing latency. Edge server network latency includes task access latency, migration latency, computation latency at the server, and result return latency. In contrast, cloud processing latency includes task upload latency, computation latency, and result return latency. Each type of latency will be analyzed in detail in the following sections.

[0022] (1) Task access delay. The task starts from... The latency of transmission to the edge server needs to be calculated by combining the communication rate, task size, and channel allocation. The specific formula is as follows.

[0023] (3)

[0025] in, yes Task splitting ratio yes In the time slot The size of the generated task, yes right The decision of choosing an unloading server. yes Total uplink bandwidth yes and The signal-to-noise ratio of communication between them , Indicates the transmission power. This indicates the channel noise power.

[0026] (2) Task migration delay. Migration delay mainly comes from the time spent migrating tasks from the offloading server to the service server. Assume... The communication link between them is stable, and we can express the migration delay as a function of the migration hop count, as shown in the following formula.

[0027] (4)

[0029] in, yes The decision to select a service server It is to uninstall the server and The shortest communication hops between the selected servers. This value is calculated by traversing the entire network using Dijkstra's algorithm. This indicates the communication rate per unit link.

[0030] If a portion of the task is selected to be uploaded to the cloud for processing, there is no migration time; instead, it is processed from... The processing time for uploading to the cloud is expressed as:

[0031] (5)

[0033] in, It is the transmission waiting time for each unit of task in the cloud.

[0034] (3) Server computation latency. Computation latency is mainly divided into two parts: latency on the server side and latency in the cloud. Due to... Due to limitations such as storage and memory space, we will The task processing is modeled as a FIFO (First-In, First-Out) computation queue model. When a task arrives at the service server, it is added to the computation queue, and subsequent tasks are processed sequentially after the previous task is completed. Therefore, the computation latency at the service server mainly consists of queue waiting time and computation time, which can be expressed as:

[0035] (6)

[0037] in, express The computational speed Indicates in time slot Each The length of the task queue on the server. In formula (6), the first term considers the computation latency on the server, while the second term represents the waiting latency in the task queue on the server.

[0038] Given that cloud infrastructure provides ample storage and parallel computing capabilities, the cloud generally does not experience task queue waiting latency. Therefore, computing latency in the cloud can be expressed as:

[0039] (7)

[0041] in It refers to the speed of cloud computing.

[0042] (4) Result return delay. After the server completes its calculations, the tasks exchanged between interacting users may also involve a considerable amount of data, resulting in a significant return delay. The calculated task size is expressed as... The initial task size is The return latency through a Mobile Edge Computing (MEC) network can be expressed as follows:

[0043] (8)

[0045] Among them, among them, It is by The server will make the final decision. It is the shortest path from the service server to the return server.

[0046] Similarly, the latency of a task returned via the cloud can be expressed as:

[0047] (9)

[0049] (5) Total Latency: Based on the above modeling and calculations, we can determine the computation offloading latency of the MEC network and cloud processing. The latency of these two paths can be defined as follows:

[0050] (10)

[0052] (11)

[0054] Since the user task is executed simultaneously through two channels, the entire process can only be considered complete when both channels have finished their computations and sent the results to the receiving user. Therefore, the total computation offloading latency of the user task should be expressed as the maximum latency between the two channels, which can be represented as:

[0055] (12)

[0057] (III) Price Model. In order to... When unloading computing tasks, it's necessary to consider not only latency but also the cost of processing the tasks. Here, the total price includes both cloud pricing and edge server pricing.

[0058] (1) Cloud pricing. The cost of processing tasks in the cloud depends primarily on the task size and the initial cost of renting a server, which can be expressed by the formula:

[0059] (13)

[0061] in, Price per unit of task processing in the cloud. Price for cloud-based startup.

[0062] (2) Edge server price. The price of an edge server includes not only the computational cost of the tasks executed on the service server, but also the related costs of hops to other servers, which can be expressed by the formula:

[0063] (14)

[0065] in, Price per task processed by an edge server The unit task migration price for edge servers. The starting price for edge servers, The task split ratio, This is the initial size of the task. This represents the shortest communication hops between the offloading server and the service server.

[0066] (3) Total Price. The formula for calculating the total cost of unloading a specific task is as follows.

[0067] (15)

[0069] The ultimate goal of this invention is to minimize the long-term total cost by effectively partitioning and offloading user tasks generated at each moment in the network system. Assuming that at time... shared within indivual The objective function can be defined as follows when the task is offloaded to the MEC network:

[0070] (16)

[0072]

[0073] (16-a)

[0074]

[0075] (16-b)

[0076]

[0077] (16-c)

[0078]

[0079] (16-d)

[0080]

[0081] (16-e)

[0082]

[0083] (16-f)

[0084] The objective function is a weighted sum of latency and price. Constraint (16-a) represents the communication coverage limit of the edge server, and constraint (16-b) represents the storage capacity of cached data on the edge server. Constraint (16-c) indicates that the sum of the ratios used to allocate user communication resources for offloading on the same server must be 1. Constraint (16-d) represents the decision of task selection between the local server, service server, and return server as discrete variables, and the parameters in constraints (16-e) and (16-f) are... and The task partitioning ratio and communication resource allocation ratio are treated as continuous variables and are considered as deviation coefficients. Adjusting these coefficients yields the optimal task offloading scheme under different weighted scenarios. Since the units and value ranges of latency and price are inconsistent, we introduce a normalization factor. and Latency and cost are standardized separately. These reference values ​​are the maximum observed values ​​determined through extensive experiments, ensuring that the normalized results fall within the range [0,1]. This invention demonstrates that by employing the above strategy, comprehensive communication coverage for interactive users on mobile edge networks can be guaranteed. Simultaneously, task migration and result delivery are achieved through connections between edge servers. Furthermore, the complementarity of cloud and edge resources can be fully utilized. In practical applications, cloud-based model training and deployment can be employed, reducing the offloading complexity of interactive user tasks, optimizing system latency and cost, and enhancing system adaptability and robustness, thus meeting the required requirements.

[0085] Compared with the prior art, the beneficial effects of this invention are:

[0086] (1) A three-layer collaborative unloading architecture was designed for interactive user scenarios. For the first time, the multi-user result distribution requirements, service caching constraints and resource allocation were jointly optimized, which significantly reduced the unloading complexity of interactive user tasks.

[0087] (2) The proposed dual-output head mechanism effectively solves the decoupling decision problem in the discrete-continuous mixed action space. Combined with the maximum entropy characteristic of the SAC algorithm, it improves the strategy exploration capability and optimization accuracy. Compared with traditional algorithms, this invention can achieve a lower convergence cost.

[0088] (3) The FTDHSAC algorithm, which incorporates a transfer learning mechanism, does not need to retrain the model when the dynamic environment changes, such as when the cached data is updated. It achieves rapid adaptation through policy transfer, improves the initial reward, and significantly speeds up the convergence.

[0089] (4) The system can adapt to changes in parameters such as task size, communication rate, and computing resource ratio. In scenarios such as increased task quantity and limited cache space, the latency and cost increase are significantly lower than those of traditional algorithms, and the system is more adaptable and robust. Attached Figure Description

[0090] Figure 1 This is a schematic diagram of the cloud-edge-device three-layer collaborative offloading architecture of the present invention.

[0091] Figure 2 This is a schematic diagram of the dual-output head mechanism framework of the strategy network of the present invention.

[0092] Figure 3 This is a diagram of the fast transmission architecture based on the dual-head SAC algorithm of the present invention. Detailed Implementation

[0093] The collaborative offloading method for interactive users in a cloud-edge-device network includes the following steps:

[0094] (1) Construct a cloud-edge-device three-layer collaborative offloading architecture. The architecture includes a centralized cloud, an edge server consisting of multiple edge servers, and a user device consisting of multiple IoT mobile terminals. The edge servers are connected through a high-speed communication link to form an undirected circular communication network. Each edge server is configured with a service cache module with limited capacity to meet the constraint that the storage capacity of cached data on the edge server does not exceed the maximum cache capacity of the server.

[0095] (2) Define the system state space, action space, and reward function, and model the task unloading optimization problem of interactive users as a Markov decision process. The optimization objective is to minimize the sum of the weighted delay and price cost of task processing. The system state space can be defined as:

[0096] (17)

[0098] in, The cache service data status for all edge servers. This is a set of uninstallation servers that can be selected for each mobile terminal. This is the matrix of shortest distances between edge servers. This refers to the task queue length for each edge server. The action space... Defined as:

[0099] (18)

[0101] in, To unload the server index, For service server indexing, To return the server index, The task split ratio, The allocation ratio for wireless communication resources;

[0102] The weighted delay is defined as shown in formula (12), and the price cost formula is shown as shown in formula (15).

[0103] (3) Based on the Soft Actor-Critic (SAC) framework, a dual-output head SAC algorithm (DHSAC) is designed. The policy network of the algorithm adopts a dual-output head mechanism, which uses the softmax activation function to process discrete actions and the sigmoid activation function to process continuous actions, thereby realizing the decoupled decision-making of the discrete-continuous mixed action space.

[0104] (4) Integrating transfer learning mechanism to construct a fast transfer dual-output head SAC algorithm (FTDHSAC). When the cached data of the edge server is updated, the historical best strategy and the new training strategy are fused through a time-varying strategy transfer function to achieve rapid model adaptation. The time-varying strategy transfer function is as follows:

[0105] (19)

[0107] in, This is the best historical strategy before the cache update. To cache the updated training strategy, The initial value is a time-dependent function that monotonically increases with the number of training iterations. When the new strategy converges .

[0108] (5) Generate the optimal unloading decision based on the FTDHSAC algorithm, including the task splitting ratio of the terminal device, the unloading allocation ratio of the edge server and the cloud, the communication resource allocation scheme, and the index selection of the unloading server, service server and return server, to complete the collaborative unloading of interactive user tasks.

[0109] Example:

[0110] Example 1: Specific Implementation of the Collaborative Unloading Method

[0111] In this embodiment, the parameter configuration of the cloud-edge-device three-layer collaborative architecture is as follows:

[0112] Edge server layer: Nine edge servers are distributed in a diamond shape within a 10×90 meter area. Each edge server has a circular communication coverage area, a storage capacity of 8000 bits, and caches 13 types of service data.

[0113] Terminal layer: 50 mobile terminals are randomly distributed within the coverage area of ​​the edge server. Each time slot generates 250 interactive user tasks, each 5000-8000 bits in size. Each task includes a departure / return function. Indexes, caching requirements, data size, and target user indexes;

[0114] Cloud computing: 20GHz computing speed, unit task processing price is Bit, the starting price is a fixed value;

[0115] Communication parameters: The wireless communication bandwidth of the edge server is 10Mbps, the cloud communication rate is 4Mbps, the communication between edge servers adopts OFDMA technology, and the signal-to-noise ratio (SNR) is calculated based on the transmission power and channel noise.

[0116] The specific steps of the collaborative uninstallation method in this embodiment are as follows:

[0117] (1) Architecture construction: Nine edge servers are deployed and connected through high-speed communication links to form an undirected loop network. Each edge server is configured with a service caching module and 13 preset service data caching schemes. Fifty mobile terminals complete network access and determine the set of optional offload servers for each terminal. .

[0118] (2) Problem modeling: Construct the state space as follows ,in The cache state matrix for 9 edge servers. The shortest hop count matrix between edge servers is calculated using Dijkstra's algorithm. The task queue length for each edge server; the action space is constructed as follows. ,in , , satisfy The objective function is ,in , , .

[0119] (3) Algorithm training: First, initialize the value network, policy network, and target network of the SAC algorithm. Both the value network and policy network use 3 fully connected layers with 256 hidden neurons. Next, a dual-output head mechanism is adopted for the policy network. The discrete output head outputs the server index probability through the softmax activation function, and the continuous output head outputs the split ratio and resource allocation ratio through the sigmoid activation function. Then, the experience replay buffer capacity is set to The training batch size is 256, and the discount factor is... Soft update rate Finally, when the edge server's cached data is updated, the transfer learning mechanism is triggered, and the time-varying function is activated. Using a linear growth model, after 20,000 iterations .

[0120] (4) Unloading execution: First, after the mobile terminal generates the task, the algorithm outputs the optimal unloading decision, such as the task splitting ratio. (70% edge uninstallation, 30% cloud uninstallation), uninstallation server index Service server index (Cache matching), communication resource allocation ratio Next, the edge offloading part is transmitted to edge server 3 via the M2E link, directly calling the cache service to complete the calculation, and the result is transmitted to all target users through the return server 3; subsequently, the cloud offloading part is uploaded to the cloud to complete the calculation, and the result is returned to the target users; after the time slot ends, the task queue length is updated. The system collects latency and cost data to calculate rewards and update network parameters. When cached data is updated, it integrates historical and new strategies through transfer learning to achieve seamless adaptation.

[0121] Example 2: Specific Implementation of the Collaborative Unloading System

[0122] The collaborative unloading system in this embodiment includes an architecture deployment module, a problem modeling module, an algorithm training module, a decision execution module, and a state update module. The architecture deployment module uses a cloud computing platform as the cloud, NVIDIA Jetson Xavier NX as the edge server, and smartphones as the mobile terminal. A 5G communication module connects the edge server and the terminal, and the edge servers are connected via Gigabit Ethernet. The problem modeling module uses a Python-based data acquisition program to collect data such as cache status, queue length, and communication rate from the edge server in real time, constructing a state space matrix. The algorithm training module implements the FTDHSAC algorithm based on the PyTorch framework. The training process is performed on a GPU server (NVIDIA A100), and the trained model is deployed to the edge server. In the decision execution module, the edge server runs a decision-making program. After receiving a task request from the mobile terminal, it calls the model to generate an unloading decision and issues an execution command. The state update module can monitor cache data update events and environmental parameter changes in real time, triggering a transfer learning mechanism to update model parameters.

[0123] Performance Verification: This embodiment verifies the superiority of the present invention by comparing it with the Random Actor (RA), Monte Carlo (MC), Genetic Algorithm (GA), TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm, and SAC algorithm.

[0124] Convergence Performance: Compared to deep reinforcement learning algorithms based on SAC and TD3, the dual-output mechanism in the policy network significantly improves both training speed and final performance. The SAC algorithm benefits from entropy regularization, which facilitates action exploration and is particularly suitable for tasks requiring extensive exploration and high-dimensional action spaces. Both SAC and DHSAC exhibit faster convergence speeds than TD3.

[0125] Task Scale Adaptability: As the number of tasks increases, the evaluation reward of all algorithms decreases. However, compared to other algorithms, the increase in total cost for DHTD3 and DHSAC is less significant, and the performance gap between DHTD3 and DHSAC and other algorithms widens with the number of tasks. This indicates that deep reinforcement learning algorithms are better suited to handling large-scale spatial optimization problems compared to traditional optimization algorithms.

[0126] Dynamic environment adaptability: When the cloud-edge computing rate ratio increases, the cost of RA, MC and GA increases significantly with the increase of the ratio, while the cost growth of DHTD3 and DHSAC remains relatively moderate. The DRL algorithm is more adaptable to environmental changes than other metaheuristic algorithms.

[0127] Transfer performance: When initialized with a combination of suboptimal strategies, the FTDHSAC algorithm achieves higher initial total reward and converges faster.

[0128] The above specific embodiments are used to explain the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A collaborative offloading method for interactive users in a cloud-edge-device network, characterized in that... Comprising the following steps: (I) Constructing a cloud-edge-end three-layer collaborative offloading architecture, the architecture comprising a centralized cloud end, an edge server end composed of a plurality of edge servers (ES), and a user device end composed of a plurality of Internet of Things mobile terminals (MD), the edge servers being connected through high-speed communication links to form an undirected cyclic communication network, and each edge server being configured with a service cache module with limited capacity; (II) Defining a system state space, an action space, and a reward function, modeling the task offloading optimization problem of interactive users as a Markov decision process (MDP), and the optimization objective being to minimize the sum of the weighted delay and the price cost of task processing; (III) Designing a double-output-head SAC algorithm (DHSAC) based on a Soft Actor-Critic (SAC) framework, the policy network of the algorithm adopting a double-output-head mechanism, respectively processing discrete actions through a softmax activation function and processing continuous actions through a sigmoid activation function, to realize decoupled decision-making in a discrete-continuous hybrid action space; (IV) Integrating a transfer learning mechanism to construct a fast transfer double-output-head SAC algorithm (FTDHSAC), when the cache data of the edge server is updated, fusing the historical optimal policy and the newly trained policy through a time-varying policy transfer function to realize rapid adaptation of the model; (V) Generating an optimal offloading decision based on the FTDHSAC algorithm, including the task splitting ratio of the terminal device, the offloading allocation ratio of the edge server and the cloud end, the communication resource allocation scheme, and the index selection of the offloading server, the service server, and the return server, to complete the collaborative offloading of the interactive user tasks. 2.The method of claim 1, wherein, In step (I), the constraint of the service cache module is satisfied: (1) in, For edge servers In the time slot For caching services caching decisions, For caching services The scale of data For edge servers Maximum storage capacity This represents the total number of types of cached services in the system. 3.The method of claim 1, wherein, In step (II), the system state space is defined as: (2) wherein, a buffer service data state for all edge servers, a set of offload servers selectable for each mobile terminal, a shortest distance matrix between edge servers, a task queue length for each edge server. The action space is defined as: (3) wherein, is an offload server index, is a service server index, is a return server index, is a task split ratio, is a wireless communication resource allocation ratio.

4. The method of claim 1, wherein the method further comprises: In step (III), the double-output-head mechanism is specifically divided into a discrete action head and a continuous action head: the discrete action head is for the index selection actions of the offloading server, the service server, and the return server, generates the probability distribution of each action through a softmax activation function, and adjusts the discrete decision result based on the probability distribution and the cache constraint and the communication coverage constraint; the continuous action head is for the task splitting ratio and the communication resource allocation ratio, normalizes the output to the [0, 1] interval through a sigmoid activation function, and adjusts the continuous decision result in combination with the resource allocation constraint.

5. The method of claim 1, wherein the method further comprises: In step (IV), the time-varying policy transfer function is: (4) wherein, is the history optimal policy before the cache update, is the new trained policy after the cache update, is a time-dependent function monotonically increasing with the number of training iterations, with initial value when the new policy converges, .

6. The method of claim 1, wherein the method further comprises: In step (II), the total task delay includes an edge network processing delay and a cloud end processing delay, and the maximum of the two is taken as the final delay: (5) (6) (7) wherein, is a migration delay, is an edge server computation delay, is a result return delay, is a task access delay, is a cloud upload delay, is a cloud computation delay, is a cloud result return delay.

7. The method of claim 1, wherein the method further comprises: In step (II), the price cost is the weighted sum of the edge server processing cost and the cloud end processing cost, wherein: (8) (9) wherein, is a cloud per unit task processing price, is a cloud start-up price, is an edge server per unit task processing price, is an edge server per unit task migration price, is an edge server start-up price, is a task split ratio, is a task initial size, is a shortest communication hop count between the offload server and the service server.

8. The method of claim 1, wherein the method further comprises: In step (II), the optimization objective is: (10) (10-a) (10-b) (10-c) (10-d) (10-e) (10-f) where the objective function is the weighted sum of latency and price, constraint (10-a) represents the communication coverage limit of the edge server; constraint (10-b) represents the storage capacity of the cached data on the edge server; constraint (10-c) indicates that the sum of the ratios for allocating user communication resources on the same server for offloading must be 1; constraint (10-d) represents the decision of task selection of the local server, service server and return server as a discrete variable, and the parameters in constraint (10-e) and constraint (10-f) and represent the division ratio of the task and the communication resource allocation ratio as a continuous variable, is the deviation coefficient, and adjusting these coefficients can obtain the optimal task offloading scheme under different weighting scenarios, since the units and value ranges of latency and price are inconsistent, we introduce normalization factors and respectively for the standardization of latency and price cost, and these reference values are the maximum observed values determined through a large number of experiments, which ensure that the normalization result falls within the range [0, 1]. 9.A method for collaborative offloading of interactive users in a cloud-edge-end network, the method comprising: Comprising: The architecture deployment module is used for constructing a cloud-edge-end three-layer collaborative architecture, deploying centralized cloud, edge server cluster and mobile terminal device, and configuring communication link and service cache module of the edge server; the problem modeling module is used for collecting cache state, communication resource state, computing resource state and task characteristic data of the system, defining state space, action space and reward function, and modeling the task offloading optimization problem as a Markov decision process; the algorithm training module is used for training a double output head algorithm based on a SAC framework, constructing a fast migration algorithm by incorporating a transfer learning mechanism, and storing interaction data in an experience replay buffer and updating network parameters; the decision execution module is used for loading the trained algorithm model, generating an optimal offloading decision and issuing it to the edge server and mobile terminal, and realizing task splitting, resource allocation and server selection; and the state updating module is used for monitoring system state changes in real time, triggering a transfer learning mechanism to update the offloading decision when cache data is updated or environmental parameters mutate.

10. The method of claim 9, wherein the method further comprises: Each edge server in the edge server cluster is configured with a task queue management module, adopts a first-in-first-out (FIFO) mechanism to process received offloading tasks, and the task queue management module feeds back queue length to the problem modeling module in real time; and the mobile terminal device supports task slicing function, splits a task into an edge offloading part and a cloud offloading part according to a splitting ratio issued by the decision execution module, and transmits the two parts to corresponding processing nodes through a wireless communication link and a cloud communication link respectively.

Images