An iot data offloading scheduling method for service cache edge cloud computing network

Abstract

The application discloses an Internet of Things data offloading scheduling method for a service cache edge cloud computing network and belongs to the technical field of edge computing. The method is applied to a service cache edge cloud computing network composed of one user equipment, one base station and one cloud server and comprises the following steps: S1, training the service cache edge cloud computing network according to the Internet of Things data collected by the user equipment until all training rounds are traversed; and S2, obtaining the total decision of a required time slot by using the trained service cache edge cloud computing network to complete the offloading scheduling of the Internet of Things data in the corresponding time slot. The method can jointly optimize the subchannel allocation decision, the task offloading decision, the task selection decision and the task scheduling sequence, realizes the minimization of the long-term cumulative average information age of the service cache edge cloud computing network and realizes low-delay service.

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology, specifically relating to an IoT data offloading and scheduling method for service caching edge cloud computing networks. Background Technology

[0002] Mobile edge computing (MEC) has emerged as a new paradigm for network computing, deploying abundant computing resources on MEC servers at the network edge to provide computing services to user devices. However, providing low-latency services for emerging applications using traditional MEC remains challenging. First, simultaneous task offloading by multiple user devices can easily lead to spectrum resource contention. Second, due to the limited computing resources of MEC servers, it is difficult to guarantee that the real-time requirements of tasks can be met when faced with a large number of computing tasks. Finally, many emerging applications are data-driven and require caching services such as libraries, code, and machine learning models on MEC servers. Due to the limited storage space of MEC servers, a single server can only cache a portion of these services.

[0003] Edge-cloud collaborative computing combines edge computing and cloud computing, effectively addressing the aforementioned challenges. First, cloud servers provide richer computing resources, alleviating the computing resource pressure on MEC servers. Second, the ample cache space of cloud servers can cache all services; therefore, edge-cloud collaborative computing can handle situations where MEC servers cannot process specific computing tasks.

[0004] In emerging applications within the Internet of Things (IoT) landscape, the real-time performance of computing tasks is paramount. Compared to traditional metrics, information age can better reflect the real-time completion status of tasks. However, current service caching edge cloud computing networks still have significant shortcomings in optimizing resource allocation and task offloading scheduling decisions to minimize information age while meeting user device energy consumption constraints, making it difficult to achieve low-latency services. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by proposing an IoT data offloading and scheduling method for a service cache edge cloud computing network. This method jointly optimizes sub-channel allocation decisions, task offloading decisions, task selection decisions, and task scheduling order to minimize the long-term cumulative average information age of the service cache edge cloud computing network and achieve low-latency service.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention proposes a method for scheduling IoT data offloading in a service cache edge cloud computing network, applicable to... Individual user devices A service caching edge cloud computing network consisting of one base station and one cloud server includes the following steps:

[0008] S1. Train the service cache edge cloud computing network based on the IoT data collected by the user equipment until all training rounds have been traversed. In each time slot of each training round, the service cache edge cloud computing network performs the following operations:

[0009] S11. Each user device generates a task based on the collected IoT data and stores it in its own cache. Each base station collects the status information of the corresponding user device based on its own service cache. The cloud server aggregates the status information of all user devices and the status information of the base stations to form the overall status information.

[0010] S12. Based on the overall state information, use the deep reinforcement learning model deployed on the cloud server to output sub-channel allocation decisions, task offloading decisions, and task selection decisions;

[0011] S13. Based on the sub-channel allocation decision, task unloading decision, and task selection decision, the cloud server uses a queue-jump-based scheduling order algorithm to obtain the task scheduling order.

[0012] S14. The cloud server issues a general decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data. The general decision includes sub-channel allocation decision, task offloading decision, task selection decision and task scheduling order.

[0013] S2. Utilize the trained service cache edge cloud computing network to obtain the overall decision for the required time slot and complete the offloading and scheduling of IoT data for the corresponding time slot.

[0014] Preferably, during the training of the service cache edge cloud computing network based on the IoT data collected by the user equipment until all training rounds are traversed, the geographical coordinates of the user equipment and the service cache of the base station are initialized before the start of each training round. The initialization of the geographical coordinates of the user equipment is achieved by randomly generating the user equipment within the coverage area of ​​each base station through an independently distributed Poisson point process, and the initialization of the service cache of the base station is based on a Gaussian distribution.

[0015] Preferably, the status information of the user equipment includes the task of the user equipment and the information age of the user equipment, the status information of the base station includes the service cache of the base station, the user equipment is a sensor, and the types of tasks include 10 categories such as environmental temperature and humidity data, vibration data, pressure data, flow data, electrical parameter data, position and displacement data, gas concentration data, liquid level data, rotation speed data, and image data.

[0016] Preferably, the objective function of the deep reinforcement learning model is to optimize sub-channel allocation decisions, task offloading decisions, and task selection decisions to minimize the impact of all user equipment on the process. The average age of the cumulative information within each time slot;

[0017] The constraints of deep reinforcement learning models include:

[0018] 1) Service caching constraint: The unloading task is calculated by the cloud server or by the base station with the corresponding caching service;

[0019] 2) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time;

[0020] 3) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age;

[0021] 4) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

[0022] Preferably, the objective function of the queue-based scheduling ordering algorithm is to optimize the task scheduling order to minimize the impact of queue jumps on all user devices. The average age of the cumulative information within each time slot;

[0023] The constraints of the scheduling order algorithm based on queue jumping include:

[0024] 1) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time;

[0025] 2) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age;

[0026] 3) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

[0027] Preferably, the scheduling order algorithm based on queue jumping includes the following steps:

[0028] S131. Initialize time slot The task scheduling order is an empty list, and the user equipment is calculated. In the time slot Uninstallation preference value The formula is as follows:

[0029] ;

[0030] in, Indicates user equipment In the time slot Total delay of the unloading task Indicates user equipment In the time slot The generation time of the most recently completed task. Indicates user equipment In the time slot The generation time of the most recently completed task. , , This represents the total number of time slots in a training epoch;

[0031] S132, All user equipment in the time slot The unloading preference values ​​are sorted in ascending order to obtain the unloading order of tasks as time slots. Initial task scheduling strategy ;

[0032] S133, Judgment Does it satisfy the constraints of the queue-based scheduling ordering algorithm? If so, then... As a time slot The task scheduling order should be determined accordingly; otherwise, it should be adjusted according to the Johnson Law. Until the constraints of the queue-based scheduling ordering algorithm are met, the adjusted... As a time slot The task scheduling order.

[0033] Preferably, communication between user equipment and base stations, and between base stations themselves, is based on orthogonal frequency division multiple access (OFDM) protocol, with sub-channels divided and time-division multiplexing within each sub-channel. Base stations communicate directly with the cloud server via wired links. In the time slot Total delay of unloading tasks The details are as follows:

[0034] 1) When user equipment In the time slot When offloading tasks to associated base stations For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots Computing User Equipment The computational delays of the unloading tasks are summed;

[0035] 2) When user equipment In the time slot When offloading tasks to unrelated base stations For user equipment In the time slot The transmission delay of offloading the task to the associated base station, and the associated base station in the time slot. Transmission delay to unassociated base stations, and the time slot of unassociated base stations. Computing User Equipment The computational delays of the unloading tasks are summed;

[0036] 3) When user equipment In the time slot When offloading the task to the cloud server For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots The transmission delays to the cloud server are added together.

[0037] Preferably, after the cloud server issues the overall decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data, the following steps are also performed:

[0038] S15. The user device calculates its own information age. The cloud server calculates the average information age of the user devices based on the information age of all user devices as the reward for the deep reinforcement learning model, and generates experience for the deep reinforcement learning model. When the number of experience is greater than or equal to the preset number, the deep reinforcement learning model updates the model parameters based on the randomly selected experience, and the current time slot ends.

[0039] Preferably, the deep reinforcement learning model is used in time slots. The experience is represented by time slots Status, time slot Actions and time slots Rewards and time slots The quadruple composed of states, time slot The status includes all user equipment in the time slot Tasks, all user devices in time slots Information age, all base stations in time slots Service cache and all sub-channels in time slots Rayleigh fading factor, time slot Actions include time slots Sub-channel allocation decision, task offloading decision, and task selection decision, time slot The reward is for all user devices in the time slot The negative of the sum of information ages minus the time slot The penalty value, , This represents the total number of time slots in a training round.

[0040] Preferably, the deep reinforcement learning model is a SAC-D agent, which includes an Actor network, two first Critic networks, and two second Critic networks. Both first Critic networks are connected to the Actor network, and the second Critic networks are connected one-to-one with the first Critic networks. The objective function of the Actor network in the SAC-D agent is... The formula is as follows:

[0041] ;

[0042] in, Indicates obtaining Expectations The largest Actor network model parameters , This represents the decay discount factor of the reward. Indicates time slot The reward Indicates the temperature coefficient. Indicates time slot entropy, , This represents the total number of time slots in a training round.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] 1) A problem was constructed to minimize the long-term cumulative average information age of the service cache edge cloud computing network under constraints. The sub-channel allocation, task offloading, task selection and task scheduling order were dynamically adjusted according to the real-time tasks of user devices. While meeting the computing needs of user devices, it ensured that the IoT data collected by user devices was processed in a timely manner (low information age) and guaranteed the fairness constraints among different user devices.

[0045] 2) The complex non-convex integer nonlinear programming problem is decomposed into sub-channel allocation and task offloading sub-problems and task scheduling order sub-problems. The sub-channel allocation and task offloading sub-problems are optimized by a deep reinforcement learning model, and the task scheduling order sub-problems are optimized by a queue-jump-based scheduling order algorithm to accelerate the learning speed of the SAC-D agent and reduce the complexity of solving the long-term cumulative average information age minimization problem.

[0046] 3) The deep reinforcement learning model is an extension of the SAC-D agent for discrete action space. It transforms the model output from a continuous action distribution to a discrete action distribution. The Critic network outputs the weighted sum of the values ​​corresponding to each action to represent the expected value of the action. This enables the SAC-D agent to maximize the overall performance of the service cache edge cloud computing network in the long run. It avoids the problem that traditional static optimization methods (such as successive convex approximation) cannot effectively perform long-term optimization and solves the problem that traditional soft actor-critic models cannot effectively optimize discrete actions.

[0047] This invention is particularly applicable to IoT scenarios with dense user equipment distribution and high requirements for real-time performance and fairness (such as industrial IoT, smart transportation, smart logistics, smart security, etc.), such as service cache edge cloud computing network scenarios that combine the Orthogonal Frequency Division Multiple Access (OFDMA) protocol with time division transmission communication. Attached Figure Description

[0048] Figure 1 A flowchart of the task offloading and sequential scheduling method for the service cache edge cloud computing network of the present invention;

[0049] Figure 2 This is a schematic diagram of the service caching edge cloud computing network of the present invention;

[0050] Figure 3 This is a graph showing the variation of standardized cumulative reward with training rounds in Embodiment 1 of the present invention.

[0051] Figure 4 This is a graph showing the changes in task offloading ratio and long-term cumulative average information age as a function of base station computing power in Embodiment 1 of the present invention.

[0052] Figure 5 This is a graph showing the changes in task offloading ratio and long-term cumulative average information age with base station transmission power in Embodiment 1 of the present invention.

[0053] Figure 6 This is a graph showing the change in task offloading ratio as a function of base station service cache capacity in Embodiment 1 of the present invention.

[0054] Figure 7 This is a graph showing the changes in task offloading ratio and long-term cumulative average information age as a function of user device task generation probability in Embodiment 1 of the present invention.

[0055] Figure 8 This is a graph showing the long-term cumulative average information age as a function of the number of user devices in Embodiment 1 of the present invention.

[0056] Figure 9 This is a graph showing the long-term cumulative average information age as a function of user equipment transmission power in Embodiment 1 of the present invention.

[0057] Figure 10This is a graph showing the change in the long-term cumulative average information age as a function of user device capacity in Embodiment 1 of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0060] like Figures 1-10 As shown, an IoT data offloading and scheduling method for a service cache edge cloud computing network is applied to... Individual user devices A service caching edge cloud computing network consisting of one base station and one cloud server includes the following steps:

[0061] S1. Train the service cache edge cloud computing network based on the IoT data collected by the user equipment until all training rounds have been traversed. In each time slot of each training round, the service cache edge cloud computing network performs the following operations:

[0062] S11. Each user device generates a task based on the collected IoT data and stores it in its own cache. Each base station collects the status information of the corresponding user device based on its own service cache. The cloud server aggregates the status information of all user devices and the status information of the base stations to form the overall status information.

[0063] In one embodiment, the status information of the user equipment includes the user equipment's task and the user equipment's information age, the status information of the base station includes the base station's service cache, the user equipment is a sensor, and the types of tasks include 10 categories such as environmental temperature and humidity data, vibration data, pressure data, flow data, electrical parameter data, position and displacement data, gas concentration data, liquid level data, rotation speed data, and image data.

[0064] like Figure 2 As shown, the service caching edge cloud computing network includes Individual user devices A configuration of 8 user devices, 5 base stations, and 1 cloud server is provided for illustrative purposes only; the actual number will be adjusted based on specific needs. The initial locations of the user devices are randomly generated within the base station coverage area following an independently distributed Poisson point process, with no overlap between base station coverage areas. User devices generate tasks and store them in their own cache, which can then be offloaded to the base station or cloud server for computation. Base stations possess service caching and computing capabilities (e.g., the number of CPU cycles), allowing them to process tasks corresponding to cached services for user devices. Due to cache space limitations, a single base station can only cache a portion of services. Cloud servers have stronger service caching and computing capabilities, allowing them to cache all services. Wireless communication channels between user devices and base stations, and between base stations themselves, are divided based on the Orthogonal Frequency Division Multiple Access (OFDMA) protocol. Each sub-channel allows user equipment and base station to perform wireless transmission via time-division multiplexing within the assigned sub-channel.

[0065] At the start of each time slot, all user devices, based on the collected IoT data, preferentially generate tasks randomly following a Poisson point process, or can directly generate tasks and store them in their own caches. The total number of tasks generated by all user devices is [number missing]. Class, number Tasks of this type can only be computed by base stations that have corresponding task caching services. The base station collects the status information of user equipment and uploads it, along with its own status information, to the cloud server. The cloud server then aggregates the status information from both user equipment and the base station to obtain the overall status information. The service cache corresponds to the executed task, and the overall status information is the sum of the status information from all user equipment and the base station.

[0066] In one embodiment, during the training of the service cache edge cloud computing network based on IoT data collected by user equipment until all training rounds are traversed, the geographical coordinates of the user equipment and the service cache of the base station are initialized before the start of each training round. The initialization of the geographical coordinates of the user equipment is achieved by randomly generating the user equipment within the coverage area of ​​each base station through an independently distributed Poisson point process, and the initialization of the service cache of the base station is based on a Gaussian distribution.

[0067] Before training begins, the model parameters of the deep reinforcement learning model deployed on the cloud server must first be initialized. Specifically, this involves initializing the model parameters of the deep reinforcement learning model on the cloud server and initializing the total number of training epochs. Initialize the total number of time slots for each training round. Set the current training round. Secondly, the geographic location coordinates of the user's device need to be initialized: in the service cache edge cloud computing network, The initial location of each user equipment is randomly generated within the coverage area of ​​each base station using an independently distributed Poisson point process. The coverage areas of each base station do not overlap. A wired connection is preferred between the cloud server and the base station, but a wireless connection can also be used. Finally, initialization based on a Gaussian distribution is required. Service cache for each base station.

[0068] After traversing all time slots in a training round, the geographic location coordinates of the user equipment are initialized following an independently distributed Poisson point process, and the service cache state of the base station is initialized following a Gaussian distribution. The next training round then begins, and this process continues until all training rounds are completed. ~ After all training rounds are completed, the trained service caching edge cloud computing network can be directly executed according to steps S11 to S14 in each time slot.

[0069] S12. Based on the overall state information, use the deep reinforcement learning model deployed on the cloud server to output sub-channel allocation decisions, task unloading decisions, and task selection decisions.

[0070] In one embodiment, the objective function of the deep reinforcement learning model is to optimize subchannel allocation decisions, task offloading decisions, and task selection decisions to minimize the impact of all user equipment on [the following]. The average age of the cumulative information within each time slot;

[0071] The constraints of deep reinforcement learning models include:

[0072] 1) Service caching constraint: The unloading task is calculated by the cloud server or by the base station with the corresponding caching service;

[0073] 2) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time;

[0074] 3) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age;

[0075] 4) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

[0076] The cloud server uses a deep reinforcement learning model based on discrete action space (such as the SAC-D agent in this embodiment) to optimize the sub-channel allocation and task unloading sub-problems. That is, the deep reinforcement learning model outputs sub-channel allocation decisions, task unloading decisions and task selection decisions based on the total state information.

[0077] The objective function for the sub-channel allocation and task offloading sub-problem (i.e., the objective function of the deep reinforcement learning model) is as follows:

[0078]

[0079]

[0080] In the formula, This indicates that through optimization To minimize the corresponding The value inside, This represents the set of sub-channel allocation decisions. , Indicates time slot Sub-channel allocation decision, , This represents the total number of time slots in a training epoch. This represents the set of task offloading decisions for user devices. , Indicates time slot User device task offloading decisions This represents the set of task selection decisions made by the user device. , Indicates time slot The user device's task selection decision, and , , Indicates user equipment In the time slot Select the task corresponding to the index from its own cache and uninstall it. Indicates user equipment In the time slot Information age is used to indicate the time elapsed from the generation of information to its acquisition by the receiving end, thus reflecting timeliness. Indicates the length of the time slot. Indicates user equipment At any moment The generation time of the most recently completed task. Indicates time The derivative of the task. In base station computation constraints, the same base station can simultaneously receive data for one task and compute data for another task, and can choose to perform either task reception or task computation separately.

[0081] In one embodiment, the deep reinforcement learning model is a SAC-D agent. The SAC-D agent includes an Actor network, two first Critic networks, and two second Critic networks. Both first Critic networks are connected to the Actor network, and the second Critic networks are connected one-to-one with the first Critic networks. The objective function of the Actor network in the SAC-D agent is... The formula is as follows:

[0082]

[0083] in, Indicates obtaining Expectations The largest Actor network model parameters , This represents the decay discount factor of the reward. Indicates time slot The reward Indicates the temperature coefficient. Indicates time slot entropy, , This represents the total number of time slots in a training round.

[0084] The Soft Actor-Critic-Discrete (SAC-D) agent consists of one Actor network, two first Critic networks, and two second Critic networks (target Critic networks). Both first Critic networks are connected to the Actor network, and each second Critic network is connected to a first Critic network in a one-to-one correspondence. Specifically, the Actor network outputs an action based on the state of the current time slot, followed by steps S13, S14, and S15. In step S15, the two first Critic networks provide the action value for the current state based on experience from the current time slot. The Actor network updates its model parameters based on the action values ​​output by the two first Critic networks. Then, the two second Critic networks each provide the state value for the current state based on experience from the current time slot. The two first Critic networks update their network parameters based on the state values ​​output by their respective second Critic networks. Finally, based on the model parameters of the two first Critic networks, the two second Critic networks update their own model parameters. The SAC-D agent then updates its own temperature coefficient, completing one parameter update for the SAC-D agent.

[0085] The goal of the SAC-D agent is to find a policy that maximizes cumulative reward. The objective function of the Actor network in the SAC-D agent is... In the middle, temperature coefficient Used to determine time slots entropy The relative importance of the rewards can be continuously updated. , Indicates time slot Entropy, a measure of uncertainty for SAC-D agents, is used to encourage SAC-D agents to seek higher returns and maintain time slots. action The diversity.

[0086] SAC-D agents in time slots status The value of the action below The formula is as follows:

[0087]

[0088] in, Indicates in time slot status Next One action The value of the action, , The total number of actions. Indicates transpose;

[0089] SAC-D agents in time slots status The value of the state under the following conditions The formula is as follows:

[0090]

[0091] in, Indicates in time slot status The weight vector of the next action selection.

[0092] S13. Based on the sub-channel allocation decision, task unloading decision, and task selection decision, the cloud server uses a queue-jump-based scheduling order algorithm to obtain the task scheduling order.

[0093] In one embodiment, the objective function of the queue-jump-based scheduling ordering algorithm is to optimize the task scheduling order to minimize the impact of queue jumps on all user equipment scheduling. The average age of the cumulative information within each time slot;

[0094] The constraints of the scheduling order algorithm based on queue jumping include:

[0095] 1) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time;

[0096] 2) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age;

[0097] 3) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

[0098] The cloud server uses a queue-jump-based scheduling order algorithm to solve the task scheduling order subproblem based on the obtained sub-channel allocation decision, task unloading decision, and task selection decision, and obtains the task scheduling order (composed of transmission order and calculation order).

[0099] The objective function of the task scheduling order subproblem (the objective function of the scheduling order algorithm based on queue jumping) is as follows:

[0100]

[0101] In the formula, Indicates obtaining satisfaction Corresponding under the conditions The minimum value, A set representing the task scheduling order. , Indicates time slot The task scheduling order.

[0102] In one embodiment, the scheduling order algorithm based on queue jump includes the following steps:

[0103] S131. Initialize time slot The task scheduling order is an empty list, and the user equipment is calculated. In the time slot Uninstallation preference value The formula is as follows:

[0104]

[0105] in, Indicates user equipment In the time slot Total delay of the unloading task Indicates user equipment In the time slot The generation time of the most recently completed task. Indicates user equipment In the time slot The generation time of the most recently completed task. , , This represents the total number of time slots in a training epoch;

[0106] S132, All user equipment in the time slot The unloading preference values ​​are sorted in ascending order to obtain the unloading order of tasks as time slots. Initial task scheduling strategy ;

[0107] S133, Judgment Does it satisfy the constraints of the queue-based scheduling ordering algorithm? If so, then... As a time slot The task scheduling order should be determined accordingly; otherwise, it should be adjusted according to the Johnson Law. Until the constraints of the queue-based scheduling ordering algorithm are met, the adjusted... As a time slot The task scheduling order.

[0108] Among them, the task scheduling order subproblem is the long-term cumulative average information age (i.e., the total number of user equipments in the given sub-channel allocation decision, task offloading decision, and task selection decision) under the given sub-channel allocation decision, task offloading decision, and task selection decision. The problem is to minimize the average age of the accumulated information within each time slot.

[0109] like Figure 2 As shown, the service caching edge cloud computing network includes Individual user devices With one base station and one cloud server, user equipment can directly offload tasks to associated base stations for computation, or the associated base station can further forward tasks to non-associated base stations for computation (the task is offloaded to other base stations via the relay of the associated base station), or the associated base station can further forward tasks to the cloud server for computation (the task is offloaded to the cloud server via the relay of the associated base station). Figure 2 Black arrows indicate tasks being offloaded to associated base stations; green arrows indicate tasks being offloaded to other base stations via relay from associated base stations; and blue arrows indicate tasks being offloaded to cloud servers via relay from associated base stations. User equipment (UE) and base stations, as well as base stations themselves, communicate via wireless links in time-division multiplexing within sub-channels. Base stations and cloud servers communicate via wired links. A single base station can simultaneously receive one task and compute another, but at any given time, a base station can compute at most one task. In the service cache edge cloud computing network, UEs continuously offload tasks to base stations in time-division multiplexing until all tasks are offloaded. Base stations and cloud servers only stop computing when there are no more tasks to receive and compute.

[0110] The cloud server, based on the calculated transmission and computation delays, uses a queue-jump-based scheduling order algorithm to solve the task scheduling order subproblem and obtain the task scheduling order. User equipment In the time slot Uninstallation preference value Used to measure user equipment In the time slot The value of uninstallation The smaller, the better for user equipment. In the time slot The younger the information, the more priority it should be given to scheduling. Indicates in time slot User equipment The rate of decline in information age. The cloud server calculates user equipment based on sub-channel allocation decisions, task offloading decisions, and task selection decisions. In the time slot Total delay of unloading tasks .

[0111] In one embodiment, communication between user equipment and base stations, as well as between base stations, is based on orthogonal frequency division multiple access (OFDM) protocol, with sub-channels divided and time-division multiplexing within each sub-channel. Base stations communicate directly with the cloud server via wired links. In the time slot Total delay of unloading tasks The details are as follows:

[0112] 1) When user equipment In the time slot When offloading tasks to associated base stations For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots Computing User Equipment The computational delays of the unloading tasks are summed;

[0113] 2) When user equipment In the time slot When offloading tasks to unrelated base stations For user equipment In the time slot The transmission delay of offloading the task to the associated base station, and the associated base station in the time slot. Transmission delay to unassociated base stations, and the time slot of unassociated base stations. Computing User Equipment The computational delays of the unloading tasks are summed;

[0114] 3) When user equipment In the time slot When offloading the task to the cloud server For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots The transmission delays to the cloud server are added together.

[0115] In this system, user equipment (UE) is used to directly offload tasks to associated base stations for computation, or for associated base stations to forward tasks to cloud servers or non-associated base stations for computation. Associated base stations are those that cover the corresponding UE, while non-associated base stations are those that do not cover the corresponding UE. In the time slot Total delay of unloading tasks The details are as follows:

[0116] 1) When user equipment In the time slot Offload the task to the associated base station hour, The formula is as follows:

[0117]

[0118] in,

[0119]

[0120]

[0121] in, Indicates user equipment In the time slot Offload the task to the associated base station Transmission delay, Indicates associated base station In the time slot Computing User Equipment The computational delay of the unloading task. Indicates user equipment Task size, Indicates user equipment In the time slot To associated base station transmission rate, , , Indicates user equipment The number of CPU cycles required to complete each bit of the task is calculated from the number of cloud server or base station cycles needed. Indicates associated base station The calculation frequency.

[0122] If user equipment 1 is a pressure sensor, and the collected IoT data is pressure data, the corresponding task in the service cache of base station 1 is pressure data. User equipment 1 is within the coverage area of ​​base station 1, meaning base station 1 is an associated base station. Therefore, user equipment 1 offloads pressure data to base station 1. Based on the above formula, the pressure data at this time can be calculated. The value of .

[0123] 2) When user equipment In the time slot Offload the task to an unrelated base station hour, The formula is as follows:

[0124]

[0125] in,

[0126]

[0127]

[0128] in, Indicates user equipment In the time slot Offload the task to an unrelated base station Transmission delay, including user equipment In the time slot Offload the task to the associated base station Transmission delay and associated base stations In the time slot To non-associated base stations The transmission delay consists of two parts. Indicates associated base station In the time slot To non-associated base stations transmission rate, , .

[0129] If user equipment 1 is a pressure sensor, and the IoT data it collects is pressure data, and the corresponding task in the service cache of base station 2 is also pressure data, and user equipment 1 is not within the coverage area of ​​base station 2 (i.e., base station 2 is a non-associated base station), then user equipment 1 offloads the pressure data to base station 2. The result can be calculated using the formula above. The value is easy to understand. The sum of associated base stations and non-associated base stations is the total number of all base stations. For simplicity, the numbers of associated and non-associated base stations are kept consistent with the numbers of all base stations. If there are 5 base stations, for user equipment 1, the first base station is an associated base station, and the rest are non-associated base stations. Then the 5 base stations are represented as associated base station 1, non-associated base station 2, non-associated base station 3, non-associated base station 4, and non-associated base station 5, respectively. The rest are similar and will not be elaborated further.

[0130] 3) When user equipment In the time slot When offloading the task to the cloud server The formula is as follows:

[0131]

[0132] in,

[0133]

[0134] in, Indicates user equipment In the time slot Offloading tasks to cloud servers reduces transmission latency. Indicates associated base station In the time slot The transmission delay to the cloud server.

[0135] Due to the abundant computing resources of cloud servers, the computation latency is negligible, or considered to be zero, when a task is processed by the cloud server. (Associated base station) In the time slot The transmission latency to the cloud server is a preset value, such as 50 milliseconds.

[0136] S14. The cloud server issues a general decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data. The general decision includes sub-channel allocation decision, task offloading decision, task selection decision and task scheduling order.

[0137] In one embodiment, after the cloud server issues a general decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data, the following steps are also performed:

[0138] S15. The user device calculates its own information age. The cloud server calculates the average information age of the user devices based on the information age of all user devices as the reward for the deep reinforcement learning model, and generates experience for the deep reinforcement learning model. When the number of experience is greater than or equal to the preset number, the deep reinforcement learning model updates the model parameters based on the randomly selected experience, and the current time slot ends.

[0139] In one embodiment, the deep reinforcement learning model is in time slots The experience is represented by time slots Status, time slot Actions and time slots Rewards and time slots The quadruple composed of states, time slot The status includes all user equipment in the time slot Tasks, all user devices in time slots Information age, all base stations in time slots Service cache and all sub-channels in time slots Rayleigh fading factor, time slot Actions include time slots Sub-channel allocation decision, task offloading decision, and task selection decision, time slot The reward is for all user devices in the time slot The negative of the sum of information ages minus the time slot The penalty value, , This represents the total number of time slots in a training round.

[0140] The process involves the cloud server issuing overall decisions (including sub-channel allocation decisions, task offloading decisions, task selection decisions, and task scheduling order) to the corresponding base stations and user equipment. Specifically, the cloud server transmits the corresponding decisions from the overall decision to each user equipment and base station. Then, the user equipment performs task offloading based on the received decisions, the base station performs task offloading and calculations based on the received decisions, and the cloud server performs task calculations based on the decisions. After completion, the user equipment calculates its own information age and uploads it to the cloud server. The cloud server then aggregates these calculations to obtain the average information age of all user equipment.

[0141] The cloud server uses the average information age of all user devices as the reward for the deep reinforcement learning model (corresponding to...). (Partial), and generate the experience (quadruple) of the current time slot of the deep reinforcement learning model, which is stored in the experience replay pool of the deep reinforcement learning model. When the number of experiences in the experience replay pool is greater than or equal to the preset number, the deep reinforcement learning model updates the model parameters according to the randomly selected experience until the preset number is reached, and the current time slot ends.

[0142] Among them, the deep reinforcement learning model can adopt the SAC-D agent, and the SAC-D agent performs time-slot-based learning. Experience is represented as a quadruple consisting of the state, action, reward of the current time slot, and the state of the next time slot. .in:

[0143] 1) Time slot status , means as follows:

[0144]

[0145] in, Indicates all user equipment in the time slot The task Indicates all user equipment in the time slot Information age, This indicates that all base stations are in the time slot. Service cache, Indicates that all sub-channels are in the time slot Rayleigh fading factor;

[0146] 2) Time slot action , means as follows:

[0147]

[0148] in, Indicates time slot Sub-channel allocation decision, Indicates time slot Task unloading decision, Indicates time slot Task selection decision;

[0149] 3) Time slot Rewards , means as follows:

[0150]

[0151] in, Indicates user equipment In the time slot Information age, , Indicates time slot The penalty value when the SAC-D agent makes a time slot action When the corresponding constraint is violated, it needs to be subtracted. As a time slot Rewards As a form of punishment, From time slot The number of times the constraint is violated is multiplied by the preset single penalty value.

[0152] 4) Time slot status , means as follows:

[0153]

[0154] in, Indicates all user equipment in the time slot The task Indicates all user equipment in the time slot Information age, This indicates that all base stations are in the time slot. Service cache, Indicates that all sub-channels are in the time slot Rayleigh fading factor.

[0155] When the cloud server receives the age information uploaded by the user's device, it aggregates the data to form the experience of the SAC-D agent in the current time slot and stores it in the experience replay pool of the SAC-D agent. When the number of experiences in the experience replay pool is greater than or equal to the preset number, the SAC-D agent updates the model parameters based on the randomly selected experience, that is, the model parameters are updated once for each time slot.

[0156] Deep reinforcement learning models update their parameters based on randomly selected empirical data, as follows:

[0157] Based on random sampling The model parameters of the Actor network are updated using a set of empirical data, as shown in the following formula:

[0158]

[0159] in, Represents the model parameters of the Actor network The optimization objective function, Indicates in Mid-time slot status Down Expectations Indicates random sampling A collection of group experiences Indicates the first Model parameters of the first Critic network Down-output time slot status The value of the action below ;

[0160] No. The update formula for the first Critic network is as follows:

[0161]

[0162] in, Indicates the first Model parameters of the first Critic network The optimization objective function, express Expectations Indicates the first Model parameters of the second Critic network Down-output time slot status State value;

[0163] No. The update formula for the second Critic network is as follows:

[0164]

[0165] in, Indicates the first Model parameters of the second Critic network The optimization objective function, Indicates the update coefficients;

[0166] Temperature parameters The update formula is as follows:

[0167]

[0168] in, Indicates temperature coefficient The optimization objective function.

[0169] Among them, the SAC-D agent randomly samples from the experience replay pool. It is the SAC-D agent in the time slot status The action value, obtained from the output of the first Critic network, is used to measure the time slot. status The value of Actor networks. It is the SAC-D agent in the time slot status The state value is obtained from the output of the second Critic network. This soft state value function is used to measure the value of the state. The target Critic network is updated using a soft update method to maintain the stability of the update.

[0170] S2. Utilize the trained service cache edge cloud computing network to obtain the overall decision for the required time slot and complete the offloading and scheduling of IoT data for the corresponding time slot.

[0171] Example 1:

[0172] To facilitate understanding of this application and more clearly illustrate the application scenarios and advantages of the present invention, the following is a case study of sensor monitoring in a factory production environment within a smart industrial IoT scenario. In this factory, a service cache edge cloud computing network is deployed in a circular area with a radius of 150 meters, comprising one data center (i.e., a cloud server), five edge servers (i.e., base stations deployed on various production lines), and 20 sensors (i.e., user devices, such as temperature and humidity sensors, vibration sensors, pressure sensors, flow sensors, electrical parameter sensors, position sensors, gas concentration sensors, liquid level sensors, speed sensors, and vision sensors in the production environment). The base stations have a coverage range of 50 meters and do not overlap. The 20 sensors are randomly generated within the base station coverage area following an independently distributed Poisson point process.

[0173] The parameters are set as follows: The experimental process includes 1000 training rounds, each with 1000 time slots, and each time slot lasts for 0.5 seconds. All user equipment generates 10 types of tasks (environmental temperature and humidity data, vibration data, pressure data, flow data, electrical parameter data, location and displacement data, gas concentration data, liquid level data, rotational speed data, and image data). Each type of task can only be computed by a base station with the corresponding service cache. The size of each task is [100, 200] KB, and the number of CPU cycles required to process each bit of task from the user equipment is 100. The user equipment's space capacity for caching tasks (its own cache area, i.e., the user equipment capacity) is 500 KB. The user equipment's transmission power is 23 dBm. The maximum tolerable information age for the user equipment is 2 seconds. The base station's cache space is 30 GB, and each service cache requires [5, 8] GB of cache space. The base station's CPU computing frequency is 1 GHz, and the transmission power is 40 dBm. Regarding the channel model, the OFDMA protocol divides the wireless channel of the service cache edge cloud computing network into four sub-channels, adopts the Rayleigh fading model, and each sub-channel has a bandwidth of 20 MHz and a path loss exponent of 3. The learning rate of the SAC-D agent is 0.0001, and the reward decay discount factor is... It is 0.95.

[0174] The factory area needs to process monitoring data (tasks) from sensors in real time. It is preferable to collect only abnormal monitoring data, but all monitoring data can also be collected and processed through base stations and cloud servers for data analysis, abnormal data monitoring and early warning, etc.

[0175] This embodiment performs the following operations:

[0176] Initialization: At the start of each training round, 20 sensors are randomly positioned around each production line, and tasks are generated at regular intervals (time slots). The sensors send their tasks and information ages to the edge server. After collecting the sensors' tasks and information ages, the edge server uploads the sensors' tasks, information ages, and its own service cache to the cloud server. The cloud server then aggregates the data to obtain the overall status information.

[0177] Cloud server decision-making: The cloud server uses a SAC-D agent based on discrete action space to make decisions based on the overall state information, outputting sub-channel allocation decisions, task offloading decisions, and task selection decisions. Then, based on these decisions, a queue-jump-based scheduling algorithm is used to obtain the task scheduling order. For example, for sub-channel allocation decisions, the SAC-D agent tends to allocate sub-channels to sensors with older information to complete task computations as quickly as possible; for task offloading decisions, the SAC-D agent considers both the information age of the sensors and the computational load of the edge server; for task selection decisions, the SAC-D agent tends to select tasks with lower latency for offloading; and for task scheduling order, the queue-jump-based scheduling algorithm considers both the transmission and computational latency of each task, prioritizing the transmission and offloading of tasks with lower latency.

[0178] Decision Execution and Parameter Update: The cloud server aggregates sub-channel allocation decisions, task offloading decisions, task selection decisions, and task scheduling order into a master decision, and distributes this master decision to the corresponding sensors and edge servers. Sensors offload tasks based on the received decisions, and edge servers offload and compute tasks based on the received decisions. Sensors calculate their own information age and upload it to the cloud server. The cloud server aggregates this information age to calculate the average information age of all sensors, using it as the reward for the deep reinforcement learning model, generating and storing the SAC-D agent's experience for the current time slot. The SAC-D agent updates its parameters based on randomly selected experience. Through numerous training rounds, the SAC-D agent ultimately learns how to dynamically coordinate sub-channel allocation, task offloading, task selection, and task scheduling order in a complex service cache edge cloud computing network environment.

[0179] This method dynamically adjusts sub-channel allocation, task offloading, task selection, and task scheduling order based on the real-time tasks of user equipment. While meeting the computational needs of sensors, it ensures timely processing of sensor monitoring data (low information age) and guarantees fairness among different sensors (information age constraint). Through deep reinforcement learning, the SAC-D agent learns to maximize the overall performance of the service cache edge cloud computing network in long-term operation, avoiding long-term optimization problems that traditional static optimization methods struggle to address. A queue-based scheduling order algorithm accelerates the learning speed of the SAC-D agent. This invention is particularly suitable for IoT scenarios with densely distributed user equipment, high real-time requirements, and stringent sensor fairness requirements (such as industrial IoT, smart transportation, smart logistics, and smart security).

[0180] Furthermore, through comprehensive comparative analysis with various existing algorithms, the effectiveness and superiority of the task offloading and sequential scheduling method for service caching edge cloud computing networks proposed in this invention are verified:

[0181] (1) The SAC-D agent of this application is replaced with the following existing technologies: Proximal Policy Optimization (PPO, Proximal Scheduling), Dual-Delay Deep Deterministic Policy Gradient Algorithm (Policy Gradient), Deep Q-Network Algorithm (DQN), Traditional Soft Actor-Critic Algorithm (Traditional SAC), and Stochastic Decision Benchmark Scheme (Stochastic Scheme) to output sub-channel allocation decisions, task offloading decisions, and task selection decisions. This is used to compare and verify the effectiveness of the SAC-D agent used in this invention as a deep reinforcement learning model. The Stochastic Decision Benchmark Scheme (Stochastic Scheme) randomly selects sub-channel allocation decisions, task offloading decisions, and task selection decisions through a cloud server. This scheme serves as a performance benchmark to measure the performance improvement brought by the SAC-D agent to the service cache edge cloud computing network.

[0182] (2) Replace the scheduling order algorithm based on queue jumping in this application with the first-come-first-served strategy (first-come-first-served strategy) in the prior art to determine the task scheduling order after the output sub-channel allocation decision, task unloading decision and task selection decision, so as to compare and verify the effectiveness of the scheduling order algorithm based on queue jumping adopted in this invention.

[0183] The experimental environment was set up and trained according to the above parameter settings and existing methods. The trained SAC-D agent was then subjected to 10 complete cycles of experimentation in a random environment (i.e., randomly initialized user equipment geographic coordinates and base station service cache). The long-term cumulative average information age under each method was obtained. The experimental data are as follows: Figures 3-10 As shown. Figure 3The horizontal axis represents the number of training rounds, and the vertical axis represents the standardized cumulative reward. The standardized cumulative reward represents the percentage obtained by dividing the reward of the present application in the corresponding training round by the reward of the optimal solution. This figure demonstrates the significant superiority of the present invention in terms of cumulative reward when training rounds are superimposed. Figure 4 The middle horizontal axis represents the base station's computing power, the left vertical axis represents the task offloading ratio, and the right vertical axis represents the long-term cumulative average information age. The task offloading ratio refers to the proportion of tasks that are computed by associated base stations, computed by unassociated base stations, computed by cloud servers, and not offloaded. This figure illustrates the impact of base station computing power on the task offloading ratio and the long-term cumulative average information age performance index. Figure 5 The horizontal axis represents the base station transmission power. The higher the base station transmission power, the higher the base station transmission rate. This figure illustrates the impact of base station transmission power on the task offloading ratio and the long-term cumulative average information age performance index. Figure 6 The horizontal axis represents the base station service cache capacity. This parameter affects the number of service types that a base station can cache. The larger the capacity, the more types of tasks each base station can execute. This graph illustrates the impact of base station service cache capacity on the task offloading ratio. Figure 7 The horizontal axis represents the probability of user device task generation. The higher the probability, the easier it is for the user device to generate tasks. This figure reflects the impact of the user device task generation probability on the task offloading ratio and the long-term cumulative average information age performance index. Figure 8 The horizontal axis represents the number of user devices. This graph illustrates the impact of the number of user devices on the long-term cumulative average information age performance index. Figure 9 The horizontal axis represents the user equipment transmission power. The higher the user equipment transmission power, the higher the transmission rate of the user equipment. This figure reflects the impact of user equipment transmission power on the long-term cumulative average information age performance index. Figure 10 The horizontal axis represents user equipment capacity. A larger user equipment capacity allows the device to store more tasks. This graph illustrates the impact of user equipment capacity on the long-term cumulative average information age performance index. Based on the experimental results... Figures 3-10 The data shows that the task offloading and sequential scheduling method for service caching edge cloud computing networks of this invention exhibits significant superiority in information age performance metrics. Specifically, compared with existing methods, this invention excels in information age performance. Experimental data shows that the method of this invention can more efficiently reduce information age in different scenarios and ensures fairness among user devices through information age constraints, demonstrating its versatility and efficiency in dynamic environments. In summary, this invention has advantages in task completion real-time performance, user device fairness, and versatility, providing an efficient and reliable solution for service caching edge cloud computing networks.

[0184] It is readily understood that this invention can be deployed on an IoT data offloading and scheduling system within a service-cached edge cloud computing network. This system includes a memory and a processor, which are electrically connected directly or indirectly to enable data transmission or interaction. The memory stores a computer program that can run on the processor. The processor implements the IoT data offloading and scheduling method of this invention by running the computer program stored in the memory.

[0185] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for scheduling IoT data offloading in a service cache edge cloud computing network, applicable to... Individual user devices A service caching edge cloud computing network consisting of one base station and one cloud server is characterized by: Includes the following steps: S1. Train the service cache edge cloud computing network based on the IoT data collected by the user equipment until all training rounds have been traversed. In each time slot of each training round, the service cache edge cloud computing network performs the following operations: S11. Each user device generates a task based on the collected IoT data and stores it in its own cache. Each base station collects the status information of the corresponding user device based on its own service cache. The cloud server aggregates the status information of all user devices and the status information of the base stations to form the overall status information. S12. Based on the overall state information, use the deep reinforcement learning model deployed on the cloud server to output sub-channel allocation decisions, task offloading decisions, and task selection decisions; S13. Based on the sub-channel allocation decision, task offloading decision, and task selection decision, the cloud server uses a queue-jump-based scheduling order algorithm to obtain the task scheduling order. The queue-jump-based scheduling order algorithm includes the following steps: S131. Initialize time slot The task scheduling order is an empty list, and the user equipment is calculated. In the time slot Uninstallation preference value The formula is as follows: ； in, Indicates user equipment In the time slot Total delay of the unloading task Indicates user equipment In the time slot The generation time of the most recently completed task. Indicates user equipment In the time slot The generation time of the most recently completed task. , , This represents the total number of time slots in a training epoch; S132, All user equipment in the time slot The unloading preference values ​​are sorted in ascending order to obtain the unloading order of tasks as time slots. Initial task scheduling strategy ; S133, Judgment Does it satisfy the constraints of the queue-based scheduling ordering algorithm? If so, then... As a time slot The task scheduling order should be determined accordingly; otherwise, it should be adjusted according to the Johnson Law. Until the constraints of the queue-based scheduling ordering algorithm are met, the adjusted... As a time slot Task scheduling order; S14. The cloud server issues a general decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data. The general decision includes sub-channel allocation decision, task offloading decision, task selection decision and task scheduling order. S2. Utilize the trained service cache edge cloud computing network to obtain the overall decision for the required time slot and complete the offloading and scheduling of IoT data for the corresponding time slot.

2. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: During the process of training the service cache edge cloud computing network based on IoT data collected by user equipment until all training rounds are traversed, the geographical coordinates of the user equipment and the service cache of the base station are initialized before the start of each training round. The initialization of the geographical coordinates of the user equipment is achieved by randomly generating the user equipment within the coverage area of ​​each base station through an independently distributed Poisson point process. The initialization of the service cache of the base station is based on a Gaussian distribution.

3. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: The status information of the user equipment includes the user equipment's task and the user equipment's information age. The status information of the base station includes the base station's service cache. The user equipment is a sensor. The types of tasks include 10 categories: environmental temperature and humidity data, vibration data, pressure data, flow data, electrical parameter data, position and displacement data, gas concentration data, liquid level data, rotation speed data, and image data.

4. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: The objective function of the deep reinforcement learning model is to optimize subchannel allocation decisions, task offloading decisions, and task selection decisions to minimize the total number of user devices. The average age of the cumulative information within each time slot; The constraints of the deep reinforcement learning model include: 1) Service caching constraint: The unloading task is calculated by the cloud server or by the base station with the corresponding caching service; 2) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time; 3) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age; 4) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

5. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: The objective function of the queue-jump-based scheduling order algorithm is to optimize the task scheduling order to minimize the impact of all user devices on scheduling. The average age of the cumulative information within each time slot; The constraints of the scheduling order algorithm based on queue jumping include: 1) Base station computation constraints: The same base station can receive one task and compute another task simultaneously, and the same base station can compute at most one task at any given time; 2) Information age constraint: The information age of the user's device is lower than the maximum tolerable information age; 3) Task completion time constraint: The unloaded task must be calculated and completed within the corresponding time slot.

6. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: The user equipment and the base station, as well as the base stations themselves, communicate within sub-channels based on the Orthogonal Frequency Division Multiple Access (OFDM) protocol, using time division multiplexing. The base station communicates directly with the cloud server via a wired link. In the time slot Total delay of unloading tasks The details are as follows: 1) When user equipment In the time slot When offloading tasks to associated base stations For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots Computing User Equipment The computational delays of the unloading tasks are summed; 2) When user equipment In the time slot When offloading tasks to unrelated base stations For user equipment In the time slot The transmission delay of offloading the task to the associated base station, and the associated base station in the time slot. Transmission delay to unassociated base stations, and the time slot of unassociated base stations. Computing User Equipment The computational delays of the unloading tasks are summed; 3) When user equipment In the time slot When offloading the task to the cloud server For user equipment In the time slot The transmission delay of offloading tasks to associated base stations and the associated base stations in time slots The transmission delays to the cloud server are added together.

7. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: After the cloud server issues the overall decision to the corresponding base station and user equipment to complete the offloading and scheduling of IoT data, the following steps are also performed: S15. The user device calculates its own information age. The cloud server calculates the average information age of the user devices based on the information age of all user devices as the reward for the deep reinforcement learning model, and generates experience for the deep reinforcement learning model. When the number of experience is greater than or equal to the preset number, the deep reinforcement learning model updates the model parameters based on the randomly selected experience, and the current time slot ends.

8. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 7, characterized in that: The deep reinforcement learning model in time slots The experience is represented by time slots Status, time slot Actions and time slots Rewards and time slots The time slot is composed of a quadruple of states. The status includes all user equipment in the time slot Tasks, all user devices in time slots Information age, all base stations in time slots Service cache and all sub-channels in time slots Rayleigh fading factor, the time slot Actions include time slots Sub-channel allocation decision, task offloading decision, and task selection decision, the time slot The reward is for all user devices in the time slot The negative of the sum of information ages minus the time slot The penalty value, , This represents the total number of time slots in a training round.

9. The IoT data offloading and scheduling method for a service cache edge cloud computing network as described in claim 1, characterized in that: The deep reinforcement learning model is a SAC-D agent, which includes an Actor network, two first Critic networks, and two second Critic networks. Both first Critic networks are connected to the Actor network, and the second Critic networks are connected one-to-one with the first Critic networks. The objective function of the Actor network in the SAC-D agent is... The formula is as follows: ； in, Indicates obtaining Expectations The largest Actor network model parameters , This represents the decay discount factor of the reward. Indicates time slot The reward Indicates the temperature coefficient. Indicates time slot entropy, , This represents the total number of time slots in a training round.

Images