A Method for Offloading and Resource Allocation in Heterogeneous Edge Computing Networks Based on Information Age

Abstract

This invention belongs to the field of edge computing technology, and particularly relates to an offloading and resource allocation method for heterogeneous edge computing networks based on information age. The invention includes: performing federated learning based on the experience of each mobile device; updating the local neural network of each mobile device based on the global neural network parameters obtained from the federated learning; training the local neural network and using the trained local neural network to output the offloading decision for the task corresponding to the current time slot; adjusting offloading decisions that do not meet energy consumption constraints using an elite-preserving genetic algorithm; calculating the computing and bandwidth resources allocated by each base station to each mobile device; executing the offloading decision and computing and bandwidth resource allocation tasks; and calculating the reward for each mobile device. This invention reduces the complexity of problem solving and ensures strict compliance with energy consumption constraints through a secure learning mechanism, significantly improving the convergence speed and robustness of the strategy.

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology, and in particular relates to a method for offloading and resource allocation in heterogeneous edge computing networks based on information age. Background Technology

[0002] The rapid development of the Internet of Things (IoT) has driven the emergence of new applications such as autonomous driving and virtual reality, which place stringent demands on the real-time performance of computing tasks and the battery life of devices. However, mobile devices are typically limited by battery capacity and computing power, making it difficult for them to independently complete computationally intensive and latency-sensitive tasks. Mobile edge computing (MEC) can effectively reduce latency and save device power by offloading computing tasks to servers at the network edge.

[0003] However, mobile edge computing networks based on single base stations still have significant limitations in task offloading optimization. Due to the limited coverage and computing resources of a single base station, as the number of mobile devices or the task load increases, the base station is prone to overload, leading to problems such as increased task processing latency and energy consumption. Furthermore, single-base station mobile edge computing networks struggle to meet the heterogeneous needs of different mobile devices in terms of computing power, energy consumption, and task types. This single-base station service model cannot be optimized for the personalized needs of different devices, easily resulting in wasted computing resources or uneven service quality.

[0004] To address the aforementioned problem, a heterogeneous mobile edge computing network consisting of one macro base station (MBS) and multiple micro base stations (SBS) is considered. Both the macro and micro base stations are equipped with mobile edge computing servers to compute tasks offloaded from mobile devices. The macro base station provides service to users over a wide area, while micro base stations are deployed in hotspot areas to alleviate the computational and communication resource pressure on the macro base station. Mobile devices in remote areas can offload tasks to the macro base station, while mobile devices in hotspot areas can choose to offload tasks to either micro or macro base stations based on their needs.

[0005] This heterogeneous edge computing network, through the collaborative work of multiple base stations, can not only extend network coverage but also achieve flexible scheduling of computing resources, providing lower-latency computing offloading services for mobile devices. However, how to optimize binary task offloading decisions and communication and computing resource allocation in heterogeneous mobile edge computing networks to minimize the long-term weighted information age (AoI) while satisfying the energy consumption constraints of mobile devices has not yet been thoroughly studied.

[0006] Information age, as a metric for data freshness, reflects the real-time nature of computation results better than traditional latency metrics. Existing heterogeneous mobile edge computing networks do not consider information age optimization, which significantly impacts their performance. Furthermore, traditional optimization methods struggle to adapt to dynamic network environments, and conventional distributed deep reinforcement learning (DRL) is prone to training instability in multi-agent scenarios due to policy conflicts.

[0007] Patent application CN118139114A discloses a data collection and computation offloading method for low-altitude intelligent networks. It calculates the information age on the basis of the entire time slot, but does not consider the time delay constraints of each time slot on the task, making the measurement of information age inaccurate.

[0008] Patent application CN116321299A discloses a method for task offloading and resource allocation in vehicle networking based on MEC under information age constraints. However, it does not directly optimize information age, and its modeling does not consider the changes in the location of devices in the network and the changes in the channel state of each time slot, which cannot adapt to the needs of highly dynamic networks.

[0009] Patent application CN119383667A discloses a scheduling method for vehicular edge computing networks based on information age, but it does not consider the allocation of communication and computing resources in the network. With the increasing number of IoT devices, communication and computing resources are becoming increasingly scarce. Summary of the Invention

[0010] The purpose of this invention is to solve the problems of the prior art and propose an offloading and resource allocation method for heterogeneous edge computing networks based on information age. This method is applicable to heterogeneous edge computing network scenarios using Orthogonal Frequency Division Multiple Access (OFDMA) communication. It jointly optimizes binary task offloading decisions and bandwidth and computing resource allocation. The binary task offloading decisions are optimized by a distributed method assisted by federated learning, thereby minimizing the long-term weighted information age of the heterogeneous edge computing network while strictly satisfying energy consumption constraints.

[0011] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0012] A method for offloading and resource allocation in a heterogeneous edge computing network based on information age, wherein the heterogeneous edge computing network includes one macro base station, K micro base stations, and I mobile devices, and each base station is equipped with an edge computing server, the method comprising:

[0013] At the start of each time slot, all mobile devices move and transmit their first state information to the macro base station. The macro base station then broadcasts the collected first state information and the reward from the previous time slot to all mobile devices.

[0014] Each mobile device stores its experience and performs federated learning based on that experience. It updates the local neural network parameters of each mobile device based on the global neural network parameters obtained from the federated learning, and uses the trained local neural network to output the unloading decision for the task corresponding to the current time slot. The experience includes the third state information of the previous time slot, the unloading decision of the previous time slot, the reward obtained in the previous time slot, and the third state information of the current time slot. The third state information includes the first state information, the geographical coordinates of the current time slot, and the priority of all mobile devices.

[0015] Each mobile device sends an offloading decision to the macro base station. The macro base station outputs the optimal offloading decision based on the offloading decisions of all mobile devices and sends the optimal offloading decision to the corresponding mobile device.

[0016] The macro base station calculates the computing and bandwidth resources allocated to each mobile device by each micro base station and the macro base station, and sends the calculation results to the corresponding micro base station.

[0017] Mobile devices execute corresponding offloading decisions, while micro base stations and macro base stations execute corresponding computing and bandwidth resource allocation tasks.

[0018] The macro base station calculates the reward for each mobile device, and the time slot ends;

[0019] After traversing all time slots of a training round, initialize the priority and geographic coordinates of the mobile device, and start the next training round, until all training rounds have been traversed.

[0020] Furthermore, each mobile device stores its experience and performs federated learning based on that experience. The local neural network parameters of each mobile device are updated based on the global neural network parameters obtained from the federated learning, including:

[0021] Each mobile device stores experience in its own experience replay pool, and when the number of experience in the experience replay pool is greater than or equal to a preset number, it randomly extracts a preset number of experience.

[0022] Each mobile device updates its local neural network parameters based on extracted experience;

[0023] After each preset round, each mobile device sends its current local neural network parameters to the macro base station for aggregation, and uses the aggregated global neural network parameters to update the local neural network parameters of each mobile device.

[0024] Furthermore, each mobile device updates its local neural network parameters based on extracted experience, including:

[0025] The local neural network parameters include local Q-network parameters and local target Q-network parameters. Each mobile device updates its local neural network parameters using randomly selected experience according to the following formula:

[0026]

[0027]

[0028]

[0029] in, Mobile devices exist The first time slot extraction The local target Q value of the experience in the corresponding time slot, Mobile devices exist The rewards obtained from time slots It is a discount factor. Mobile devices exist The first time slot extraction The experience is the third state information in the corresponding time slot. Mobile devices exist The offloading decision made by the time slot Mobile devices exist The set of all actions that a time slot can perform. Is to make in The first time slot extraction The experience corresponds to the next time slot mobile device The action with the largest estimated Q-value in the local Q-network. Mobile devices Local Q network parameters, Mobile devices The local target Q network parameters, Mobile devices The local target Q network for in The first time slot extraction The experience corresponds to the third state information of the next time slot and the estimated local Q value of the action. Mobile devices The loss function of the local Q network, Mobile devices The number of randomly selected experience samples. Mobile devices The local Q network for in The first time slot extraction The experience corresponds to the third state information of the time slot and the estimated local Q value of the action. It's the learning rate. Mobile devices The gradient of the local neural network parameters. Represents the loss function gradient The derivative;

[0030] If the result of modulo 2 in the current training epoch is equal to 0, then each mobile device updates the local target Q network parameters as follows: .

[0031] Furthermore, the step of using the trained local neural network to output the offloading decision for the task corresponding to the current time slot includes:

[0032] The third state information of the current time slot is input into the trained local neural network, and the offloading decision of the current time slot is output. The offloading decision of the mobile device within the coverage area of ​​the micro base station includes not offloading, offloading to the micro base station, offloading to the macro base station, and offloading the task relay to the macro base station through the micro base station. The offloading decision of the mobile device outside the coverage area of ​​the micro base station includes not offloading and offloading to the macro base station.

[0033] Furthermore, the macro base station outputs an optimal offloading decision based on the offloading decisions of all mobile devices, and sends the optimal offloading decision to the corresponding mobile device, including:

[0034] Determine whether all unloading decisions meet the energy consumption constraints. If not, use an elite-preserving genetic algorithm to adjust the unloading decisions and send the adjusted optimal unloading decision to the corresponding mobile device. If they meet the constraints, send the current unloading decision to the corresponding mobile device.

[0035] Furthermore, the problem of adjusting the unloading decision is transformed into a knapsack problem, expressed by the following formula:

[0036]

[0037]

[0038] in, This represents the set of all mobile devices that choose to offload tasks to the current micro base station or macro base station. This indicates the option to offload the task to the current micro base station or macro base station. The decision of whether to continue uninstalling the task on a mobile device. Indicates the unloading decision The set, Indicates uninstallation. Indicates that you will not uninstall. Indicates mobile device The reward value that can be increased after choosing to uninstall Indicates mobile device The weight of the backpack required to unload the task. This indicates the total capacity of the backpack.

[0039] Furthermore, the process of adjusting the unloading decision using an elite-preserving genetic algorithm includes:

[0040] Initialize a population The population It contains N chromosomes, each of which... That is, the unloading decision set ;

[0041] Assess the fitness of each chromosome. and feasibility ;

[0042] The chromosomes will be directly replaced by the next generation of chromosomes;

[0043] Chromosome 1 will be The probability is retained in the next generation;

[0044] After a preset number of iterations, the chromosome with the highest fitness is selected as the optimal unloading decision.

[0045] Furthermore, the macro base station calculates the computing and bandwidth resources allocated by the macro base station to each micro base station and each mobile device, including:

[0046] The macro base station uses the Caro-Kuhn-Tucker condition to calculate the optimal computing resources allocated to each mobile device by each micro base station and macro base station, expressed by the formula:

[0047]

[0048] in, Indicates base station In the time slot Assigned to mobile devices The optimal computing resource size is determined by the base station, which includes macro base stations and micro base stations. Indicates base station Total computing power Indicates mobile device priority, Indicates mobile device In the time slot The task buffer stores the size of the task. This represents the number of CPU cycles required to compute 1 bit of data. Indicates in time slot Offload the task to the base station A collection of mobile devices;

[0049] The bandwidth resources allocated to each mobile device are calculated using the interior-point method.

[0050] Furthermore, the macro base station calculates the reward for each mobile device, including:

[0051] The macro base station calculates the corresponding reward for each mobile device based on the information age of all mobile devices and the constraint violations of each mobile device. The calculation formula is as follows:

[0052]

[0053] in, and This represents a constant used to balance the magnitudes. Indicates mobile device priority, Indicates mobile device In the time slot Information age, , and They represent time slots respectively. mobile device The penalty for exceeding the maximum tolerance thresholds in terms of energy consumption, information age, and service time is calculated using the following formula:

[0054]

[0055]

[0056]

[0057] in, This indicates a penalty imposed on the mobile device whose uninstallation decision was adjusted. For mobile devices In the time slot Offload the task to the base station The energy consumed, the base stations include macro base stations and micro base stations, Maximum power consumption limit for mobile devices in each time slot, The maximum tolerance information age threshold, For mobile devices The service time of the current time slot, It represents the time of a time slot.

[0058] Furthermore, if mobile devices If the task is successfully uninstalled, then the mobile device... In the time slot The information age is the time elapsed from the generation of a successfully uninstalled task to its uninstallation, plus the control time and the time spent transmitting and calculating the time spent on successfully uninstalling the task; if the mobile device If the task is not uninstalled, then the mobile device In the time slot Information age for mobile devices In the time slot The information age plus a time slot.

[0059] Compared with the prior art, the significant advantages of this invention are: (1) Introducing a hierarchical decomposition and collaborative optimization framework: The complex non-convex mixed integer nonlinear programming problem is decomposed into a top-level problem of unloading decision optimization and a sub-problem of resource allocation optimization. Discrete unloading decisions are optimized by multi-agent deep reinforcement learning algorithm, and continuous bandwidth and computing resource allocation are optimized by convex optimization method. This not only reduces the complexity of problem solving, but also ensures that energy consumption constraints are strictly met through a safe learning mechanism; (2) Designing federated learning-assisted multi-agent deep reinforcement learning, where each mobile device acts as an independent agent to make distributed decisions, avoiding the algorithm training caused by the huge action space. The problems of slow training speed and poor performance are solved by aggregating global knowledge through federated learning to alleviate the instability of multi-agent training, which significantly improves the convergence speed and robustness of the strategy; (3) A more detailed information age calculation method is considered. The information age is not only calculated on the basis of the entire time slot, but the specific time required to process the computing task within a time slot is also included in the calculation of the information age. At the same time, the time delay constraint of each time slot on the task is also considered; (4) The cooperation method between micro base stations and macro base stations is considered. That is, the device can relay the task to the macro base station through the micro base station, which reduces the energy consumption of the device and alleviates the bandwidth resource pressure of the macro base station. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the heterogeneous edge computing network of the present invention;

[0061] Figure 2 This is a time-slot structure diagram of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] This invention provides a method for offloading and resource allocation in heterogeneous edge computing networks based on information age, the specific steps of which include:

[0064] Step 1: Initialize the neural network parameters at each mobile device and initialize the neural network parameters at the macro base station.

[0065] Specifically, the neural network parameters of the agent at each mobile device are initialized, including the local Q-network parameters. and local target Q network parameters Initialize the global neural network parameters at the macro base station, including the global Q-network parameters. and global target Q network parameters Initialize the total number of training rounds Initialize the total number of time slots for each training round. Set the current training round. .

[0066] Step 2: Initialize the priority and geographic coordinates of the mobile device.

[0067] Priority for initializing mobile devices With geographic location coordinates, where priority The range of values ​​is .

[0068] like Figure 1 As shown, in a heterogeneous edge computing network, the initial locations of K micro base stations and I mobile devices follow a static Poisson point distribution. The coverage areas of the K micro base stations do not overlap and are all within the coverage area of ​​the macro base station.

[0069] In a heterogeneous edge computing network, each mobile device is equipped with a task buffer and generates tasks with a certain probability in each time slot. If the mobile device generates data in the current time slot, the task is stored in the task buffer, and previously unloaded tasks are discarded. Otherwise, unloaded tasks remain stored in the task buffer.

[0070] Step 3: During the time slot control phase, the mobile device moves and then transmits its local first state information to the macro base station. After collecting the first state information of all mobile devices in the network, the macro base station broadcasts the collected first state information to all mobile devices. The mobile device generates the second state information of the current time slot based on the received first state information. If the current time slot is not the first time slot, the macro base station also broadcasts the reward calculated in the previous time slot to all mobile devices.

[0071] The first state information includes the size of all tasks in the current time slot of the mobile device and the task generation time, the information age of the current time slot of the mobile device, the coverage area of ​​which base station the current time slot of the mobile device is within, and the channel gain of the current time slot of the mobile device with respect to micro base stations and macro base stations.

[0072] like Figure 2As shown, each time slot includes a duration of The control phase and duration are The unloading phase, including the control phase, lasts for a period of time. The mobile phase of mobile devices.

[0073] In a heterogeneous edge computing network, I mobile devices move according to a Gauss-Markov model during the movement phase.

[0074] In the time slot ,mobile device movement speed and direction Updated to:

[0075]

[0076]

[0077] in, and These are constants used to adjust the influence of the velocity and direction of the previous time slot on the velocity and direction of the current time slot. It is the average speed of all mobile devices. Mobile devices The average direction, This represents two Gaussian processes with different mean and variance parameters.

[0078] In the time slot ,mobile device Geographic coordinates Updated to:

[0079]

[0080] In the time slot If mobile device Select the uninstall task, then move the device. In the time slot The information age is updated to the time elapsed from the generation of the successfully uninstalled task to its uninstallation, plus the control time and the time spent transmitting and calculating the task. or ,in Indicates mobile device The time elapsed from the creation of a successfully uninstalled task to its uninstallation. Indicates mobile device The transmission time required to directly offload the task to a micro base station or macro base station. Indicates mobile device Through its affiliated micro base station The transmission time required to offload the task relay to the macro base station. Indicates base station Computing mobile devices The computation time required for the unloading task. Indicates macro base station computing mobile device The computation time required for the unloading task, base station This indicates a macro base station or one of the micro base stations.

[0081] If mobile device If you choose not to uninstall tasks or if there are no tasks in the task buffer, then the mobile device... The information age is updated by adding the time length of one time slot. .

[0082] The problem of minimizing the age of long-term weighted information in heterogeneous edge computing networks is expressed by the following formula: ,in, Indicates the total time slot length. Indicates mobile device priority, Indicates mobile device In the time slot Information age.

[0083] In the time slot The second state information generated by the mobile device includes: its current time slot geographic location coordinates. The base station to which all mobile devices currently belong in the time slot (If the mobile device is within the coverage area of ​​a micro base station, its base station is a micro base station; if the mobile device is not within the coverage area of ​​a micro base station, its base station is a macro base station.) All mobile devices' current timeslots are connected to their respective micro base stations. The ratio of transmission rate to bandwidth (If the base station is a macro base station, the corresponding ratio is 0), the ratio of the transmission rate to the bandwidth of all mobile devices in the current time slot to the macro base station. All mobile devices in time slots Information Age All mobile devices in time slots The time elapsed since task creation in the task buffer (If there are no tasks in the task buffer, the corresponding time value is 0), all mobile devices in the time slot The task buffer stores the size of the task. (If there are no tasks in the task buffer, the corresponding task size value is 0), priority of all mobile devices. .

[0084] Step 4: If the current time slot is not the first time slot, each mobile device stores the experience from the previous time slot into its own experience replay pool, and randomly extracts the preset number of experience when the number of experience in the experience replay pool is greater than or equal to the preset number of experience.

[0085] The experience includes the third state information of the mobile device in the previous time slot, the offloading decision output by the mobile device in the previous time slot, the reward received by the mobile device, and the third state information of the mobile device in the current time slot. The third state information includes the first state information, the geographical coordinates of the current time slot, and the priority of all mobile devices.

[0086] If the current time slot is the first time slot, based on the third state information, each mobile device uses the initialized local neural network to output the offloading decision for the task corresponding to the current time slot.

[0087] If the current time slot is not the first time slot, each mobile device will use the experience from the previous time slot. Store the experience in its own experience replay pool, and the number of experiences in the experience replay pool is greater than or equal to the preset number. At that time, randomly select One experience, among which Mobile devices exist The third state information of the time slot, Mobile devices exist The offloading decision made by the time slot Mobile devices exist The rewards obtained from time slots Mobile devices exist The third state information of the time slot.

[0088] Step 5: Once the experience replay pool has been randomly sampled, each mobile device updates the neural network parameters based on the sampled experience.

[0089] If the experience replay pool is fully sampled, each mobile device will use a randomly selected... This experience updates the local Q network parameters according to the following formula:

[0090]

[0091]

[0092]

[0093] in, Mobile devices exist The first time slot extraction The local target Q value of the experience in the corresponding time slot, Mobile devices exist The rewards obtained from time slots It is a discount factor. Mobile devices exist The first time slot extraction The experience is the third state information in the corresponding time slot. Mobile devices exist The offloading decision made by the time slot Mobile devices exist The set of all actions that a time slot can perform. Is to make in The first time slot extraction The experience corresponds to the next time slot mobile device The action with the largest estimated Q-value in the local Q-network. Mobile devices Local Q network parameters, Mobile devices The local target Q network parameters, Mobile devices The local target Q network for in The first time slot extraction The experience corresponds to the third state information of the next time slot and the estimated local Q value of the action. Mobile devices The loss function of the local Q network, It is the number of randomly selected experiences. Mobile devices The local Q network for in The first time slot extraction The experience corresponds to the third state information of the time slot and the estimated local Q value of the action. It's the learning rate. Mobile devices The gradient of the local neural network parameters. Represents the loss function gradient The derivative;

[0094] If the current training round If the result of modulo 2 equals 0, then the parameters of the local target Q network updated by each mobile device are: .

[0095] Step 6: If the result of the current round modulo the preset federated learning round is equal to 0, each mobile device sends the currently trained local neural network parameters to the macro base station for aggregation, and then uses the aggregated parameters to update the local neural network, thereby completing the federated learning.

[0096] If the current training round Regarding the pre-defined federated learning aggregation rounds The result of modulo operation is 0. Each mobile device will use the currently trained local neural network parameters. and Sending to macro base stations, the macro base stations use the formula and The parameters of all mobile devices are aggregated into global Q-network parameters and global target Q-network parameters. The aggregated global neural network parameters are then broadcast to all mobile devices, and all mobile devices process them according to the formula... and Update the parameters of the local Q network and the local target Q network to complete federated learning.

[0097] Step 7: Each mobile device uses its trained local neural network to output an offloading decision based on the input third-state information. For mobile devices within the micro base station's coverage area, the offloading decision includes not offloading, offloading to the micro base station, offloading to the macro base station, and offloading the task relay from the micro base station to the macro base station. For mobile devices outside the micro base station's coverage area, the offloading decision includes not offloading and offloading to the macro base station. After outputting the offloading decision, each mobile device sends it to the macro base station.

[0098] In the time slot ,mobile device The unloading decision is defined as:

[0099]

[0100] in, Indicates mobile device Located in micro base station Within its coverage area, This indicates that the mobile device is in this situation. Possible unloading decisions, Indicates mobile device Choose not to uninstall Indicates mobile device Choose to directly offload the task to the micro base station. , Indicates mobile device Choose to directly offload the task to the macro base station. Indicates mobile device Select via micro base station The task relay is offloaded to the macro base station. Indicates mobile device It is not within the coverage area of ​​any micro base station, but only within the coverage area of ​​a macro base station. This indicates that the mobile device is in this situation. Possible unloading decisions, Indicates mobile device Choose not to uninstall Indicates mobile device Choose to offload the task to the macro base station.

[0101] This embodiment adopts binary task offloading decision, that is, the wireless device can only offload the complete task to the micro base station or macro base station, or choose not to offload it.

[0102] Step 8: After receiving offloading decisions from all mobile devices, the macro base station first determines whether the current combination of offloading decisions strictly satisfies the energy consumption constraint. If not, it uses an elite-preserving genetic algorithm to adjust the offloading decisions, ultimately outputting an offloading decision combination that strictly satisfies the energy consumption constraint, and sends the adjusted offloading decision to the corresponding mobile device. If it satisfies the constraint, it sends the current offloading decision to the corresponding mobile device.

[0103] mobile device Energy consumption constraints need to be met in each time slot. ,in, For mobile devices In the time slot Offload the task to the base station Energy consumed For mobile devices Transmission power, Maximum power consumption limit for mobile devices in each time slot. Mobile devices The energy consumption constraint can be transformed into a minimum demand bandwidth constraint using the following formula:

[0104]

[0105]

[0106] in, Indicates base station Assigned to mobile devices bandwidth ratio, Indicates base station Total bandwidth size, Indicates from mobile device to base station Channel gain, This represents Gaussian white noise. Since the bandwidth of each base station is independent and their bandwidth allocation does not interfere with each other, determining whether the offloading decision strictly satisfies the energy consumption constraint can be transformed into whether the total bandwidth of each base station is greater than or equal to the sum of the minimum bandwidth requirements of all mobile devices choosing to offload tasks to that base station. If not, then for a single base station, the offloading decision adjustment problem for all mobile devices choosing to offload tasks to it can be transformed into the following knapsack problem:

[0107]

[0108]

[0109] in, This represents the set of all mobile devices that choose to offload their tasks to this base station. This indicates the offloading decision for all mobile devices that have currently chosen to offload tasks to this base station, and whether they should continue to offload tasks. The set, each unloading decision Indicates mobile device Should the task continue to be uninstalled from the base station that it has currently selected to uninstall? Indicates uninstallation. This indicates that you will not uninstall. Indicates mobile device The value is equivalent to that of mobile devices. The reward value added by choosing to uninstall is the difference between the updated information age after choosing to uninstall and after choosing not to uninstall. Due to current limitations in bandwidth and computing resource allocation, mobile devices... After selecting uninstallation, the updated information age is approximately equal to the time elapsed since the task was created, as stored in its task buffer. So, mobile devices The difference between the updated information age after choosing to uninstall and choosing not to uninstall can be expressed by the formula:

[0110]

[0111] in, Indicates mobile device The penalty imposed if the information age exceeds the maximum information age after choosing not to uninstall the task.

[0112] Indicates mobile device The weight of the backpack required to unload a task is equivalent to its minimum bandwidth requirement, i.e. .in, This indicates the total capacity of the backpack, which is... .

[0113] This knapsack problem can be solved using an elitist genetic algorithm. Specifically, a population is first initialized. It contains N chromosomes, each of which... That is, the unloading decision set That is, population There are a total of N unloading decision sets. Then, the fitness of each chromosome is evaluated. and feasibility The feasibility conditions are not met, that is... The chromosomes of the current generation will be directly replaced by the next generation's chromosomes. According to the elite preservation strategy, this meets the feasibility condition, namely... Chromosome 1 will be The probability of a change in the unloading decision is preserved for the next generation. In each generation replacement, the preserved chromosomes remain unchanged, while other chromosomes exchange some genes (i.e., alter some unloading decisions) through single-point crossover, multi-point crossover, or uniform crossover to generate new individuals. Simultaneously, these chromosomes also randomly alter some genes (i.e., alter some unloading decisions) with a low probability to generate new individuals. The new generation replaces the old population, and fitness and feasibility are repeatedly evaluated, followed by another population replacement. This process repeats until a certain number of generations are reached. The final output is the optimal unloading decision that strictly satisfies energy constraints. Furthermore, to facilitate deep reinforcement learning training, corresponding penalties are imposed on these mobile devices whose unloading decisions have been adjusted. .

[0114] Step 9: Based on the offloading decision, the macro base station uses the Cartesian-Kuhn-Tucker (KKT) condition to calculate the optimal computing resources (i.e., computing frequency) that each base station needs to allocate to each mobile device. Then, it uses the interior-point method to calculate the bandwidth resources (i.e., bandwidth size) that need to be allocated to each mobile device. After allocating the output bandwidth and computing resources, the macro base station sends it to the corresponding micro base station.

[0115] Because the computing resources of each base station are independent and their allocation does not interfere with each other, the formula for allocating computing resources for each base station is as follows:

[0116]

[0117]

[0118] in, Indicates base station In the time slot The set of computing resource sizes allocated to each mobile device, where each element... Indicates base station In the time slot Assigned to mobile devices The size of computing resources, Indicates in time slot Offload the task to the base station A collection of mobile devices, This represents the number of CPU cycles required to compute 1 bit of data. Indicates base station The total computing power. The computing resource allocation problem for each of the above base stations is a convex problem, and the optimal closed-form solution can be obtained directly from the KKT conditions. .

[0119] Step 10: The mobile device makes the corresponding offloading decision, and the micro base station and macro base station make the corresponding bandwidth and computing resource allocation.

[0120] Step 11: The macro base station calculates the corresponding reward.

[0121] The macro base station calculates the corresponding reward for each mobile device based on the information age changes of all mobile devices and the constraint violations of each mobile device. The calculation formula is as follows:

[0122]

[0123] in, and It is a constant used to balance the magnitudes. , and These represent mobile devices. The penalty for exceeding the maximum tolerance thresholds in terms of energy consumption, information age, and service time is calculated using the following formula:

[0124]

[0125]

[0126]

[0127] in, The maximum tolerance information age threshold, Service time is the sum of control time, transmission time, and computation time.

[0128] Step 12: If the current time slot is less than the total number of time slots, then use the next time slot as the current time slot and return to step 3.

[0129] That is, if the current time slot ,but Then proceed to step 3.

[0130] Step 13: If the current training round is less than the total number of training rounds, return to step 2 to continue training; otherwise, end the training.

[0131] That is, if and ,but And proceed to step 2 to continue training; if and Then all training is complete.

[0132] After all training is completed, the heterogeneous edge computing network can directly use the trained model to perform task offloading and resource allocation according to steps 7-10.

[0133] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for offloading and resource allocation in heterogeneous edge computing networks based on information age, characterized in that, The heterogeneous edge computing network includes one macro base station, K micro base stations, and I mobile devices. Each base station is equipped with an edge computing server. The heterogeneous edge computing network offloading and resource allocation method based on information age includes: After each time slot begins, all mobile devices move and transmit their first state information to the macro base station. The macro base station then broadcasts the collected first state information and the reward from the previous time slot to all mobile devices. The first state information includes the size of all tasks in the current time slot of the mobile device and the task generation time, the information age of the current time slot of the mobile device, the coverage area of ​​which base station the current time slot of the mobile device is within, and the channel gain between the current time slot of the mobile device and the micro base station and the macro base station. Each mobile device stores its experience and performs federated learning based on that experience. The local neural network parameters of each mobile device are updated based on the global neural network parameters obtained from the federated learning. The trained local neural network outputs the unloading decision for the task corresponding to the current time slot. The experience includes the third state information of the previous time slot, the unloading decision of the previous time slot, the reward obtained in the previous time slot, and the third state information of the current time slot. The third state information of the previous time slot includes the first state information of the previous time slot, the geographical coordinates of the previous time slot, and the priorities of all mobile devices. The third state information of the current time slot includes the first state information of the current time slot, the geographical coordinates of the current time slot, and the priorities of all mobile devices. Each mobile device sends an offloading decision to the macro base station. The macro base station outputs the optimal offloading decision based on the offloading decisions of all mobile devices and sends the optimal offloading decision to the corresponding mobile device. The macro base station calculates the computing and bandwidth resources allocated to each mobile device by each micro base station and the macro base station, and sends the calculation results to the corresponding micro base station. Mobile devices execute corresponding offloading decisions, while micro base stations and macro base stations execute corresponding computing and bandwidth resource allocation tasks. The macro base station calculates the reward for each mobile device, and the time slot ends; After traversing all time slots of a training round, initialize the priority and geographic coordinates of the mobile device, and start the next training round, until all training rounds have been traversed; in, The macro base station outputs the optimal offloading decision based on the offloading decisions of all mobile devices, and sends the optimal offloading decision to the corresponding mobile devices, including: Determine whether all unloading decisions meet the energy consumption constraints. If not, use an elite-preserving genetic algorithm to adjust the unloading decisions and send the adjusted optimal unloading decision to the corresponding mobile device. If they meet the constraints, send the current unloading decision to the corresponding mobile device. The macro base station calculates the reward for each mobile device, including: The macro base station calculates the corresponding reward for each mobile device based on the information age of all mobile devices and the constraint violations of each mobile device. The calculation formula is as follows: in, Indicates mobile device exist The rewards obtained from time slots and This represents a constant used to balance the magnitudes. Indicates mobile device priority, Indicates mobile device In the time slot Information age, , and They represent time slots respectively. mobile device The penalty for exceeding the maximum tolerance thresholds in terms of energy consumption, information age, and service time is calculated using the following formula: in, This indicates a penalty imposed on the mobile device whose uninstallation decision was adjusted. For mobile devices In the time slot Offload the task to the base station The energy consumed, the base stations include macro base stations and micro base stations, Maximum power consumption limit for mobile devices in each time slot, Indicates mobile device In the time slot Information age, The maximum tolerance information age threshold, For mobile devices In the time slot Service hours, It represents the time of a time slot.

2. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age as described in claim 1, characterized in that, Each mobile device stores its experience and performs federated learning based on that experience. It then updates the local neural network parameters of each mobile device based on the global neural network parameters obtained from the federated learning, including: Each mobile device stores experience in its own experience replay pool, and when the number of experience in the experience replay pool is greater than or equal to a preset number, it randomly extracts a preset number of experience. Each mobile device updates its local neural network parameters based on extracted experience; After each preset round, each mobile device sends its current local neural network parameters to the macro base station for aggregation, and uses the aggregated global neural network parameters to update the local neural network parameters of each mobile device.

3. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age according to claim 1, characterized in that, The step of using a trained local neural network to output the offloading decision for the task corresponding to the current time slot includes: The third state information of the current time slot is input into the trained local neural network, and the offloading decision of the current time slot is output. The offloading decision of the mobile device within the coverage area of ​​the micro base station includes not offloading, offloading to the micro base station, offloading to the macro base station, and offloading the task relay to the macro base station through the micro base station. The offloading decision of the mobile device outside the coverage area of ​​the micro base station includes not offloading and offloading to the macro base station.

4. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age as described in claim 1, characterized in that, The problem of adjusting the unloading decision is transformed into a knapsack problem, which can be expressed by the following formula: in, This represents the set of all mobile devices that choose to offload tasks to the current micro base station or macro base station. This indicates the option to offload the task to the current micro base station or macro base station. The decision of whether to continue uninstalling the task on a mobile device. Indicates the unloading decision The set, Indicates uninstallation. Indicates that you will not uninstall. Indicates mobile device The reward value added after selecting uninstallation Indicates mobile device The weight of the backpack required to unload the task. This indicates the total capacity of the backpack.

5. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age according to claim 4, characterized in that, The process of adjusting the unloading decision using an elite-preserving genetic algorithm includes: Initialize a population The population It contains N chromosomes, each of which... That is, the unloading decision set ; Assess the fitness of each chromosome. and feasibility ; The chromosomes will be directly replaced by the next generation of chromosomes; Chromosome 1 will be The probability is retained in the next generation; among them, Chromosomes The fitness of; After a preset number of iterations, the chromosome with the highest fitness is selected as the optimal unloading decision.

6. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age according to claim 1, characterized in that, The macro base station calculates the computing resources and bandwidth resources allocated by the macro base station to each micro base station and each mobile device, including: Macro base stations use the Caro-Kuhn-Tucker condition to calculate the optimal computing resources allocated to each mobile device by each micro base station and macro base station; The bandwidth resources allocated to each mobile device are calculated using the interior-point method.

7. The method for offloading and resource allocation in heterogeneous edge computing networks based on information age according to claim 1, characterized in that, If mobile device If the task is successfully uninstalled, then the mobile device... In the time slot The information age is the time elapsed from the generation of a successfully uninstalled task to its uninstallation, plus the control time and the time spent transmitting and calculating the successfully uninstalled task. If mobile device If the task is not uninstalled, then the mobile device In the time slot Information age for mobile devices In the time slot The information age plus a time slot.

Images