Task offloading method, system and device oriented to long-term dependence and multi-user competition

Abstract

The application relates to the field of network resource management, and provides a task offloading method, system and device facing long-term dependence and multi-user competition, which comprises the following steps: obtaining an initial population; constructing a multi-objective optimization function; generating an extreme solution; selecting the initial population according to a utility vector and mixing the initial population with the extreme solution to obtain parents; obtaining an operator set action value of a candidate operator set according to the multi-objective optimization function; selecting a target operator; obtaining offspring solutions by using the target operator and the parents; calculating a penalty degree of the offspring solutions, and iterating the initial population according to the penalty degree to obtain an iteration population; calculating an operator improvement value of the offspring solutions according to the penalty degree; obtaining an operator reward value according to the operator improvement value; constructing a reward queue through the operator reward value to obtain an iteration action value evaluation network; and iterating through the iteration population, an iteration value of the utility vector and the iteration action value evaluation network to obtain a target solution set. The application can execute task offloading according to the target solution set.

Description

Technical Field

[0001] This invention relates to the field of network resource management technology, and in particular to a method, system, and device for offloading tasks that are subject to long-term dependency and multi-user competition. Background Technology

[0002] With the rapid development of communication and IoT technologies, and the surge in the number of mobile devices, a large number of new applications with extremely high latency sensitivity have emerged. These applications are often computationally intensive and energy-intensive. Limited by the battery capacity, computing power, and heat dissipation performance of mobile devices, efficient computing cannot be performed locally, making it difficult to effectively guarantee and improve the user experience. To address this challenge, mobile edge computing, as a new technology that brings cloud computing down to the network edge, shows great promise. By offloading some or all of the terminal's tasks from remote data centers to nearby network edge servers, it effectively reduces end-to-end transmission latency and energy consumption, and improves user experience and service quality, providing a better solution to the challenges of these new applications.

[0003] However, efficiently and quickly offloading computational tasks to network edge servers is no easy task. Task offloading is essentially an extremely complex decision-making process involving resource allocation and task scheduling, and its efficiency is highly dependent on the dynamic network environment, resource heterogeneity, and accurate perception of task characteristics. In densely heterogeneous environments, designing a universally applicable and efficient computational offloading strategy must simultaneously address comprehensive challenges from four dimensions: users, tasks, time, and objectives.

[0004] Specifically, achieving low latency, low power consumption, and efficient task offloading simultaneously still faces many significant challenges. First, in multi-user scenarios, numerous terminal devices compete for limited edge computing and wireless channel resources. Any user's offloading strategy dynamically alters the state of the server queue and channel congestion, leading to reduced wireless speeds and increased queuing latency, thus impacting the performance of other users. Second, these applications typically have long-running characteristics, with task data arriving sequentially and network conditions constantly changing. This necessitates that the offloading strategy possess a certain degree of online awareness and real-time adjustment capability, rather than a one-time static decision in a static scenario. Finally, the system's optimization objectives are often multi-dimensional and conflicting. Simply optimizing latency may lead to soaring device power consumption and wasted resources, while solely focusing on energy saving fails to meet the application's real-time requirements and may degrade user performance. Therefore, addressing the multi-dimensional and comprehensive challenges of computational offloading strategies becomes even more difficult.

[0005] When dealing with conflicting objectives, the MOEA / D algorithm, while effectively handling complex problems due to strict constraints from task dependencies and server capacity, suffers from insufficient adaptability of its static genetic operators in dynamic, high-dimensional environments, limiting its solution efficiency. Unlike evolutionary computation, which primarily addresses static optimization problems, deep reinforcement learning focuses on finding optimal behavioral policies for agents in dynamic environments to address sequential decision-making problems in the state space. Therefore, unloading methods based on deep reinforcement learning have become mainstream in recent years. However, most current solutions, when handling multi-objective optimization, generally adopt a linearly weighted approach, merging multiple objectives into a single scalar reward. This simplification ignores the dynamic changes in user preferences over different times or scenarios, and fixed weights cannot accurately reflect these changes, resulting in a lack of policy flexibility and ultimately failing to meet the diverse service quality requirements in practical applications. Summary of the Invention

[0006] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a method, system, and device for task unloading in the face of long-term dependence and multi-user competition, thereby optimizing the task unloading process and minimizing energy consumption and latency during task unloading.

[0007] This invention provides a method for unloading tasks that are subject to long-term dependencies and multi-user competition, including: S1: Establish a task unloading unit, and randomly and uniformly generate the population of the task unloading unit to obtain the initial population; S2: Calculate the task completion delay and task completion energy consumption of the initial population, and construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; S3: Determine the utility vector, generate extreme solutions, select the initial population based on the utility vector and mix it with the extreme solutions to obtain the parent generation, construct the candidate operator set and the action value evaluation network, and the action value evaluation network obtains the action value of the candidate operator set based on the multi-objective optimization function; S4: Read the iteration count value of the candidate operator set, select the target operator based on the iteration count value and the action value of the operator set, and use the target operator and the parent to obtain the child solution; S5: Calculate the penalty degree of the child solution, iterate the initial population according to the penalty degree to obtain the iterative population, calculate the operator improvement value of the child solution through the penalty degree, and obtain the operator reward value according to the operator improvement value. S6: Construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value; iterate through the iterative population, utility vector iterative value, and iterative action value evaluation network to obtain the target solution set, and perform task unloading based on the target solution set.

[0008] According to the task offloading method for long-term dependencies and multi-user competition provided by the present invention, step S1 further includes: S11: Determine the number of access points, the number of devices, and the number of tasks, and establish the task offloading unit based on the number of access points, the number of devices, and the number of tasks; S12: Determine the task unloading constraints, obtain a uniform weight vector through adaptive adjustment using a hybrid uniform design, and use the uniform weight vector and task unloading constraints to randomly and uniformly generate a population for the task unloading units to obtain the initial population.

[0009] According to the task offloading method for long-term dependencies and multi-user competition provided by the present invention, step S2 further includes: S21: Calculate the communication delay, execution delay, and queuing delay of the initial population, and obtain the task completion delay through the communication delay, execution delay, and queuing delay; S22: Calculate the communication energy consumption and execution energy consumption of the initial population, and obtain the task completion energy consumption through the communication energy consumption and execution energy consumption; S23: Construct the multi-objective optimization function with the goal of minimizing the energy consumption and delay of task completion.

[0010] According to the task offloading method for long-term dependencies and multi-user competition provided by the present invention, step S3 further includes: S31: Determine the utility vector, determine the extreme weight vector, and use the extreme weight vector to generate the extreme solution; S32: In the initial population, a tournament selection is performed using the utility vector to obtain the first parent set. The second parent set is selected from the extreme solutions. The first parent set and the second parent set are combined to obtain the parent generation. S33: Construct a candidate operator set including a simulated binary crossover operator, a crossover operator, a first difference evolution operator, and a second difference evolution operator; construct the action value evaluation network; the action value evaluation network obtains the action value of each operator in the candidate operator set according to a multi-objective optimization function.

[0011] According to the task offloading method for long-term dependency and multi-user competition provided by the present invention, in step S4, an iteration count value threshold is determined. If the iteration count value is greater than the iteration count value threshold, the operator with the largest action value in the candidate operator set is selected as the target operator; otherwise, an operator is randomly selected from the candidate operator set as the target operator.

[0012] According to the task offloading method for long-term dependencies and multi-user competition provided by the present invention, step S5 further includes: S51: Calculate the penalty degree of the child solution, and replace the initial solutions in the initial population with the child solutions whose penalty degree is higher than that of the child solutions with the child solutions to obtain the iterative population; S52: Calculate the operator improvement value of the child solution using the penalty degree, and accumulate the operator improvement values ​​of the child solutions in the iterative population to obtain the operator reward value.

[0013] According to the task unloading method for long-term dependency and multi-user competition provided by the present invention, in step S6, an initial reward queue is established, the offspring solutions used to obtain the iterative population are filled into the initial reward queue, and the queue reward value is obtained through the operator reward value. The reward queue is constructed through the queue reward value and the initial reward queue.

[0014] According to the task offloading method for long-term dependency and multi-user competition provided by the present invention, in step S6, the iteration termination condition is determined, the initial population is replaced by the iterative population, the utility vector is replaced by the iterative value of the utility vector, the action value evaluation network is replaced by the iterative action value evaluation network, and steps S3 to S6 are repeated until the iteration termination condition is reached to obtain the target solution set.

[0015] This invention also provides a task offloading system for long-term dependencies and multi-user competition, comprising: Initial Population Module: Used to establish task unloading units, and to randomly and uniformly generate the population of the task unloading units to obtain the initial population; Multi-objective optimization function module: used to calculate the task completion delay and task completion energy consumption of the initial population, and to construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; Operator set action value module: used to determine utility vector, generate extreme solutions, select the initial population based on utility vector and mix it with extreme solutions to obtain the parent generation, construct candidate operator sets and action value evaluation network, and the action value evaluation network obtains the operator set action value of candidate operator sets based on multi-objective optimization function; Sub-solution module: Used to read the iteration count value of the candidate operator subset, select the target operator based on the iteration count value and the action value of the operator subset, and use the target operator and the parent to obtain the sub-solution; Operator reward value module: used to calculate the penalty degree of the sub-solution, and iterate the initial population according to the penalty degree to obtain the iterative population. The operator improvement value of the sub-solution is calculated through the penalty degree, and the operator reward value is obtained according to the operator improvement value. The target solution set module is used to construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value. Iteratively, through iterative population, utility vector iterative value, and iterative action value evaluation network, the target solution set is obtained, and task unloading is performed based on the target solution set.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the above-described task offloading methods for long-term dependencies and multi-user competition.

[0017] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention provides a task offloading method, system, and device for long-term dependencies and multi-user competition. It obtains a diverse parent solution by using a relatively balanced initial population and extreme solutions, and selects appropriate operators from a candidate operator set to generate child solutions, thereby improving the performance of the obtained child solutions. By continuously iterating the parent solution through penalty degrees and continuously iterating the population and action value evaluation network, the method can be continuously optimized during execution, ultimately obtaining a target solution set with low energy consumption and low latency, effectively improving the task offloading process.

[0018] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the task unloading method for long-term dependencies and multi-user competition provided by the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the task unloading system for long-term dependencies and multi-user competition provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of the task offloading device for long-term dependency and multi-user competition provided by the present invention.

[0023] Figure label: 100. Initial Population Module; 200. Multi-Objective Optimization Function Module; 300. Operator Set Action Value Module; 400. Sub-Solution Module; 500. Operator Reward Value Module; 600. Objective Solution Set Module; 810. Processor; 820. Communication Interface; 830. Memory; 840. Communication Bus. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0025] In the description of the embodiments of the present invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of the present invention. In addition, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] In the description of the embodiments of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention based on the specific circumstances.

[0027] In embodiments of the present invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0028] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0029] The following is combined with Figures 1 to 3 Specific embodiments of the present invention are described below. Figure 1 This is a flowchart illustrating the task unloading method for long-term dependencies and multi-user competition provided by the present invention, including: S1: Establish a task unloading unit, and randomly and uniformly generate the population of the task unloading unit to obtain the initial population; Furthermore, the objective of this stage is to establish a task unloading unit and obtain an initial population. Specifically, step S1 further includes: S11: Determine the number of access points, the number of devices, and the number of tasks, and establish the task offloading unit based on the number of access points, the number of devices, and the number of tasks; S12: Determine the task unloading constraints, obtain a uniform weight vector through adaptive adjustment using a hybrid uniform design, and use the uniform weight vector and task unloading constraints to randomly and uniformly generate a population for the task unloading units to obtain the initial population.

[0030] The specific implementation method for the above steps in this embodiment is as follows: First, a task offloading unit needs to include multiple mobile devices and multiple access points. Each mobile device runs m tasks that need to be executed sequentially. Therefore, the number of mobile devices N, the number of access points K, and the number of tasks M must first be determined to establish the task offloading unit. Within the task offloading unit, each mobile device can connect to any access point and distribute its tasks to the access point for processing via a single-carrier wireless channel and orthogonal frequency division multiple access (OFDM). After processing, the access point returns the data to the mobile device. Each task can be quantified by its data volume and the number of CPU cycles required for processing.

[0031] Next, task offloading constraints need to be defined. Here, each task must be assigned to one and only one access point within a specified time period to avoid inconsistencies caused by multiple access points processing the same task. Furthermore, each task must be offloaded to an access point. Under these constraints, an initial population satisfying the task offloading constraints is first generated to allocate tasks from each device to each access point. Then, based on the number of access points and tasks, a uniformly distributed weight vector is adaptively adjusted using a mixed uniform design to ensure a high-quality weight distribution regardless of population size. By adjusting the initial population based on the weight vector obtained from the weights, an initial population with a relatively uniform distribution of values ​​in all aspects can be obtained.

[0032] S2: Calculate the task completion delay and task completion energy consumption of the initial population, and construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; Furthermore, the objective of this stage is to construct a multi-objective optimization function based on task completion delay and task completion energy consumption. Specifically, step S2 further includes: S21: Calculate the communication delay, execution delay, and queuing delay of the initial population, and obtain the task completion delay through the communication delay, execution delay, and queuing delay; S22: Calculate the communication energy consumption and execution energy consumption of the initial population, and obtain the task completion energy consumption through the communication energy consumption and execution energy consumption; S23: Construct the multi-objective optimization function with the goal of minimizing the energy consumption and delay of task completion.

[0033] The specific implementation method for the above steps in this embodiment is as follows: First, we need to calculate the communication delay of the m-th task on the n-th device in the initial population. Execution delay and queue delay : in Let m be the allocation strategy for the m-th task on the n-th device in the initial population. The CPU frequency required to process the m-th task on the n-th device. The CPU frequency of the kth access point. Let r be the data transmission volume of the m-th task on the n-th device, and r be the communication bandwidth between the access point and the device. The communication latency is 0 when the access point already has data for the m-th task of the n-th device. Let be the amount of residual data that the k-th access point has not yet processed at time t. Let be the amount of data that the k-th access point needs to add due to the newly arrived task at time t. Let be the frequency control coefficient for the k-th access point at time t for the new task. The task completion delay can be obtained by adding the communication delay, execution delay, and queuing delay.

[0034] Next, we need to calculate the communication energy consumption of the initial population. and execution energy consumption : in, This represents the power consumed by the k-th access point when performing calculations at a frequency of GHz. This represents the power consumed during access point communication. Adding the communication energy consumption and execution energy consumption yields the task completion loss. Thus, a multi-objective optimization function can be constructed to minimize both task completion energy consumption and task completion latency, allowing for the optimization of task allocation strategies.

[0035] S3: Determine the utility vector, generate extreme solutions, select the initial population based on the utility vector and mix it with the extreme solutions to obtain the parent generation, construct the candidate operator set and the action value evaluation network, and the action value evaluation network obtains the action value of the candidate operator set based on the multi-objective optimization function; Furthermore, the objective of this stage is to obtain the parent data and construct a candidate operator set and an action value evaluation network, thereby obtaining the action value of the candidate operator set. Specifically, step S3 further includes: S31: Determine the utility vector, determine the extreme weight vector, and use the extreme weight vector to generate the extreme solution; S32: In the initial population, a tournament selection is performed using the utility vector to obtain the first parent set. The second parent set is selected from the extreme solutions. The first parent set and the second parent set are combined to obtain the parent generation. S33: Construct a candidate operator set including a simulated binary crossover operator, a crossover operator, a first difference evolution operator, and a second difference evolution operator; construct the action value evaluation network; the action value evaluation network obtains the action value of each operator in the candidate operator set according to a multi-objective optimization function.

[0036] The specific implementation method for the above steps in this embodiment is as follows: First, a utility vector needs to be set. Then, an extreme weight vector is determined, which is used to generate extreme solutions. Here, the extreme weight vector means setting the weight of a certain access point or device to be extremely high, while the weights of others are very low. The extreme solution obtained in this way is also extremely biased towards a certain access point or device, and the result is also quite extreme.

[0037] Subsequently, some populations are selected from the initial population to obtain the first parent set through a multi-round tournament selection. Here, the utility values ​​between adjacent populations participating in the tournament selection are initialized as utility vectors, which helps to filter out individuals with large crowding distances, thus selecting a more unique population as the first parent set. Furthermore, some populations need to be selected from extreme solutions to obtain the second parent set. Merging the first and second parent sets yields the parent generation.

[0038] Next, a candidate operator set needs to be constructed, including the simulated binary crossover operator, the crossover operator, the first difference evolution operator, and the second difference evolution operator. The simulated binary crossover operator is as follows: in, To simulate the offspring solution obtained by the binary crossover operator, β is the simulated binary crossover coefficient. The first parent generation selected from the parent generation. The second parent selected from the parent generation, and having: Where μ is a position parameter obtained by randomly sampling between 0 and 1. Preset parameters for the mutation.

[0039] The crossover operator is: in, The offspring solution obtained by the crossover operator. The first random sample value is obtained by randomly sampling between 0 and 1. The second random sample value is obtained by randomly sampling between 0 and 1, gen is the current iteration number, and maxgen is the preset maximum iteration number.

[0040] The first difference evolution operator is: The first differential evolution operator represents the offspring solution, and CR represents the preset first differential evolution parameters. The third parent selected from the parent generation, F is the preset second differential evolution parameter, and r is the first differential random sample value obtained by randomly sampling between 0 and 1.

[0041] The second difference evolution operator is: in, This is the offspring solution obtained by the second difference evolution operator. The fourth paternal generation selected from the paternal generation. The fifth parent generation is selected from the parent generations. Here, after obtaining the offspring solutions, the first and second difference evolution operators will perform crossover mutation on the offspring solutions. Specifically, the process is as follows: in, The offspring solution obtained by the first or second difference evolution operator. The solution is the offspring after crossover mutation. This is the upper bound for all variables in the child solution. This is the lower bound for all variables in the child solution. Let be the coefficient of variation, and we have: This yields the candidate operator set. A SARSA neural network is then constructed as the action value evaluation network. This network evaluates the action value based on the objective of the multi-objective optimization function, obtaining the action value of the child solutions obtained by each operator in the candidate operator set—that is, evaluating which child solution is more suitable for the multi-objective optimization function. In selecting the first and second parent solutions from the parent set, there is a 90% probability of selection from the parent's neighbors and a 10% probability of selection from the initial population and extreme solutions, ensuring the diversity of the parent set.

[0042] S4: Read the iteration count value of the candidate operator set, select the target operator based on the iteration count value and the action value of the operator set, and use the target operator and the parent to obtain the child solution; Furthermore, the objective of this stage is to select a target operator based on the iteration count value and the action value of the operator set, thereby obtaining the sub-solution. Specifically, in step S4, an iteration count value threshold is determined. If the iteration count value is greater than the iteration count value threshold, the operator with the highest action value in the candidate operator set is selected as the target operator; otherwise, an operator is randomly selected from the candidate operator set as the target operator.

[0043] The specific implementation method for the above steps in this embodiment is as follows: Next, the iteration count of the candidate operator set needs to be read. Here, the iteration count represents the cumulative number of times the target operator has been selected. Then, a threshold for the iteration count is determined; here, the threshold is 300. If the iteration count is greater than the threshold, the action value evaluation network has been trained successfully, and the initial population diversity is sufficient. In this case, the operator with the highest action value in the candidate operator set can be prioritized as the target operator. Otherwise, due to incomplete training of the action evaluation network or insufficient diversity in the initial population, the action value of the operator set may not objectively reflect the fitness of each operator, and it is not conducive to obtaining a highly unbiased target solution set. In this case, an operator needs to be randomly selected from the candidate operator set as the target operator. After selecting the target operator, the child solution obtained from the parent operator is used as the child solution for this iteration.

[0044] S5: Calculate the penalty degree of the child solution, iterate the initial population according to the penalty degree to obtain the iterative population, calculate the operator improvement value of the child solution through the penalty degree, and obtain the operator reward value according to the operator improvement value. Furthermore, the objective of this stage is to iterate the initial population based on the penalty degree to obtain an iterative population, and to calculate the operator reward value. Specifically, step S5 further includes: S51: Calculate the penalty degree of the child solution, and replace the initial solutions in the initial population with the child solutions whose penalty degree is higher than that of the child solutions with the child solutions to obtain the iterative population; S52: Calculate the operator improvement value of the child solution using the penalty degree, and accumulate the operator improvement values ​​of the child solutions in the iterative population to obtain the operator reward value.

[0045] The specific implementation method for the above steps in this embodiment is as follows: First, it is necessary to evaluate the fitness of the sub-solution to the task unloading process, and then calculate the penalty of the sub-solution. : Where x is the sub-solution, and P() is a penalty function used to measure the degree to which the sub-solution violates the task unloading constraint. To constrain the penalty coefficient, This represents the overall fitness value of the offspring solutions. Let be the fitness value of the sub-solution for the unloading process of the m-th task. Let be the fitness coefficient of the sub-solution for the m-th task, and max[ ] denotes taking the maximum value. Here, the fitness value is measured by the degree to which the sub-solution satisfies the multi-objective optimization function, and the fitness value for a single task is measured by the degree to which it satisfies the multi-objective optimization function for that single task.

[0046] Then, the penalty degree of each initial solution in the initial population is calculated, and the penalty degree of the offspring solution is compared with the penalty degree of the initial population. For initial solutions in the initial population with a higher penalty degree than the offspring solution, the offspring solution can be used to replace them. Multiple offspring solutions can replace multiple initial solutions, with priority given to replacing the initial solutions with higher penalty degrees. This is how the iterative population is obtained.

[0047] Subsequently, for the child solutions that successfully replace the initial solution, their operator improvement value KI needs to be calculated: in, This is the penalty for the initial solution that is replaced. The operator reward value for this iteration is obtained by summing the operator improvement values ​​for the child solutions that successfully replace the initial solution.

[0048] S6: Construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value; iterate through the iterative population, utility vector iterative value, and iterative action value evaluation network to obtain the target solution set, and perform task unloading based on the target solution set.

[0049] Furthermore, the objective of this stage is to calculate the action value of the child solution, thereby training and iterating the action value evaluation network to obtain the target solution set, and then performing task unloading based on the target solution set.

[0050] Specifically, in step S6, an initial reward queue is established, the offspring solutions used to obtain the iterative population are filled into the initial reward queue, and the queue reward value is obtained through the operator reward value. A reward queue is constructed using the queue reward value and the initial reward queue.

[0051] In step S6, the iteration termination condition is determined, the initial population is replaced by the iterative population, the utility vector is replaced by the iterative utility vector value, the action value evaluation network is replaced by the iterative action value evaluation network, and steps S3 to S6 are repeated until the iteration termination condition is reached to obtain the target solution set.

[0052] The specific implementation method for the above steps in this embodiment is as follows; First, an empty initial reward queue needs to be established. Sub-solutions that successfully replaced the initial solution (i.e., those used to obtain the iterative population) are then added to the initial reward queue. New sub-solutions are added to this queue in each iteration. The initial reward queue has a preset maximum capacity. If the number of sub-solutions exceeds this capacity, the oldest sub-solution is removed and replaced with a new one. The maximum operator reward value from each iteration is used as the queue reward value, which is added to the initial reward queue to obtain the reward queue. The reward queue can also store the weight vectors of each sub-solution, the queuing status of each access point, etc.

[0053] Subsequently, sub-solutions are extracted from the reward queue. The action reward value obtained by the extracted sub-solution through the action value function in Q-learning is calculated, and the action reward value of the sub-solution obtained in the next iteration is also obtained. The action reward value of the extracted sub-solution is then weighted and summed with the action reward value of the sub-solution obtained in the next iteration to obtain the current action reward value. The difference between the current action reward value and the action value of the operator set used to obtain the target operator of the extracted sub-solution is then obtained to obtain the loss value of the action value evaluation network. The action value evaluation network is adjusted with the goal of minimizing the loss value of the action value evaluation network, and the adjustment intensity is determined by the queue reward value. The larger the queue reward value, the stronger the adjustment. In this way, the action value evaluation network can be trained to obtain an iterative action value evaluation network.

[0054] In addition, the utility vector iterative value needs to be calculated every 50 iterations. : in, For utility vectors, The cumulative improvement is calculated as follows: in, This is the penalty for the sub-solutions of the current generation's sub-solutions in the previous 50 iterations. If the number of iterations since the last calculation of the utility vector iteration value is less than 50, then the original utility vector is used as the utility vector iteration value.

[0055] Finally, the initial population is replaced by an iterative population, the utility vector is replaced by the iterative utility vector value, and the action value evaluation network is replaced by an iterative action value evaluation network. Steps S3 to S6 are repeated to obtain the sub-solutions for the next iteration. Furthermore, the iteration termination condition needs to be determined. Here, a complete work cycle of the task unloading unit is divided into several time slots, and the tasks to be run by each device in each time slot are fixed. The sub-solutions with the highest fitness values ​​obtained in each iteration are then collected as the target solution set. The number of iterations is then equal to the number of time slots. When the number of iterations reaches the number of time slots, the iteration termination condition is met, and the iteration terminates. Next, the sub-solution with the highest fitness value obtained in the first iteration of the target solution set is used in the first time slot, the sub-solution with the highest fitness value obtained in the second iteration is used in the second time slot, and so on, until all time slots have a corresponding sub-solution with the highest fitness value. Task unloading is then performed based on the sub-solution with the highest fitness value.

[0056] Here, the effectiveness of the task offloading method for long-term dependencies and multi-user competition provided by this invention is also verified. Three existing task offloading planning methods are selected for comparison with this method, and the latency and energy consumption of task execution during the task offloading process are calculated. Energy consumption is measured in joules, latency in milliseconds, and the number of devices participating in task offloading is 60, with each device handling 5 tasks. The test results are shown in Table 1. Table 1. Comparison of task unloading test results between this method and other methods.

[0057] Among them, MOEA / D-DYTS, MOEA / D, and MOEA / D-psl are all existing methods. The bolded value indicates that the value has achieved the best performance among all values. It can be seen that the method provided by the present invention can provide the best performance in terms of energy consumption and delay.

[0058] The following describes the task unloading device for long-term dependencies and multi-user competition provided by the present invention. The task unloading device for long-term dependencies and multi-user competition described below can be referred to in correspondence with the task unloading method for long-term dependencies and multi-user competition described above.

[0059] Figure 2 This example illustrates the architecture of a task offloading system that addresses long-term dependencies and multi-user competition, such as... Figure 2 As shown, the task unloading method for handling long-term dependencies and multi-user competition as described above includes: Initial Population Module 100: Used to establish task unloading units, and to randomly and uniformly generate the population of the task unloading units to obtain the initial population; Multi-objective optimization function module 200: used to calculate the task completion delay and task completion energy consumption of the initial population, and to construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; Operator set action value module 300: used to determine the utility vector, generate extreme solutions, select the initial population based on the utility vector and mix it with the extreme solutions to obtain the parent generation, construct candidate operator sets and action value evaluation network, and the action value evaluation network obtains the operator set action value of the candidate operator set based on the multi-objective optimization function; Sub-solution module 400: Used to read the iteration count value of the candidate operator set, select the target operator based on the iteration count value and the action value of the operator set, and use the target operator and the parent to obtain the sub-solution; Operator reward module 500: It is used to calculate the penalty degree of the sub-solution, and iterate the initial population according to the penalty degree to obtain the iterative population. The operator improvement value of the sub-solution is calculated through the penalty degree, and the operator reward value is obtained according to the operator improvement value. The target solution set module 600 is used to construct a reward queue through operator reward values, train the action value evaluation network through the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value; iteratively obtain the target solution set through iterative population, utility vector iterative value, and iterative action value evaluation network, and perform task unloading based on the target solution set.

[0060] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call a computer program in the memory 830 to execute a task offloading method oriented towards long-term dependencies and multi-user contention, the method including: S1: Establish a task unloading unit, and randomly and uniformly generate the population of the task unloading unit to obtain the initial population; S2: Calculate the task completion delay and task completion energy consumption of the initial population, and construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; S3: Determine the utility vector, generate extreme solutions, select the initial population based on the utility vector and mix it with the extreme solutions to obtain the parent generation, construct the candidate operator set and the action value evaluation network, and the action value evaluation network obtains the action value of the candidate operator set based on the multi-objective optimization function; S4: Read the iteration count value of the candidate operator set, select the target operator based on the iteration count value and the action value of the operator set, and use the target operator and the parent to obtain the child solution; S5: Calculate the penalty degree of the child solution, iterate the initial population according to the penalty degree to obtain the iterative population, calculate the operator improvement value of the child solution through the penalty degree, and obtain the operator reward value according to the operator improvement value. S6: Construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value; iterate through the iterative population, utility vector iterative value, and iterative action value evaluation network to obtain the target solution set, and perform task unloading based on the target solution set.

[0061] Furthermore, when the computer program in the aforementioned memory 830 can be implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0063] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task unloading method for long-term dependencies and multi-user competition, characterized in that, include: S1: Establish a task unloading unit, and randomly and uniformly generate the population of the task unloading unit to obtain the initial population; S2: Calculate the task completion delay and task completion energy consumption of the initial population, and construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; S3: Determine the utility vector, generate extreme solutions, select the initial population based on the utility vector and mix it with the extreme solutions to obtain the parent generation, construct the candidate operator set and the action value evaluation network, and the action value evaluation network obtains the action value of the candidate operator set based on the multi-objective optimization function; S4: Read the iteration count value of the candidate operator set, select the target operator based on the iteration count value and the action value of the operator set, and use the target operator and the parent to obtain the child solution; S5: Calculate the penalty degree of the child solution, iterate the initial population according to the penalty degree to obtain the iterative population, calculate the operator improvement value of the child solution through the penalty degree, and obtain the operator reward value according to the operator improvement value. S6: Construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value; iterate through the iterative population, utility vector iterative value, and iterative action value evaluation network to obtain the target solution set, and perform task unloading based on the target solution set.

2. The task offloading method for long-term dependencies and multi-user competition as described in claim 1, characterized in that, Step S1 further includes: S11: Determine the number of access points, the number of devices, and the number of tasks, and establish the task offloading unit based on the number of access points, the number of devices, and the number of tasks; S12: Determine the task unloading constraints, obtain a uniform weight vector through adaptive adjustment using a hybrid uniform design, and use the uniform weight vector and task unloading constraints to randomly and uniformly generate a population for the task unloading units to obtain the initial population.

3. The task unloading method for long-term dependencies and multi-user competition as described in claim 1, characterized in that, Step S2 further includes: S21: Calculate the communication delay, execution delay, and queuing delay of the initial population, and obtain the task completion delay through the communication delay, execution delay, and queuing delay; S22: Calculate the communication energy consumption and execution energy consumption of the initial population, and obtain the task completion energy consumption through the communication energy consumption and execution energy consumption; S23: Construct the multi-objective optimization function with the goal of minimizing the energy consumption and delay of task completion.

4. The task unloading method for long-term dependencies and multi-user competition according to claim 1, characterized in that, Step S3 further includes: S31: Determine the utility vector, determine the extreme weight vector, and use the extreme weight vector to generate the extreme solution; S32: In the initial population, a tournament selection is performed using the utility vector to obtain the first parent set. The second parent set is selected from the extreme solutions. The first parent set and the second parent set are combined to obtain the parent generation. S33: Construct a candidate operator set including a simulated binary crossover operator, a crossover operator, a first difference evolution operator, and a second difference evolution operator; construct the action value evaluation network; the action value evaluation network obtains the action value of each operator in the candidate operator set according to a multi-objective optimization function.

5. The task unloading method for long-term dependencies and multi-user competition according to claim 1, characterized in that, In step S4, an iteration count threshold is determined. If the iteration count is greater than the iteration count threshold, the operator with the highest action value in the candidate operator set is selected as the target operator. Otherwise, an operator is randomly selected from the candidate operator set as the target operator.

6. The task unloading method for long-term dependencies and multi-user competition according to claim 1, characterized in that, Step S5 further includes: S51: Calculate the penalty degree of the child solution, and replace the initial solutions in the initial population with the child solutions whose penalty degree is higher than that of the child solutions with the child solutions to obtain the iterative population; S52: Calculate the operator improvement value of the child solution using the penalty degree, and accumulate the operator improvement values ​​of the child solutions in the iterative population to obtain the operator reward value.

7. The task unloading method for long-term dependencies and multi-user competition according to claim 1, characterized in that, In step S6, an initial reward queue is established, and the child solutions used to obtain the iterative population are filled into the initial reward queue. The queue reward value is obtained through the operator reward value, and a reward queue is constructed using the queue reward value and the initial reward queue.

8. The task unloading method for long-term dependencies and multi-user competition according to claim 1, characterized in that, In step S6, the iteration termination condition is determined, the initial population is replaced by the iterative population, the utility vector is replaced by the iterative utility vector value, the action value evaluation network is replaced by the iterative action value evaluation network, and steps S3 to S6 are repeated until the iteration termination condition is reached to obtain the target solution set.

9. A task unloading system for long-term dependencies and multi-user competition, used to execute the task unloading method for long-term dependencies and multi-user competition as described in any one of claims 1 to 8, characterized in that, include: Initial Population Module: Used to establish task unloading units, and to randomly and uniformly generate the population of the task unloading units to obtain the initial population; Multi-objective optimization function module: used to calculate the task completion delay and task completion energy consumption of the initial population, and to construct a multi-objective optimization function based on the task completion delay and task completion energy consumption; Operator set action value module: used to determine utility vector, generate extreme solutions, select the initial population based on utility vector and mix it with extreme solutions to obtain the parent generation, construct candidate operator sets and action value evaluation network, and the action value evaluation network obtains the operator set action value of candidate operator sets based on multi-objective optimization function; Sub-solution module: Used to read the iteration count value of the candidate operator subset, select the target operator based on the iteration count value and the action value of the operator subset, and use the target operator and the parent to obtain the sub-solution; Operator reward value module: used to calculate the penalty degree of the sub-solution, and iterate the initial population according to the penalty degree to obtain the iterative population. The operator improvement value of the sub-solution is calculated through the penalty degree, and the operator reward value is obtained according to the operator improvement value. The target solution set module is used to construct a reward queue using operator reward values, train the action value evaluation network using the reward queue to obtain an iterative action value evaluation network, and calculate the utility vector iterative value. Iteratively, through iterative population, utility vector iterative value, and iterative action value evaluation network, the target solution set is obtained, and task unloading is performed based on the target solution set.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the task offloading method for long-term dependencies and multi-user competition as described in any one of claims 1 to 8.

Images