Dynamic service migration and resource allocation method and device for edge Internet of Things

Abstract

The invention provides a dynamic service migration and resource allocation method and device for an edge Internet of Things, which can be used in the technical field of artificial intelligence, and the method comprises the steps: generating a task processing total cost according to obtained edge calculation data between an Internet of Things user and an edge server; on the basis of a preset constraint condition, a joint optimization model of dynamic service migration and resource allocation is constructed with the purpose of minimizing the total task processing cost; through a Lyapunov optimization method and a deep reinforcement learning algorithm, optimization solution is carried out on the joint optimization model, an optimal service migration and resource allocation strategy is generated, user mobility can be predicted, dynamic changes of a system environment can be sensed in real time according to the user mobility and dynamic nature of an edge Internet of Things system, and user experience is improved. The optimal service migration and resource allocation strategy is flexibly adjusted, the service migration overhead is minimized, the effective service migration and resource allocation operation is executed in advance, the service continuity and the system stability are effectively guaranteed, and frequent service migration is avoided.

Description

Technical Field

[0001] The present invention relates to the field of edge Internet of Things information technology, particularly the field of artificial intelligence technology, and more particularly to a method and device for dynamic service migration and resource allocation for edge Internet of Things. Background Art

[0002] In the field of edge IoT information technology, service migration following user migration is an effective mechanism to ensure service continuity. However, service migration in related technologies brings many challenges. For example, performing service migration will increase additional migration costs, and frequent service migration will also lead to excessive energy consumption in long-term migration, affecting system stability. When the target edge server has fewer resources than the original edge server, it is unable to allocate sufficient resources to IoT users, which will inevitably increase the calculation and communication time of tasks after service migration, and ultimately lead to invalid service migration and reduced service quality. Due to the uncertainty and suddenness of the mobility behavior of IoT users, it is impossible to accurately and effectively perform service migration and resource allocation, resulting in frequent service migration and inability to ensure service continuity. Summary of the Invention

[0003] One object of the present invention is to provide a dynamic service migration and resource allocation method for edge IoT, which can predict user mobility and, based on user mobility and the dynamic nature of the edge IoT system, perceive the dynamic changes of the system environment in real time, flexibly adjust the optimal service migration and resource allocation strategy, minimize service migration overhead, perform effective service migration and resource allocation operations in advance, effectively ensure service continuity and system stability, and avoid frequent service migration. Another object of the present invention is to provide a dynamic service migration and resource allocation device for edge IoT. Another object of the present invention is to provide a computer-readable medium. Another object of the present invention is to provide a computer device.

[0004] In order to achieve the above objectives, the present invention discloses a method for dynamic service migration and resource allocation for edge IoT, comprising:

[0005] Generate the total cost of task processing based on the edge computing data obtained between IoT users and edge servers;

[0006] Based on preset constraints and aiming to minimize the total cost of task processing, a joint optimization model for dynamic service migration and resource allocation is constructed.

[0007] The joint optimization model is optimized and solved through the Lyapunov optimization method and deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0008] Preferably, the edge computing data includes first edge computing data whose service does not follow the migration of the Internet of Things user and second edge computing data whose service follows the migration of the Internet of Things user;

[0009] Based on the edge computing data obtained between IoT users and edge servers, the total cost of task processing is generated, including:

[0010] In a preset time interval, generating a first task processing cost of the service not following the migration of the IoT user according to a preset preference weight and the first edge computing data;

[0011] Generate a second task processing cost for the service to follow the IoT user migration based on the preset preference weight and the second edge computing data within a preset time interval;

[0012] According to the preset service migration decision weight, the total task processing cost is generated according to the first task processing cost and the second task processing cost.

[0013] Preferably, the constraints include an association relationship function between IoT users and edge servers, a computing resource allocation constraint function, a communication resource allocation constraint function, a long-term task processing energy consumption constraint function, and a service migration decision weight constraint function.

[0014] Preferably, the joint optimization model is optimized and solved by the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy, including:

[0015] By using the Lyapunov optimization method, the joint optimization model is decoupled to generate an independent time slot strategy optimization model;

[0016] A Markov decision process is constructed to reconstruct the independent time slot strategy optimization model, and it is solved through a deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0017] Preferably, the joint optimization model is decoupled by the Lyapunov optimization method to generate an independent time slot strategy optimization model, including:

[0018] Define the energy queue update function based on the energy consumption queue backlog of each time slot;

[0019] According to the energy queue update function, define the Lyapunov drift function and the Lyapunov drift plus penalty term;

[0020] According to the upper bound of the Lyapunov drift function and the upper bound of the Lyapunov drift plus penalty term, the joint optimization model is decoupled to generate an independent time slot strategy optimization model.

[0021] Preferably, a Markov decision process is constructed to reconstruct the independent time slot strategy optimization model, and a deep reinforcement learning algorithm is used to solve it to generate the optimal service migration and resource allocation strategy, including:

[0022] Construct a Markov decision process, which includes the time slot state of the IoT user, the time slot action space of the IoT user, and the reward function;

[0023] The time slot action space is divided into discrete action space and continuous action space, and the policy network and Q network are defined according to the discrete action space and continuous action space;

[0024] According to the time slot status, time slot action space and reward function of IoT users, the network parameters of the policy network and the network parameters of the Q network are iteratively updated according to the preset loss function through the gradient descent algorithm until the optimal solution is obtained, generating the optimal service migration and resource allocation strategy.

[0025] The present invention also discloses a dynamic service migration and resource allocation device for edge Internet of Things, comprising:

[0026] A task total cost generating unit, configured to generate a total task processing cost based on edge computing data obtained between IoT users and edge servers;

[0027] A joint optimization model building unit is used to build a joint optimization model for dynamic service migration and resource allocation based on preset constraints and with the goal of minimizing the total cost of task processing;

[0028] The optimization solution unit is used to optimize and solve the joint optimization model through the Lyapunov optimization method and deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0029] The present invention also discloses a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0030] The present invention also discloses a computer device, including a memory and a processor, wherein the memory is used to store information including program instructions, the processor is used to control the execution of program instructions, and the processor implements the above method when executing the program.

[0031] The present invention also discloses a computer program product, comprising a computer program / instruction, which implements the above method when the computer program / instruction is executed by a processor.

[0032] The present invention generates the total cost of task processing based on the edge computing data obtained between IoT users and edge servers; based on preset constraints, with the goal of minimizing the total cost of task processing, a joint optimization model of dynamic service migration and resource allocation is constructed; through the Lyapunov optimization method and deep reinforcement learning algorithm, the joint optimization model is optimized and solved to generate the optimal service migration and resource allocation strategy, which can predict user mobility and, based on user mobility and the dynamics of the edge IoT system, perceive the dynamic changes of the system environment in real time, flexibly adjust the optimal service migration and resource allocation strategy, minimize service migration overhead, execute effective service migration and resource allocation operations in advance, effectively ensure service continuity and system stability, and avoid frequent service migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A flow chart of a method for dynamic service migration and resource allocation for edge IoT provided by an embodiment of the present invention;

[0035] Figure 2 A flowchart of a method for optimizing a joint optimization model provided by an embodiment of the present invention;

[0036] Figure 3 A flow chart of another method for dynamic service migration and resource allocation for edge IoT provided by an embodiment of the present invention;

[0037] Figure 4 A flowchart of a method for generating a first task processing cost provided by an embodiment of the present invention;

[0038] Figure 5 A flowchart of a method for generating a second task processing cost provided by an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of the structure of a dynamic service migration and resource allocation device for edge IoT provided by an embodiment of the present invention;

[0040] Figure 7 A schematic structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] It should be noted that the method and device for dynamic service migration and resource allocation for edge Internet of Things disclosed in this application can be used in the field of artificial intelligence technology, and can also be used in any field other than the field of artificial intelligence technology. The application field of the method and device for dynamic service migration and resource allocation for edge Internet of Things disclosed in this application is not limited.

[0043] To facilitate understanding of the technical solutions provided by this application, the following describes the relevant aspects of the technical solutions. Mobile edge computing, a key supporting technology for the development of the Internet of Things (IoT), effectively compensates for the high transmission latency of traditional centralized cloud servers due to their long physical distances by deploying computing and storage resources on edge servers close to users. However, each edge server has limited signal coverage. When a user moves from the coverage area of one edge server to the coverage area of another, if the user still accesses the original edge server, the significantly increased communication distance will not only lead to increased communication latency, but also significantly increase device energy consumption and even cause service interruption. Service migration changes the relationship between IoT users and edge servers. The optimal service migration and resource allocation strategy should be able to predict user mobility, perceive the dynamic changes of the system environment in real time, flexibly adjust service migration and resource allocation strategies, and preemptively execute effective service migration and resource allocation operations to ensure service continuity. This application comprehensively analyzes issues such as user mobility, user resource demand and the time-varying nature of system resources, service migration costs, and system stability, providing an efficient modeling method and optimization mechanism. In practical applications, it is possible to adjust service migration and resource allocation strategies in real time based on the dynamic changes in the edge computing system status and the surrounding environment, reducing service migration costs, maintaining the stability of the system's long-term migration energy consumption, and avoiding ineffective service migration and frequent service migration. Furthermore, the stable dynamic service migration and resource allocation method for edge IoT systems provided in this application is highly automated, easy to deploy and implement, and suitable for use in large-scale edge IoT environments.

[0044] The following uses a dynamic service migration and resource allocation device for edge IoT as an example to illustrate the implementation process of the dynamic service migration and resource allocation method for edge IoT provided by an embodiment of the present invention. It is understood that the implementation subject of the dynamic service migration and resource allocation method for edge IoT provided by an embodiment of the present invention includes, but is not limited to, the dynamic service migration and resource allocation device for edge IoT.

[0045] Figure 1 A flow chart of a dynamic service migration and resource allocation method for edge IoT provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0046] Step 101: Generate a total task processing cost based on the edge computing data obtained between the IoT user and the edge server.

[0047] In an embodiment of the present invention, the edge computing data includes first edge computing data whose service does not follow the migration of the Internet of Things user and second edge computing data whose service follows the migration of the Internet of Things user.

[0048] Specifically, during the preset time interval t, if the service does not follow the IoT user migration, the IoT user will continue to access the original edge server, and calculate the weighted sum of the task processing delay and task processing energy consumption when the service does not migrate based on the first edge computing data, and use the weighted sum as the first task processing cost when the service does not migrate.

[0049] At a preset time interval t, when the service follows the IoT user migration, the IoT user will access the target edge server, calculate the weighted sum of the task processing delay and task processing energy consumption in the service migration case based on the second edge computing data, and use the weighted sum as the second task processing cost in the service migration case.

[0050] The binary service migration decision is used as the weight value, and the weighted sum of the first task processing cost when the service is not migrated and the second task processing cost when the service is migrated is calculated, and the weighted sum is used as the total task processing cost.

[0051] Step 102: Based on preset constraints and with the goal of minimizing the total cost of task processing, a joint optimization model of dynamic service migration and resource allocation is constructed.

[0052] In an embodiment of the present invention, the constraints are set according to actual conditions, and the constraints include the association relationship function between IoT users and edge servers, the computing resource allocation constraint function, the communication resource allocation constraint function, the long-term task processing energy consumption constraint function, and the service migration decision weight constraint function.

[0053] Step 103: Optimize and solve the joint optimization model through the Lyapunov optimization method and deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0054] Figure 2 A flowchart of a method for optimizing a joint optimization model provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, step 103 specifically includes:

[0055] Step 1031: Decouple the joint optimization model using the Lyapunov optimization method to generate an independent time slot strategy optimization model.

[0056] In an embodiment of the present invention, Lyapunov optimization theory is used to decouple the original problem into an independent time slot strategy optimization problem, mainly to simplify complex dynamic resource allocation or service migration problems, making them easier to handle and solve. This method is suitable for scenarios where system stability and performance need to be maintained for a long time.

[0057] Step 1032: Construct a Markov decision process to reconstruct the independent time slot strategy optimization model, and solve it through a deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0058] In an embodiment of the present invention, a Markov decision process is constructed, and the state is defined as the user location at the next moment predicted by the long short-term memory network (LSTM) algorithm, the location of the target edge server closest to the user location at the next moment, the resources allocated to the user by the original edge server, the available resources of the target edge server, and the energy queue; the action is defined as the service migration and resource allocation decision; and the reward function is defined as the negative value of the optimization target in the independent time slot strategy optimization problem.

[0059] In the technical solution provided by the embodiment of the present invention, the total cost of task processing is generated based on the edge computing data obtained between the IoT user and the edge server; based on the preset constraints, a joint optimization model of dynamic service migration and resource allocation is constructed with the goal of minimizing the total cost of task processing; the joint optimization model is optimized and solved through the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy, which can predict user mobility and, based on user mobility and the dynamics of the edge IoT system, perceive the dynamic changes of the system environment in real time, flexibly adjust the optimal service migration and resource allocation strategy, minimize service migration overhead, execute effective service migration and resource allocation operations in advance, effectively ensure service continuity and system stability, and avoid frequent service migration.

[0060] Figure 3A flow chart of another method for dynamic service migration and resource allocation for edge IoT provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the method includes:

[0061] Step 201: Within a preset time interval, based on a preset preference weight and first edge computing data, generate a first task processing cost for a service that does not follow the migration of IoT users.

[0062] In an embodiment of the present invention, the first edge computing data includes but is not limited to the size of the IoT user input data, the data transmission rate between the IoT user and the original edge server, the data transmission power between the IoT user and the original edge server, the number of CPU cycles required to calculate 1 bit of data, and the computing resources allocated to the IoT user by the original edge server.

[0063] Figure 4 A flowchart of a method for generating a first task processing cost is provided in an embodiment of the present invention, such as Figure 4 As shown, step 201 specifically includes:

[0064] Step 2011: Within a preset time interval, based on the first edge computing data, the task transmission delay and task transmission energy consumption between the user and the original edge server are calculated when the service does not follow the migration of the IoT user.

[0065] Specifically, the calculation formula is as follows:

[0066]

[0067] Where t is the time interval, is the task transmission delay, d u,e (t) is the size of IoT user input data, r u,e (t) is the data transmission rate between IoT users and the original edge server, is the task transmission energy consumption, p u,e (t) is the data transmission power between IoT users and the original edge server.

[0068] It is worth noting that the time interval can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0069] Step 2012: Within a preset time interval, based on the first edge computing data, the task computing delay and task computing energy consumption of the original edge server are calculated when the computing service does not follow the migration of the IoT user.

[0070] Specifically, the calculation formula is as follows:

[0071]

[0072] Where t is the time interval, Calculate the delay for the task, d u,e (t) is the size of IoT user input data, c u,e (t) is the number of CPU cycles required to calculate 1 bit of data, f u,e (t) is the computing resource originally allocated by the edge server to the IoT user, Calculate the energy consumption for the task, where k is the capacitance switching coefficient.

[0073] It is worth noting that the time interval can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0074] Step 2013: Generate a first task processing cost when the service does not follow the migration of the IoT user based on the preset preference weight, the task transmission delay and energy consumption between the user and the original edge server when the service does not follow the migration of the IoT user, and the task calculation delay and energy consumption of the original edge server when the service does not follow the migration of the IoT user.

[0075] Specifically, the sum of the task transmission delay and the task calculation delay when the service does not follow the IoT user migration is calculated, and the summed value is used as the task processing delay when the service does not follow the IoT user migration, that is:

[0076]

[0077] in, Task processing delay in the case where services do not follow IoT users’ migration; Delay for task transmission; Calculates the delay for the task.

[0078] The sum of the task transmission energy consumption and the task calculation energy consumption when the computing service does not follow the migration of IoT users is used as the task processing energy consumption when the service does not follow the migration of IoT users, that is:

[0079]

[0080] in, Task processing energy consumption when the service does not follow the migration of IoT users; Energy consumption for task transmission; Calculate energy consumption for the task.

[0081] According to the preset preference weights, the weighted sum of task processing delay and energy consumption when the service does not follow the IoT user migration is calculated, and the weighted sum is used as the first task processing cost when the service does not follow the IoT user migration, that is:

[0082]

[0083] in, is the first task processing cost; u is the preference weight, Task processing delay in the case where services do not follow IoT users’ migration; The energy consumption of task processing when the service does not follow the migration of IoT users.

[0084] Step 202: Within a preset time interval, generate a second task processing cost for the service following the migration of the IoT user according to the preset preference weight and the second edge computing data.

[0085] In an embodiment of the present invention, the second edge computing data includes but is not limited to the state context data size of the IoT user, the computing power of the service function required by the IoT user, the percentage of computing resources allocated to the embedded checkpoint function by the service function, the processing intensity required for suspending the service instance and resuming the service instance by the IoT user, the data transmission rate between the IoT user and the target edge server, the data transmission power between the IoT user and the target edge server, and the computing resources allocated to the user by the target edge server.

[0086] Figure 5 A flowchart of a method for generating a second task processing cost is provided in an embodiment of the present invention, such as Figure 5 As shown, step 202 specifically includes:

[0087] Step 2021: Within a preset time interval, based on the second edge computing data, calculate the migration delay and migration energy consumption generated by migrating the service from the original edge server to the target edge server.

[0088] Specifically, the calculation formula is as follows:

[0089]

[0090] Where t is the time interval; Delayed migration; is the state context data size of IoT user u, expressed in bits; Pause the service instance for IoT user u; α u (t) the percentage of computing resources allocated to the embedded checkpoint function for the service function; The computing power required for the service function of IoT user u, expressed in CPU cycles / second; The processing intensity required to recover the service instance, expressed in CPU cycles / bit; r e,e′ (t) is the data transmission rate from the source edge server to the target edge server; is the migration energy consumption; p e,e′(t) is the data transmission power from the original edge server to the target edge server;

[0091] It is worth noting that the time interval can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0092] Step 2022: Within a preset time interval, based on the second edge computing data, the computing service follows the task transmission delay and task transmission energy consumption between the user and the target edge server when the IoT user migrates.

[0093] Specifically, the calculation formula is as follows:

[0094]

[0095] Where t is the time interval, is the task transmission delay, d u,e (t) is the size of IoT user input data, r u,e′ (t) is the data transmission rate between IoT users and the target edge server, is the task transmission energy consumption, p u,e′ (t) is the data transmission power between IoT users and the target edge server.

[0096] It is worth noting that the time interval can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0097] Step 2023: Within a preset time interval, based on the second edge computing data, the computing service follows the task computing delay and task computing energy consumption of the target edge server when the IoT user migration occurs.

[0098] Specifically, the calculation formula is as follows:

[0099]

[0100] Where t is the time interval, Calculate the delay for the task, d u,e (t) is the size of IoT user input data, c u,e (t) is the number of CPU cycles required to calculate 1 bit of data, f u,e′ (t) is the computing resources allocated to IoT users by the target edge server, Calculate the energy consumption for the task, where k is the capacitance switching coefficient.

[0101] It is worth noting that the time interval can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0102] Step 2024: Generate a second task processing cost for the service following the IoT user migration based on the preset preference weight, the migration delay and migration energy consumption generated by migrating the service from the original edge server to the target edge server, the task transmission delay and task transmission energy consumption between the user and the target edge server when the service follows the IoT user migration, and the task calculation delay and task calculation energy consumption of the target edge server when the service follows the IoT user migration.

[0103] Specifically, the sum of the migration delay, task transmission delay, and task calculation delay when the service follows the IoT user migration is calculated, and the summed value is used as the task processing delay when the service follows the IoT user migration, that is:

[0104]

[0105] in, To provide services that follow IoT users while migrating, task processing delays; Delayed migration; The task transmission delay between IoT users and target edge servers; Calculates latency for tasks destined for edge servers.

[0106] The sum of the migration energy consumption, task transmission energy consumption, and task calculation energy consumption when the service follows the migration of IoT users is calculated. The summed value is used as the task processing energy consumption when the service follows the migration of IoT users, that is:

[0107]

[0108] in, To serve the task processing energy consumption in the case of following IoT users migration; is the migration energy consumption; Energy consumption for task transmission between IoT users and target edge servers; Calculate the energy consumption for the task at the target edge server.

[0109] According to the preset preference weights, the weighted sum of task processing delay and energy consumption in the case where the service follows the IoT user migration is calculated, and the weighted sum is used as the second task processing cost of the service following the IoT user migration, that is:

[0110]

[0111] in, is the processing cost of the second task; u is the preference weight, To provide services that follow IoT users while migrating, task processing delays; Task processing energy consumption for services following IoT user migration.

[0112] Step 203 : Generate a total task processing cost according to the preset service migration decision weight, the first task processing cost, and the second task processing cost.

[0113] In the embodiment of the present invention, the value of the service migration decision weight is binary. If the service migration decision weight is 0, it means that the service does not migrate with the Internet of Things user; if the service migration decision weight is 1, it means that the service migrates according to the Internet of Things user.

[0114] Specifically, the calculation formula is as follows:

[0115]

[0116] Among them, t is the time interval, Cost u (t) is the total cost of task processing, w u (t) is the service migration decision weight, is the first task processing cost, Processing cost for the second task.

[0117] Step 204: Based on the preset constraints and with the goal of minimizing the total cost of task processing, a joint optimization model of dynamic service migration and resource allocation is constructed.

[0118] In an embodiment of the present invention, the constraint conditions include an association relationship function between IoT users and edge servers, a computing resource allocation constraint function, a communication resource allocation constraint function, a long-term task processing energy consumption constraint function, and a service migration decision weight constraint function.

[0119] Specifically, with minimizing the total cost of task processing as the optimization goal, service migration and resource allocation strategies as decision variables, and the relationship between IoT users and edge servers, computing resource allocation, communication resource allocation, long-term task processing energy consumption, and service migration decisions as constraints, a joint optimization model P1 of service migration and resource allocation is established:

[0120]

[0121] Among them, Cost u (t) is the total cost of task processing, different time slots t∈T={1,2,…,T}; C1 is the association function between IoT users and edge servers, ρ u,e' (t) is the association relationship between IoT users and edge servers, if ρ u,e' The value of (t) is 1, indicating that there is an association between the IoT user and the edge server. u,e'The value of (t) is 0, indicating that there is no association between the IoT user and the edge server; C2 is the computing resource allocation constraint function, which means that the computing resources allocated to the target edge server are less than the preset maximum available computing resources, f u,e' (t) is the computing resource allocated to IoT user u by the target edge server e’, F e' (t) is the preset maximum available computing resource; C3 is the communication resource allocation constraint function, which means that the communication resource allocated to the target edge server is less than the preset maximum available communication resource, b u,e' (t) is the communication resource allocated to IoT user u by the target edge server e’, B e' (t) is the preset maximum available communication resource; C4 is the long-term average task processing energy consumption constraint function, which means that the long-term average task processing energy consumption is less than the preset migration energy consumption threshold. is the task processing energy consumption, E avg (t) is the preset energy consumption threshold, w u (t) is the service migration decision weight, is the task processing energy consumption when the service does not follow the migration of IoT users, λ u is the preference weight, is the task processing energy consumption when the service follows the migration of IoT users; C5 is the service migration decision weight constraint function, which means that the service migration decision weight is binary. If the service migration decision weight is 0, it means that the service does not follow the migration of IoT users; if the service migration decision weight is 1, it means that the service migrates according to the IoT users.

[0122] For example, the system has E edge servers and U users, where e represents the e-th base station and u represents the u-th IoT user. At time t, when IoT user u moves from the original edge server e to the target edge server e' within the signal range, if IoT user u chooses to continue to perform service migration on the original edge server e, the first task processing cost can be calculated if user u does not migrate. If IoT user u chooses to perform service migration, the second task processing cost under the migration condition of IoT user u can be calculated as Then, calculate the total cost of task processing Finally, with the goal of minimizing the total task processing cost of all IoT users in each time slot, a joint optimization model P1 of service migration and resource allocation is constructed.

[0123] The present invention constructs a joint optimization model with the goal of minimizing the total cost of task processing and uses the following method to solve it, which can significantly reduce cost factors such as energy consumption and bandwidth usage while ensuring service quality.

[0124] Step 205: Define an energy queue update function according to the backlog of the energy consumption queue in each time slot.

[0125] In the embodiment of the present invention, the energy queue is defined as the queue backlog of energy consumption in each time slot, and the energy queue update function of the energy queue update process is given. Specifically, the energy queue Q is defined as u (t) is the energy queue of user u, and the energy queue update function is:

[0126] Q u (t+1)=max{Q u (t)-E avg (t),0}+E u (t)

[0127] Among them, Q u (t) is the energy queue of user u, Q u (t+1) is the updated energy queue of user u, E u (t) is the energy consumption of task processing, E avg (t) is the preset energy consumption threshold.

[0128] Step 206: Define a Lyapunov drift function and a Lyapunov drift plus penalty term according to the energy queue update function.

[0129] In the embodiment of the present invention, the Lyapunov function is defined as Define the Lyapunov drift function ΔL(Q u (t))=L(Q u (t+1))-L(Q u (t)), the function ΔL(Q u (t)) has an upper bound ΔL(Q(t))≤C+Q u (t)(E u (t)-E avg (t)), where Among them, L(Q u (t)) is the Lyapunov function, Q u (t) is the energy queue, ΔL(Q u (t)) is the Lyapunov drift function, Q u (t+1) is the updated energy queue, E u (t) is the energy consumption of task processing, E avg (t) is the preset energy consumption threshold, and C is the intermediate parameter.

[0130] In the embodiment of the present invention, the Lyapunov drift plus penalty term is defined as The upper bound of Lyapunov drift plus penalty term is obtained through mathematical derivation Where ΔQ u (t) is the Lyapunov drift, β is the penalty weight, Cost u (t) is the total cost of task processing, Q u (t) is the energy queue of user u, Q(t) is the set of all user energy queues, E u (t) is the energy consumption of task processing, E avg (t) is the preset energy consumption threshold, For expectation.

[0131] Step 207: Decouple the joint optimization model according to the upper bound of the Lyapunov drift function and the upper bound of the Lyapunov drift plus penalty term to generate an independent time slot strategy optimization model.

[0132] In the embodiment of the present invention, the joint optimization model P1 can be converted into the independent time slot strategy optimization model P2 by removing the constant term based on the upper bound of the Lyapunov drift function and the upper bound of the Lyapunov drift plus penalty term:

[0133]

[0134] Among them, Q u (t) is the energy queue; E u (t) is the energy consumption of task processing; β is the penalty weight; Cost u (t) is the total cost of task processing; C1 is the association function between IoT users and edge servers, ρ u,e' (t) is the association relationship between IoT users and edge servers, if ρ u,e' The value of (t) is 1, indicating that there is an association between the IoT user and the edge server. u,e' The value of (t) is 0, indicating that there is no association between the IoT user and the edge server; C2 is the computing resource allocation constraint function, which means that the computing resources allocated to the target edge server are less than the preset maximum available computing resources, f u,e' (t) is the computing resource allocated to IoT user u by the target edge server e’, F e' (t) is the preset maximum available computing resource; C3 is the communication resource allocation constraint function, which means that the communication resource allocated to the target edge server is less than the preset maximum available communication resource, b u,e' (t) is the communication resource allocated to IoT user u by the target edge server e’, B e' (t) is the preset maximum available communication resource; C5 is the service migration decision weight constraint function, indicating that the service migration decision weight is binary, w u(t) is the service migration decision weight. If the service migration decision weight is 0, it means that the service does not migrate with the IoT user; if the service migration decision weight is 1, it means that the service migrates according to the IoT user.

[0135] The present invention can transform a complex long-term optimization problem into a series of simpler optimization problems within a single time slot through the Lyapunov optimization method, thereby reducing the difficulty of solving the problem.

[0136] Step 208: Construct a Markov decision process, where the Markov decision process includes the time slot state of the IoT user, the time slot action space of the IoT user, and the reward function.

[0137] In an embodiment of the present invention, a Markov decision process is constructed, and the time slot state is defined as the IoT user location at the next moment predicted by the LSTM algorithm, the target edge server location closest to the IoT user location at the next moment, the resources allocated to the IoT user by the original edge server, the available resources of the target edge server, and the energy queue; the action is defined as the service migration and resource allocation decision; and the reward function is defined as the negative value of the optimization target in the independent time slot strategy optimization problem.

[0138] Specifically, the time slot state of IoT user u at time t is defined as:

[0139]

[0140] Among them, s u (t) is the time slot status of IoT user u at time t; represents the next moment location of IoT user u predicted by the LSTM algorithm; Indicates distance The nearest target edge server location; f u,e (t) represents the computing resources allocated by the original edge server to IoT user u; b u,e (t) represents the communication resources allocated by the original edge server to IoT user u; F e′ (t) represents the available computing resources of the target edge server; B e′ (t) represents the available communication resources of the target edge server; Q u (t) is the energy queue.

[0141] Specifically, the time slot action space of IoT user u at time t is defined as:

[0142]

[0143] Among them, a u (t) is the time slot action space of IoT user u at time t; w u (t) is the service migration decision weight, fu,e′ (t) represents the computing resources allocated by the target edge server to IoT user u; b u,e′ (t) represents the communication resources allocated by the target edge server to IoT user u.

[0144] Specifically, the reward function is defined as:

[0145]

[0146] Among them, r(t) is the reward function; For expectation; E u (t) is the energy consumption of task processing; Q u (t) is the energy queue; β is the penalty weight; Cost u (t) is the total cost of task processing.

[0147] Step 209: Divide the time slot action space to generate a discrete action space and a continuous action space, and define a policy network and a Q network according to the discrete action space and the continuous action space.

[0148] In the reinforcement learning algorithm (Parametrized Deep Q-Network, or PDQN) of this invention, the core concept is to efficiently handle discrete-continuous mixed action space problems by parameterizing continuous actions, combining discrete action selection with continuous action parameter optimization. It utilizes a policy network (x-network) to generate the probability distribution of discrete actions and the parameters of continuous actions, while simultaneously employing a Q-network (Q-network) to estimate the expected reward of taking a discrete action in a specific state using specific continuous action parameters.

[0149] In the embodiment of the present invention, the x-network is initialized Q-networkQ(s,(k,m k ); θ), target network and its update frequency Learning rate {α t ,β t} t≥0 ,, small batch sample size N, experience replay buffer R and its size B, user location l u and training step Z.

[0150] The time slot action space is divided into two parts, and k is used to represent the discrete action space {w u (t)}, use m k Represents the continuous action space {f u,e′ (t),b u,e′ (t)}, the divided action space is:

[0151]

[0152] Will Approximately a deterministic network function And named x-network; and directly map the state into continuous deterministic actions through neural network, Approximately And named Q-network, the Bellman equation can be expressed as the following formula:

[0153]

[0154] Among them, s t is the time slot status at time t; k t is the discrete action space at time t; is the continuous action space at time t; r t is the reward at time t; For expectation; t is the discount factor; s t+1 is the time slot status at time t+1; is the x-network at time t+1.

[0155] Step 210: Based on the time slot status of the IoT user, the time slot action space of the IoT user, and the reward function, the network parameters of the policy network and the network parameters of the Q network are iteratively updated using a gradient descent algorithm according to a preset loss function until a converged optimal solution is obtained, thereby generating an optimal service migration and resource allocation strategy.

[0156] Specifically, the time slot state of the IoT user at time t, the time slot action space of the IoT user and the reward function are: (s t ,a t ,s t+1 ,r t ) is stored in the experience replay buffer and randomly sampled according to the priority experience replay technique. The loss functions corresponding to the Q-network and the x-network are calculated according to the following formulas:

[0157]

[0158] in, is the loss function corresponding to the Q-network; is the Q-network parameter; L ζ (θ) is the loss function corresponding to the x-network; θ is the x-network parameter; y i is the target Q value; s i is the i-th time slot state; is the time slot state s applied to the x-network i and x-network parameter θ for action type k i Specific action selection; iThe reward for the i-th time slot; γ i is the discount factor of the i-th time slot; is applied to the target network according to the time slot state s i+1 and the target network (x-network) parameters θ - The obtained action type k i+1 Specific action selection; are the target network (Q-network) parameters.

[0159] The parameters of Q-network and x-network are adjusted by gradient descent method. and θ t The update is performed, and the iterative solution step is re-executed based on the service migration and resource allocation strategy obtained after the updated parameters until the optimal solution is obtained. The iteration is stopped and the optimal service migration and resource allocation strategy is obtained.

[0160] This application uses the Lyapunov optimization method to dynamically monitor system status by constructing virtual queues and achieves long-term optimization by minimizing drift and penalty. This approach helps ensure system stability, especially when facing dynamically changing task loads. Deep reinforcement learning provides a method for automatically learning optimal policies, reducing the workload of manually designing complex rules. Combined with deep reinforcement learning algorithms, it can effectively explore the environment and learn optimal policies, further improving system performance. This combined approach allows the system to quickly respond to real-time changes in data streams and service requests, enhancing its adaptability to dynamic environments. Service migration and resource allocation strategies can be flexibly adjusted to accommodate changes in user demand and fluctuations in network conditions. These technologies enable more rational allocation of computing and storage resources, avoiding resource waste or overuse and improving resource utilization. This combination not only improves the efficiency and effectiveness of service migration and resource allocation in edge computing environments but also provides a solid technical foundation for addressing various future challenges, such as supporting large-scale access to edge Internet of Things (IoT) devices, ensuring service quality, and reducing operating costs. This makes the entire system more robust, flexible, and efficient.

[0161] It is worth noting that the acquisition, storage, use, and processing of data in the technical solutions of this application are in compliance with the relevant provisions of laws and regulations. The user information in the embodiments of this application is obtained through legal and compliant channels, and the acquisition, storage, use, and processing of user information are authorized and agreed by the customer.

[0162] It is worth noting that the information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0163] It is worth noting that the technical solution provided in this application provides users with corresponding operation entrances for them to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0164] In the technical solution of the dynamic service migration and resource allocation method for edge Internet of Things provided by the embodiment of the present invention, the total cost of task processing is generated according to the edge computing data obtained between the Internet of Things user and the edge server; based on the preset constraints, a joint optimization model of dynamic service migration and resource allocation is constructed with the goal of minimizing the total cost of task processing; the joint optimization model is optimized and solved through the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy, which can predict user mobility and perceive the dynamic changes of the system environment in real time according to the user mobility and the dynamic nature of the edge Internet of Things system, flexibly adjust the optimal service migration and resource allocation strategy, minimize the service migration overhead, execute effective service migration and resource allocation operations in advance, effectively ensure service continuity and system stability, and avoid frequent service migration.

[0165] Figure 6 A schematic diagram of the structure of a dynamic service migration and resource allocation device for edge IoT provided by an embodiment of the present invention, which is used to execute the above-mentioned dynamic service migration and resource allocation method for edge IoT, such as Figure 6 As shown, the device includes: a task total cost generating unit 11, a joint optimization model building unit 12 and an optimization solving unit 13.

[0166] The task total cost generation unit 11 is used to generate the total cost of task processing based on the edge computing data obtained between the IoT user and the edge server.

[0167] The joint optimization model building unit 12 is used to build a joint optimization model of dynamic service migration and resource allocation based on preset constraints and with the goal of minimizing the total cost of task processing.

[0168] The optimization solution unit 13 is used to optimize and solve the joint optimization model through the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0169] In an embodiment of the present invention, the edge computing data includes first edge computing data for services that do not follow the migration of IoT users and second edge computing data for services that follow the migration of IoT users; the total task cost generation unit 11 is specifically used to generate a first task processing cost for services that do not follow the migration of IoT users according to a preset preference weight and the first edge computing data within a preset time interval; generate a second task processing cost for services that follow the migration of IoT users according to the preset preference weight and the second edge computing data within a preset time interval; and generate a total task processing cost according to the first task processing cost and the second task processing cost according to the preset service migration decision weight.

[0170] In an embodiment of the present invention, the constraint conditions include an association relationship function between IoT users and edge servers, a computing resource allocation constraint function, a communication resource allocation constraint function, a long-term task processing energy consumption constraint function, and a service migration decision weight constraint function.

[0171] In an embodiment of the present invention, the optimization solution unit 13 is specifically used to decouple the joint optimization model through the Lyapunov optimization method to generate an independent time slot strategy optimization model; construct a Markov decision process to reconstruct the independent time slot strategy optimization model, and solve it through a deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

[0172] In an embodiment of the present invention, the optimization solution unit 13 is specifically used to define an energy queue update function based on the backlog of the energy consumption queue in each time slot; define a Lyapunov drift function and a Lyapunov drift plus penalty term based on the energy queue update function; and decouple the joint optimization model based on the upper bound of the Lyapunov drift function and the upper bound of the Lyapunov drift plus penalty term to generate an independent time slot strategy optimization model.

[0173] In an embodiment of the present invention, the optimization solution unit 13 is specifically used to construct a Markov decision process, which includes the time slot state of the IoT user, the time slot action space of the IoT user, and the reward function; the time slot action space is divided to generate a discrete action space and a continuous action space, and the policy network and the Q network are defined according to the discrete action space and the continuous action space; according to the time slot state of the IoT user, the time slot action space of the IoT user, and the reward function, the network parameters of the policy network and the network parameters of the Q network are iteratively updated according to the preset loss function through the gradient descent algorithm until a converged optimal solution is obtained, thereby generating an optimal service migration and resource allocation strategy.

[0174] In the solution of the embodiment of the present invention, the total cost of task processing is generated based on the edge computing data obtained between the IoT user and the edge server; based on the preset constraints, a joint optimization model of dynamic service migration and resource allocation is constructed with the goal of minimizing the total cost of task processing; the joint optimization model is optimized and solved through the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy, which can predict user mobility and, based on user mobility and the dynamics of the edge IoT system, perceive the dynamic changes of the system environment in real time, flexibly adjust the optimal service migration and resource allocation strategy, minimize service migration overhead, execute effective service migration and resource allocation operations in advance, effectively ensure service continuity and system stability, and avoid frequent service migration.

[0175] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer device. Specifically, the computer device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176] An embodiment of the present invention provides a computer device including a memory and a processor, the memory being used to store information including program instructions, the processor being used to control the execution of the program instructions, and the program instructions being loaded and executed by the processor to implement the various steps of the embodiment of the above-mentioned dynamic service migration and resource allocation method for edge Internet of Things. For a specific description, please refer to the embodiment of the above-mentioned dynamic service migration and resource allocation method for edge Internet of Things.

[0177] Reference below Figure 7 , which shows a structural diagram of a computer device 600 suitable for implementing an embodiment of the present application.

[0178] like Figure 7 As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the computer device 600 are also stored in the RAM 603. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0179] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including devices such as a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed in the storage section 608 as needed.

[0180] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication portion 609 and / or installed from removable media 611.

[0181] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0182] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0183] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0186] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0187] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0188] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0189] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0191] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0192] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A dynamic service migration and resource allocation method for edge IoT, characterized in that: The method comprises: Generate the total cost of task processing based on the edge computing data obtained between IoT users and edge servers; Based on preset constraints, with the goal of minimizing the total cost of task processing, a joint optimization model of dynamic service migration and resource allocation is constructed; The joint optimization model is optimized and solved through the Lyapunov optimization method and deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

2. The method for dynamic service migration and resource allocation for edge IoT according to claim 1, characterized in that: The edge computing data includes first edge computing data where the service does not follow the migration of the Internet of Things user and second edge computing data where the service follows the migration of the Internet of Things user; The total cost of task processing is generated based on the edge computing data obtained between the IoT user and the edge server, including: Generate a first task processing cost for a service that does not follow the migration of the IoT user according to a preset preference weight and the first edge computing data within a preset time interval; Generate a second task processing cost for serving the follow-up IoT user migration based on a preset preference weight and the second edge computing data within a preset time interval; The total task processing cost is generated according to the preset service migration decision weight and the first task processing cost and the second task processing cost.

3. The method for dynamic service migration and resource allocation for edge IoT according to claim 1, characterized in that: The constraints include an association relationship function between IoT users and edge servers, a computing resource allocation constraint function, a communication resource allocation constraint function, a long-term task processing energy consumption constraint function, and a service migration decision weight constraint function.

4. The method for dynamic service migration and resource allocation for edge IoT according to claim 1, characterized in that: The joint optimization model is optimized and solved by the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy, including: Decoupling the joint optimization model through the Lyapunov optimization method to generate an independent time slot strategy optimization model; A Markov decision process is constructed to reconstruct the independent time slot strategy optimization model, and a deep reinforcement learning algorithm is used to solve it to generate the optimal service migration and resource allocation strategy.

5. The method for dynamic service migration and resource allocation for edge IoT according to claim 4, characterized in that: The Lyapunov optimization method is used to decouple the joint optimization model to generate an independent time slot strategy optimization model, including: Define the energy queue update function based on the energy consumption queue backlog of each time slot; According to the energy queue update function, a Lyapunov drift function and a Lyapunov drift plus penalty term are defined; According to the upper bound of the Lyapunov drift function and the upper bound of the Lyapunov drift plus penalty term, the joint optimization model is decoupled to generate the independent time slot strategy optimization model.

6. The method for dynamic service migration and resource allocation for edge IoT according to claim 4, characterized in that: The Markov decision process is constructed to reconstruct the independent time slot strategy optimization model and solve it through a deep reinforcement learning algorithm to generate an optimal service migration and resource allocation strategy, including: Constructing a Markov decision process, wherein the Markov decision process includes a time slot state of an IoT user, a time slot action space of the IoT user, and a reward function; Dividing the time slot action space to generate a discrete action space and a continuous action space, and defining a policy network and a Q network according to the discrete action space and the continuous action space; According to the time slot status of the IoT user, the time slot action space of the IoT user and the reward function, the network parameters of the policy network and the network parameters of the Q network are iteratively updated according to the preset loss function through the gradient descent algorithm until a converged optimal solution is obtained, thereby generating the optimal service migration and resource allocation strategy.

7. A dynamic service migration and resource allocation device for edge IoT, characterized in that: The device comprises: A task total cost generating unit, configured to generate a total task processing cost based on edge computing data obtained between IoT users and edge servers; A joint optimization model building unit, configured to build a joint optimization model of dynamic service migration and resource allocation based on preset constraints and with the goal of minimizing the total cost of task processing; The optimization solution unit is used to optimize and solve the joint optimization model through the Lyapunov optimization method and the deep reinforcement learning algorithm to generate the optimal service migration and resource allocation strategy.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the dynamic service migration and resource allocation method for edge Internet of Things described in any one of claims 1 to 6 is implemented.

9. A computer device comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, characterized in that: When the program instructions are loaded and executed by the processor, the dynamic service migration and resource allocation method for edge Internet of Things described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the dynamic service migration and resource allocation method for edge Internet of Things described in any one of claims 1 to 6 is implemented.

Images