Edge computing network optimization method and system based on task offloading
By defining the state space and behavior space, setting reward functions and experience playback pools, and combining multi-objective optimization algorithms, we optimize task offload decisions for edge computing networks, solving the problem of performance optimization of edge computing networks in the existing technology and achieving improvement in task offload efficiency and maximize resource utilization.
Patent Information
- Application Number
- CN202410276910.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-12
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-03-12
AI Technical Summary
The prior art cannot optimize the performance of edge computing networks, resulting in low task offload efficiency and ineffective management of latency and energy consumption.
By obtaining information from terminal devices and edge computing servers, defining state space and behavioral space, setting reward functions, using experience playback pools to randomly sample and update network parameters, and implementing server switching through multi-objective optimization algorithms, optimizing task offload decisions to minimize Switch time and maximize resource utilization.
It improves the overall efficiency of edge computing task offloading, enhances the stability and generalization capabilities of the algorithm, realizes task offload decision optimization in dynamic network environments, reduces operation and maintenance costs, and improves user satisfaction.
Smart Images

Figure CN118118927B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to an edge computing network optimization method and system based on task offloading. Background Art
[0002] In the prior art, CN114564304A discloses a method for offloading tasks of edge computing, which constructs a network model including several mobile devices and several MEC servers; calculates the total offloading delay of the current task according to the network model and a pre-constructed delay calculation model; calculates the transmission energy consumption of the current task according to the network model and a pre-constructed energy consumption calculation model; constructs scenario satisfaction models for different application scenarios according to the total offloading delay and the transmission energy consumption; constructs an objective function of the total cost of offloading the current task according to the scenario satisfaction model, the total offloading delay and the transmission energy consumption, and in combination with a penalty function; optimizes the objective function according to a particle swarm algorithm to solve the optimal position for offloading the task, and completes the offloading of the current task according to the optimal position.
[0003] In summary, although the existing technology can realize task offloading based on edge computing, it cannot optimize the performance of the edge computing network. Therefore, a solution is needed to solve the problems existing in the existing technology. Summary of the invention
[0004] The embodiments of the present invention provide an edge computing network optimization method and system based on task offloading, which can at least solve some of the problems existing in the prior art.
[0005] A first aspect of an embodiment of the present invention provides an edge computing network optimization method based on task offloading, comprising:
[0006] Obtaining information corresponding to the terminal device and the edge computing server, the terminal device sends an offloading task request to the base station through the wireless network, the base station sends the offloading task request, the terminal device and the terminal device server status to the control center and generates an offloading decision, the control center sends the offloading decision to the terminal device, and according to the offloading decision, the terminal device performs the corresponding operation. After the execution is completed, the edge computing server returns the task result, and the terminal device sends a task offloading completion signal to the edge computing server;
[0007] After receiving the task offloading completion signal, define a state space according to the state of the edge computing server and the state of the terminal device, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption according to the state space and the behavior space, and define a reward function, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling;
[0008] Based on the updated edge computing network parameters and the status of the service instances running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the user's service instance and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the server switching time and maximizing the target server resource utilization, constraints are set and solved through a multi-objective optimization algorithm to obtain an edge computing switching solution.
[0009] In an optional embodiment,
[0010] The obtaining of information corresponding to the terminal device and the edge computing server, the terminal device sending an unloading task request to the base station through the wireless network, the base station sending the unloading task request, the terminal device and the terminal device server status to the control center and generating an unloading decision, the control center sending the unloading decision to the terminal device, the terminal device performing a corresponding operation according to the unloading decision, and after the execution is completed, the edge computing server returns the task result, and the terminal device sending a task unloading completion signal to the edge computing server includes:
[0011] Obtain information corresponding to the terminal device and information corresponding to the edge computing server, establish a connection between the terminal device and the base station through a wireless network and register the terminal device in the system, and the base station assigns a unique identifier to each registered terminal device;
[0012] According to the status information of the terminal device, for each terminal device, it is determined whether it can be uninstalled. If the current terminal device can be uninstalled, the terminal device sends an uninstall task request to the base station, including a task description and the current terminal device status information. After receiving the uninstall task request, the base station sends the information corresponding to the uninstall task request and the current terminal device status information to the control center;
[0013] The control center analyzes the current load of the system and the terminal device status according to the offloading task request, generates an offloading decision according to a predefined offloading strategy, and sends the offloading decision to the corresponding terminal device via the base station. The terminal device transmits the task data to the selected edge computing server according to the offloading decision, and the edge computing server performs the computing operation and returns the computing result to the terminal device;
[0014] After completing the uninstallation operation, the terminal device sends a signal indicating that the uninstallation task is completed to the edge computing server.
[0015] In an optional embodiment,
[0016] After receiving the task offloading completion signal, defining a state space according to the state of the edge computing server and the state of the terminal device, defining a behavior space based on the target server and the offloading ratio corresponding to the task offloading, and determining the delay and energy consumption and defining a reward function according to the state space and the behavior space include:
[0017] After receiving the task offloading completion signal, the edge computing server obtains the current state of the edge server and the state of the terminal device, the network congestion level and external environmental factors, and sends the obtained information to the control center, and the control center sets the state space according to the received information;
[0018] Select a target server corresponding to the task offloading, determine the proportion of the tasks offloaded to the current target server in the total offloaded tasks for the selected target server, and define a behavior space based on the proportion;
[0019] According to the offloading task, the state space and the behavior space, combined with the communication delay, server calculation delay and data transmission delay, the task execution delay and the task execution energy consumption are calculated, and a reward function is set based on the task execution delay and the task execution energy consumption.
[0020] In an optional embodiment,
[0021] The reward function is set based on the task execution delay and the task execution energy consumption as shown in the following formula:
[0022]
[0023] Among them, R represents the reward value, w1 represents the task execution delay weight, represents the theoretical benchmark performance and is a constant. represents the total delay of task execution, represents the execution delay of the current offload task, m represents the offload decision, n represents the probability of the offload task being executed on the cloud server, w2 represents the task execution energy consumption weight, represents the total energy consumption of task execution, Indicates the energy consumption of the current offload task execution.
[0024] In an optional embodiment,
[0025] The step of setting an experience replay pool based on the reward function, repeating the calculation until a preset number of iterations is reached, performing random sampling in the experience replay pool, and updating edge computing network parameters according to data obtained by random sampling includes:
[0026] Initialize the weight parameters and network parameters in the reward function, create a computing model corresponding to the edge computing task offloading optimization environment, including state space, behavior space and reward function, and create an experience replay pool for storing experience;
[0027] For the experience replay pool, in each iteration, the feedback value of the observed environment is determined according to the unloading decision, the reward value and the next state value are obtained, and the experience obtained from each interaction with the environment is stored in the experience replay pool, and the iteration is repeated until the preset stop number is reached;
[0028] Random sampling is performed in the experience replay pool, an experience sample is selected, a loss value is calculated based on the selected experience sample, and the network parameters of the edge computing network are updated through back propagation.
[0029] In an optional embodiment,
[0030] Based on the updated edge computing network parameters, the service instance status running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the service instance of the user and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the switching server time and maximizing the resource utilization of the target server, the constraints are set and solved through a multi-objective optimization algorithm, and the edge computing switching solution is obtained, including:
[0031] Based on the updated edge computing network parameters, the running status and resource usage of the service instance are collected and recorded as status data, and the status data is packaged and transmitted to the target server;
[0032] The target server obtains the environment requirements and resource requirements corresponding to the service instance according to the status data, configures the operating environment of the target server according to the environment requirements, allocates processor and storage resources according to the resource requirements, and verifies the service environment through a simple test. If the test is successful, the server is switched;
[0033] Decompressing the received data packet on the target server, restoring the state data to the current task processing process of the service instance, starting the service instance according to the restored state data, and monitoring it;
[0034] With the goal of minimizing server switching time and maximizing target server resource utilization, resource constraints and service demand constraints are set through a multi-objective optimization algorithm, and an objective function is constructed and solved to obtain the edge computing switching solution corresponding to the current service instance.
[0035] In an optional embodiment,
[0036] The objectives are to minimize the server switching time and maximize the resource utilization of the target server, set resource constraints and service demand constraints through a multi-objective optimization algorithm, and construct an objective function as shown in the following formula:
[0037]
[0038] Among them, F SA represents the objective function value, α represents the server switching weight, T switch represents the server switching time, β represents the resource utilization weight, and U resource represents the resource utilization of the target server, T max represents the initial temperature value of the simulated annealing algorithm, t represents the current number of iterations, ε represents the random perturbation coefficient, and RN represents a random number.
[0039] A second aspect of an embodiment of the present invention provides an edge computing network optimization system based on task offloading, including:
[0040] The first unit is used to obtain information corresponding to the terminal device and the edge computing server. The terminal device sends an unloading task request to the base station through the wireless network. The base station sends the unloading task request, the terminal device and the terminal device server status to the control center and generates an unloading decision. The control center sends the unloading decision to the terminal device. According to the unloading decision, the terminal device performs a corresponding operation. After the execution is completed, the edge computing server returns a task result, and the terminal device sends a task unloading completion signal to the edge computing server;
[0041] The second unit is used to define a state space according to the state of the edge computing server and the state of the terminal device after receiving the task offloading completion signal, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption and define a reward function according to the state space and the behavior space, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling;
[0042] The third unit is used to synchronize the user-specific state context data from the current edge computing server to the target server based on the updated edge computing network parameters and the state of the service instance running on the edge computing server, initialize the service function environment in the target server, and restore the user's service instance and the corresponding edge computing task processing process in the target server, with the goal of minimizing the server switching time and maximizing the target server resource utilization, set constraints and solve them through a multi-objective optimization algorithm, and obtain an edge computing switching solution.
[0043] According to a third aspect of the embodiments of the present invention,
[0044] An electronic device is provided, comprising:
[0045] processor;
[0046] a memory for storing processor-executable instructions;
[0047] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0048] A fourth aspect of the embodiments of the present invention is:
[0049] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0050] In the present invention, a behavior space is defined by offloading target servers and offloading ratios, providing a suitable selection space for subsequent decisions; a reward function is defined based on factors such as latency and energy consumption to evaluate the quality of offloading decisions, which helps the intelligent agent learn a better offloading strategy and improve overall performance; an experience replay pool is set up based on the reward function; random sampling and data updating enhance the stability and generalization ability of the algorithm, which helps to avoid overfitting; server switching is achieved by updating edge computing network parameters, synchronizing user state context data, restoring service instances and task processing processes, so that the user's services can continue to run on the new target server, achieving the goals of minimizing switching time and maximizing target server resource utilization; in summary, the present invention achieves edge computing task offloading decision optimization in a dynamic network environment, thereby minimizing server switching time and maximizing target server resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A schematic diagram of a flow chart of an edge computing network optimization method based on task offloading according to an embodiment of the present invention;
[0052] Figure 2 It is a structural diagram of an edge computing network optimization system based on task offloading according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0055] Figure 1 FIG. 1 is a flow chart of an edge computing network optimization method based on task offloading according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0056] S1. Obtain information corresponding to the terminal device and the edge computing server, the terminal device sends an unloading task request to the base station through the wireless network, the base station sends the unloading task request, the terminal device and the terminal device server status to the control center and generates an unloading decision, the control center sends the unloading decision to the terminal device, according to the unloading decision, the terminal device performs the corresponding operation, after the execution is completed, the edge computing server returns the task result, and the terminal device sends a task unloading completion signal to the edge computing server;
[0057] The uninstall task request is a request initiated by a user or a system to uninstall a specific software, application or component, and the uninstall decision is a process in which the system decides whether to accept and execute the uninstall task request based on a series of conditions and policies.
[0058] In an optional embodiment,
[0059] The obtaining of information corresponding to the terminal device and the edge computing server, the terminal device sending an unloading task request to the base station through the wireless network, the base station sending the unloading task request, the terminal device and the terminal device server status to the control center and generating an unloading decision, the control center sending the unloading decision to the terminal device, the terminal device performing a corresponding operation according to the unloading decision, and after the execution is completed, the edge computing server returns the task result, and the terminal device sending a task unloading completion signal to the edge computing server includes:
[0060] Obtain information corresponding to the terminal device and information corresponding to the edge computing server, establish a connection between the terminal device and the base station through a wireless network and register the terminal device in the system, and the base station assigns a unique identifier to each registered terminal device;
[0061] According to the status information of the terminal device, for each terminal device, it is determined whether it can be uninstalled. If the current terminal device can be uninstalled, the terminal device sends an uninstall task request to the base station, including a task description and the current terminal device status information. After receiving the uninstall task request, the base station sends the information corresponding to the uninstall task request and the current terminal device status information to the control center;
[0062] The control center analyzes the current load of the system and the terminal device status according to the offloading task request, generates an offloading decision according to a predefined offloading strategy, and sends the offloading decision to the corresponding terminal device via the base station. The terminal device transmits the task data to the selected edge computing server according to the offloading decision, and the edge computing server performs the computing operation and returns the computing result to the terminal device;
[0063] After completing the uninstallation operation, the terminal device sends a signal indicating that the uninstallation task is completed to the edge computing server.
[0064] The current load refers to the workload or pressure borne by the system or device at a specific point in time. For a single device, the current load may involve CPU usage, memory usage, and disk read and write rates. In a distributed system, the current load refers to the load conditions of each node in the entire system or network, including the request processing capacity and response time of each service. The terminal device status refers to the operating status of the terminal device (such as personal computers, smart phones, IoT devices, etc.) at a specific point in time, including the device's hardware status (such as power, temperature, network connection status, etc.) and software status (such as running applications, current user interface, system health, etc.).
[0065] After powering on, the terminal device searches for nearby base stations through the wireless network. The terminal device selects the base station with the strongest signal to establish a connection. The terminal device sends a registration request to the connected base station. The base station assigns a unique identifier (UID) to each successfully registered terminal device. The base station stores the terminal device information and the corresponding UID to complete the registration process.
[0066] The terminal device regularly collects and updates its status information, such as CPU usage, memory usage, battery power, network status, etc. The terminal device evaluates whether it meets the offloading conditions according to the preset rules or algorithms. For example, when the device is in a low power state and high CPU load, it can consider offloading computing tasks. For terminal devices that meet the offloading conditions, the device will generate an offloading task request containing the task description and current status information. The terminal device sends the offloading task request to the connected base station via the wireless network;
[0067] The base station receives the offload task request from the terminal device and extracts the task description and device status information. The base station sends the collected information to the control center of the edge computing environment.
[0068] The control center receives information from the base station, analyzes the current load of the system and the status of the terminal device, and generates an unloading decision based on the predefined unloading strategy. The unloading decision is sent back to the corresponding terminal device through the base station. The terminal device receives the unloading decision from the control center through the base station, and transmits the task data to be unloaded to the designated edge computing server. The edge computing server receives the task data and performs the calculation operation. After the calculation is completed, the edge computing server returns the result to the terminal device.
[0069] The terminal device receives the calculation results from the edge computing server and performs corresponding processing based on the results, such as updating the interface, storing data, etc. The terminal device sends a signal to the edge computing server to confirm that the task has been successfully completed and the results have been received. The edge computing server releases the resources allocated for the task based on the task completion signal and updates the server's status information for the control center to perform subsequent resource scheduling and management.
[0070] In this embodiment, by offloading computing-intensive tasks from resource-constrained terminal devices to edge computing servers, computing resources can be allocated and utilized more reasonably to achieve system load balancing. By executing computing tasks on edge computing servers that are geographically close to terminal devices, the data transmission time in the network can be greatly reduced and network latency can be reduced. The control center can dynamically generate offloading decisions based on the current system load and the status of the terminal device, making task management and scheduling more flexible and efficient. Through intelligent task offloading, unnecessary data transmission is reduced and the use of network resources is optimized, especially in environments with poor network conditions. In summary, this embodiment can significantly improve the overall performance and user satisfaction in an edge computing environment, while reducing operation and maintenance costs, providing an efficient and reliable solution for various application scenarios.
[0071] S2. After receiving the task offloading completion signal, define a state space according to the state of the edge computing server and the state of the terminal device, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption according to the state space and the behavior space, and define a reward function, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling;
[0072] The state space refers to the set of all possible states in the environment, the behavior space is the set of all possible behaviors that the system can take in a specific state, the latency is the time required to complete an operation, the energy consumption refers to the energy consumed in performing specific operations (such as data calculation and transmission), the reward function assigns a reward (or penalty) to each behavior of the system to guide the system to learn how to maximize the total reward by optimizing its behavior, the experience replay pool is a mechanism for storing the system's experience (state, behavior, reward, new state) in the process of interacting with the environment, and the random sampling is the process of randomly selecting samples from a set of data, usually referring to randomly selecting a batch of experiences from the experience replay pool for learning.
[0073] In an optional embodiment,
[0074] After receiving the task offloading completion signal, defining a state space according to the state of the edge computing server and the state of the terminal device, defining a behavior space based on the target server and the offloading ratio corresponding to the task offloading, and determining the delay and energy consumption and defining a reward function according to the state space and the behavior space include:
[0075] After receiving the task offloading completion signal, the edge computing server obtains the current state of the edge server and the state of the terminal device, the network congestion level and external environmental factors, and sends the obtained information to the control center, and the control center sets the state space according to the received information;
[0076] Select a target server corresponding to the task offloading, determine the proportion of the tasks offloaded to the current target server in the total offloaded tasks for the selected target server, and define a behavior space based on the proportion;
[0077] According to the offloading task, the state space and the behavior space, combined with the communication delay, server calculation delay and data transmission delay, the task execution delay and the task execution energy consumption are calculated, and a reward function is set based on the task execution delay and the task execution energy consumption.
[0078] The network congestion level refers to the delay and packet loss rate encountered by the network when transmitting data, which reflects the current load status of the network. The external environmental factors include but are not limited to signal interference, device mobility, user distribution density and weather conditions. The task execution delay refers to the total time required from the start of the task to its completion, including the transmission delay of the task (if the task is offloaded), the computing delay in edge server or local processing, and the transmission delay of the result returned to the terminal device.
[0079] After the task offloading is completed, the edge computing server first obtains its current status (such as CPU usage, memory usage, number of currently processed tasks, etc.) and the status of the terminal device executing the task (such as battery power, CPU load, etc.), and collects the current network congestion level (such as bandwidth usage, latency, etc.) and external environmental factors (such as device movement speed, signal strength, etc.);
[0080] The collected edge server status, terminal device status, network congestion level and external environmental factors are sent to the control center, and the control center constructs the state space based on the received information;
[0081] Select one or more target servers from all available edge computing servers as the target servers for task offloading. For each selected target server, determine the proportion of tasks offloaded to the server in the total offloaded tasks. Define the behavior space based on the task offloading proportion, that is, the set of all actions that the reinforcement learning model can take, including which server to select for offloading and the proportion of tasks to be offloaded.
[0082] According to the offloading scheme, the total delay and energy consumption of task execution are calculated by combining the communication delay (including the delay of data transmission to the edge server and the delay of result return), server computing delay (task processing time on the server) and data transmission delay (data transmission time in the network), and the reward function is set based on the task execution delay and task execution energy consumption;
[0083] In this embodiment, by comprehensively considering information such as server status, terminal device status, network congestion level, external environmental factors, etc., the system can intelligently select appropriate target servers and offloading ratios to maximize the overall system performance. By defining appropriate state space and behavior space, the system can effectively manage and schedule computing resources to optimize the utilization of edge computing servers. By calculating task execution delay and task execution energy consumption and setting a reward function, the system can find a balance between delay and energy consumption and provide more flexible and intelligent task offloading decisions. The system has the ability to adapt to different environments and scenarios by acquiring information in real time and dynamically adjusting the state space, thereby improving the stability and adaptability of the overall system. In summary, this embodiment fully considers various factors through intelligent task offloading decisions, thereby improving the overall performance, flexibility and user satisfaction of the edge computing system.
[0084] In an optional embodiment,
[0085] The reward function is set based on the task execution delay and the task execution energy consumption as shown in the following formula:
[0086]
[0087] Among them, R represents the reward value, w1 represents the task execution delay weight, represents the theoretical benchmark performance and is a constant. represents the total delay of task execution, represents the execution delay of the current offload task, m represents the offload decision, n represents the probability of the offload task being executed on the cloud server, w2 represents the task execution energy consumption weight, represents the total energy consumption of task execution, Indicates the energy consumption of the current offload task execution.
[0088] In this embodiment, by weighing latency and energy consumption, the system tends to balance execution efficiency and resource consumption in unloading decisions, which helps to improve the sustainability and economy of the system. By adjusting the weight and exponent t, the reward function is flexible and can be adjusted according to different scenarios and needs to meet various performance and energy consumption balance requirements. The reward function is adaptable and can adjust the reward according to actual conditions to better adapt to dynamically changing network and computing environments. In summary, this embodiment provides a comprehensive performance optimization guide, so that the system can make task offloading decisions more intelligently and adaptively, thereby effectively managing resources and energy consumption while ensuring user experience.
[0089] In an optional embodiment,
[0090] The step of setting an experience replay pool based on the reward function, repeating the calculation until a preset number of iterations is reached, performing random sampling in the experience replay pool, and updating edge computing network parameters according to data obtained by random sampling includes:
[0091] Initialize the weight parameters and network parameters in the reward function, create a computing model corresponding to the edge computing task offloading optimization environment, including state space, behavior space and reward function, and create an experience replay pool for storing experience;
[0092] For the experience replay pool, in each iteration, the feedback value of the observed environment is determined according to the unloading decision, the reward value and the next state value are obtained, and the experience obtained from each interaction with the environment is stored in the experience replay pool, and the iteration is repeated until the preset stop number is reached;
[0093] Random sampling is performed in the experience replay pool, an experience sample is selected, a loss value is calculated based on the selected experience sample, and the network parameters of the edge computing network are updated through back propagation.
[0094] The weight parameters are used to adjust the reward values corresponding to different states or actions. The network parameters represent the parameters of parameterized functions (such as linear functions, nonlinear functions). The feedback value refers to the reward or punishment obtained by the system based on its behavior. The experience sample is a record of the interaction between the system and the environment, including the current state, the action taken, the reward obtained, and the next state.
[0095] Define a deep Q network whose input is the dimension of the state space and whose output is the dimension of the behavior space. By adjusting the weight parameters of the network, the network will learn the best strategy for mapping states to actions, and use an optimizer (such as Adam) to update the weights of the network to minimize the loss function, that is, the mean square error between the predicted Q value and the target Q value;
[0096] Clearly define the state of edge computing tasks, including device resource utilization, task execution time, etc., define the operations available for the system to choose, and design a reward function to quantify the quality of each action taken by the system in each state to guide learning;
[0097] Initialize the experience replay pool to store the experience generated by the interaction between the system and the environment. The experience contains the current state, the action taken, the reward obtained, the next state and the termination flag in the form of a tuple. The experience replay pool has a certain capacity, and the old experience will be replaced by the new experience to maintain the size of the pool;
[0098] In each iteration, the system interacts with the environment, selects actions and observes the feedback from the environment, including rewards, next state and other information. The experience generated by each interaction is stored in the experience replay pool for subsequent training. A batch of experiences is randomly sampled from the experience replay pool, which helps to break the correlation of the data and reduce the variance of training. A deep Q network is used to calculate the prediction and target Q value of the current Q value based on the sampled experience, and then the loss function is calculated. The loss is back-propagated through the optimizer to update the parameters of the deep Q network to better approximate the optimal strategy.
[0099] In this embodiment, through the experience replay pool, the system can learn more information from historical experience, improve the efficiency and stability of learning, help overcome sample correlation, and enable the system to better understand environmental dynamics. The use of the experience replay pool for random sampling helps to improve the generalization ability of the model, so that it can better adapt to different states and environmental changes. By calculating the loss and performing back propagation through randomly sampled experience samples, the network parameters are effectively updated, so that the system can gradually optimize its strategy and improve the accuracy of reward prediction. In summary, this embodiment enables the system to optimize task offloading decisions in a complex edge computing environment and maximize system performance and efficiency.
[0100] S3. Based on the updated edge computing network parameters and the status of the service instances running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the user's service instance and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the server switching time and maximizing the target server resource utilization, constraints are set and solved through a multi-objective optimization algorithm to obtain an edge computing switching solution.
[0101] The service instance refers to a specific instance of a specific service running in the system, the state context data refers to environmental information related to an entity or system, the service function environment refers to the environment in which the service is executed and the various components that support the service functions, including hardware devices, operating systems, runtime libraries, dependent services, etc. The edge computing switching solution refers to the switching or migration of service instances in an edge computing environment according to specific conditions or policies.
[0102] In an optional embodiment,
[0103] Based on the updated edge computing network parameters, the service instance status running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the service instance of the user and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the switching server time and maximizing the resource utilization of the target server, the constraints are set and solved through a multi-objective optimization algorithm, and the edge computing switching solution is obtained, including:
[0104] Based on the updated edge computing network parameters, the running status and resource usage of the service instance are collected and recorded as status data, and the status data is packaged and transmitted to the target server;
[0105] The target server obtains the environment requirements and resource requirements corresponding to the service instance according to the status data, configures the operating environment of the target server according to the environment requirements, allocates processor and storage resources according to the resource requirements, and verifies the service environment through a simple test. If the test is successful, the server is switched;
[0106] Decompressing the received data packet on the target server, restoring the state data to the current task processing process of the service instance, starting the service instance according to the restored state data, and monitoring it;
[0107] With the goal of minimizing server switching time and maximizing target server resource utilization, resource constraints and service demand constraints are set through a multi-objective optimization algorithm, and an objective function is constructed and solved to obtain the edge computing switching solution corresponding to the current service instance.
[0108] The storage resource refers to the hardware or software components used to store and retrieve data in a computing system. The service environment refers to the context and background in which a specific service runs, including hardware, software, and network configuration. The service instance refers to a specific instance of a specific service running in the system.
[0109] Use appropriate learning algorithms to update the parameters of the edge computing network based on the operating status and performance information of the service instances, collect status data of the running service instances, including but not limited to operating status, resource utilization, latency, etc., package the collected status data, and transmit it to the target server using appropriate communication protocols and data transmission mechanisms;
[0110] After receiving the status data, the target server unpacks and obtains the environmental and resource requirements of the service instance. According to the environmental requirements of the service instance, the target server configures the operating environment of the target server, including the operating system, runtime environment, dependent services, etc. According to the resource requirements of the service instance, the processor and storage resources are dynamically allocated to ensure that the target server can meet the performance and capacity requirements of the service instance. After the configuration and resource allocation are completed, a series of simple tests are performed to verify the availability and stability of the service environment. According to the test results, if the test is successful, it means that the target server configuration and resources meet the service instance requirements, and the server can be switched;
[0111] After receiving the data packet, the target server decompresses the data packet and restores the status data therein to the current task processing process of the service instance. According to the restored status data, the corresponding task processing process of the service instance is started, and it is ensured to work in coordination with other components of the system. After the service instance is started, a monitoring mechanism is set up to track the running status, performance indicators, resource utilization and other information of the service instance. Based on the multi-objective optimization algorithm, the constraints on the resources and service requirements required by the objective function are set, and the objective function of the multi-objective optimization is formulated. At the same time, the minimization of server switching time and the maximization of target server resource utilization are considered. The constructed objective function is solved using a multi-objective optimization algorithm (such as NSGA-II, MOEA / D, etc.) to obtain the edge computing switching plan corresponding to the current service instance. The switching plan obtained by the optimization solution is verified and evaluated. If the verification is successful, the server switching operation is executed to switch the service instance from the source server to the target server to ensure data integrity and consistency.
[0112] In this embodiment, the learning algorithm is used to update the network parameters to improve the decision-making ability of the service instance and the performance of the overall system. The availability and stability of the service environment are verified through simple tests. If the test is successful, the server switching operation is performed. This ensures a smooth transition of the service and avoids service interruption and performance degradation. The multi-objective optimization algorithm is used to construct the objective function by setting resource restrictions and service demand restrictions, which helps to comprehensively consider the minimization of time and the maximization of target server resource utilization in the server switching decision.
[0113] In an optional embodiment,
[0114] The objectives are to minimize the server switching time and maximize the resource utilization of the target server, set resource constraints and service demand constraints through a multi-objective optimization algorithm, and construct an objective function as shown in the following formula:
[0115]
[0116] Among them, F SA represents the objective function value, α represents the server switching weight, T switch represents the server switching time, β represents the resource utilization weight, and U resource represents the resource utilization of the target server, T max represents the initial temperature value of the simulated annealing algorithm, t represents the current number of iterations, ε represents the random perturbation coefficient, and RN represents a random number.
[0117] In this embodiment, by emphasizing minimizing the switching time in the server switching decision, it helps to reduce service interruptions and improve the real-time performance of the system. By emphasizing maximizing the resource utilization of the target server, it helps to improve the overall system performance and efficiency. The introduction of a simulated annealing algorithm allows local search during the search process, which helps to search the solution space more extensively and improve the global search efficiency. The introduction of random disturbances helps to avoid falling into local optimal solutions and increase the robustness of the algorithm.
[0118] Figure 2 FIG. 1 is a schematic diagram of the structure of an edge computing network optimization system based on task offloading according to an embodiment of the present invention. Figure 2 As shown, the system comprises:
[0119] The first unit is used to obtain information corresponding to the terminal device and the edge computing server. The terminal device sends an unloading task request to the base station through the wireless network. The base station sends the unloading task request, the terminal device and the terminal device server status to the control center and generates an unloading decision. The control center sends the unloading decision to the terminal device. According to the unloading decision, the terminal device performs a corresponding operation. After the execution is completed, the edge computing server returns a task result, and the terminal device sends a task unloading completion signal to the edge computing server;
[0120] The second unit is used to define a state space according to the state of the edge computing server and the state of the terminal device after receiving the task offloading completion signal, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption and define a reward function according to the state space and the behavior space, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling;
[0121] The third unit is used to synchronize the user-specific state context data from the current edge computing server to the target server based on the updated edge computing network parameters and the state of the service instance running on the edge computing server, initialize the service function environment in the target server, and restore the user's service instance and the corresponding edge computing task processing process in the target server, with the goal of minimizing the server switching time and maximizing the target server resource utilization, set constraints and solve them through a multi-objective optimization algorithm, and obtain an edge computing switching solution.
[0122] According to a third aspect of the embodiments of the present invention,
[0123] An electronic device is provided, comprising:
[0124] processor;
[0125] a memory for storing processor-executable instructions;
[0126] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0127] A fourth aspect of the embodiments of the present invention is:
[0128] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0129] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. Edge computing task offloading optimization method, It is characterized in that include: Obtaining information corresponding to the terminal device and the edge computing server, the terminal device sends an offloading task request to the base station through the wireless network, the base station sends the offloading task request, the terminal device and the terminal device server status to the control center and generates an offloading decision, the control center sends the offloading decision to the terminal device, and according to the offloading decision, the terminal device performs the corresponding operation. After the execution is completed, the edge computing server returns the task result, and the terminal device sends a task offloading completion signal to the edge computing server; After receiving the task offloading completion signal, define a state space according to the state of the edge computing server and the state of the terminal device, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption according to the state space and the behavior space, and define a reward function, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling; Based on the updated edge computing network parameters and the service instance status running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the service instance of the user and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the switching server time and maximizing the target server resource utilization, constraints are set and solved through a multi-objective optimization algorithm to obtain an edge computing switching solution; After receiving the task offloading completion signal, defining a state space according to the state of the edge computing server and the state of the terminal device, defining a behavior space based on the target server and the offloading ratio corresponding to the task offloading, and determining the delay and energy consumption and defining a reward function according to the state space and the behavior space include: After receiving the task offloading completion signal, the edge computing server obtains the current state of the edge server and the state of the terminal device, the network congestion level and external environmental factors, and sends the obtained information to the control center, and the control center sets the state space according to the received information; Select a target server corresponding to the task offloading, determine the proportion of the tasks offloaded to the current target server in the total offloaded tasks for the selected target server, and define a behavior space based on the proportion; According to the offloaded task, the state space and the behavior space, in combination with the communication delay, the server calculation delay and the data transmission delay, the task execution delay and the task execution energy consumption are calculated, and a reward function is set based on the task execution delay and the task execution energy consumption; The reward function is set based on the task execution delay and the task execution energy consumption as shown in the following formula: Among them, R represents the reward value, w 1 represents the task execution delay weight, represents the theoretical benchmark performance and is a constant. represents the total delay of task execution, represents the execution delay of the current offload task, m represents the offload decision, n represents the probability of the offload task being executed on the cloud server, and w 2 represents the energy consumption weight of task execution, represents the total energy consumption of task execution, Indicates the energy consumption of the current offload task execution; Based on the updated edge computing network parameters, the service instance status running on the edge computing server, the user-specific status context data is synchronized from the current edge computing server to the target server, the service function environment in the target server is initialized, and the service instance of the user and the corresponding edge computing task processing process are restored in the target server. With the goal of minimizing the switching server time and maximizing the resource utilization of the target server, the constraints are set and solved through a multi-objective optimization algorithm, and the edge computing switching solution is obtained, including: Based on the updated edge computing network parameters, the running status and resource usage of the service instance are collected and recorded as status data, and the status data is packaged and transmitted to the target server; The target server obtains the environment requirements and resource requirements corresponding to the service instance according to the status data, configures the operating environment of the target server according to the environment requirements, allocates processor and storage resources according to the resource requirements, and verifies the service environment through a simple test. If the test is successful, the server is switched; Decompressing the received data packet on the target server, restoring the state data to the current task processing process of the service instance, starting the service instance according to the restored state data, and monitoring it; With the goal of minimizing server switching time and maximizing target server resource utilization, resource constraints and service demand constraints are set through a multi-objective optimization algorithm, and an objective function is constructed and solved to obtain an edge computing switching solution corresponding to the current service instance; The objectives are to minimize the server switching time and maximize the resource utilization of the target server, set resource constraints and service demand constraints through a multi-objective optimization algorithm, and construct an objective function as shown in the following formula: Among them, F SA represents the objective function value, α represents the server switching weight, T switch represents the server switching time, β represents the resource utilization weight, and U resource represents the resource utilization of the target server, T max represents the initial temperature value of the simulated annealing algorithm, t represents the current number of iterations, ε represents the random perturbation coefficient, and RN represents a random number.
2. The method according to claim 1, It is characterized in that The obtaining of information corresponding to the terminal device and the edge computing server, the terminal device sending an unloading task request to the base station through the wireless network, the base station sending the unloading task request, the terminal device and the terminal device server status to the control center and generating an unloading decision, the control center sending the unloading decision to the terminal device, the terminal device performing a corresponding operation according to the unloading decision, and after the execution is completed, the edge computing server returns the task result, and the terminal device sending a task unloading completion signal to the edge computing server includes: Obtain information corresponding to the terminal device and information corresponding to the edge computing server, establish a connection between the terminal device and the base station through a wireless network and register the terminal device in the system, and the base station assigns a unique identifier to each registered terminal device; According to the status information of the terminal device, for each terminal device, it is determined whether it can be uninstalled. If the current terminal device can be uninstalled, the terminal device sends an uninstall task request to the base station, including a task description and the current terminal device status information. After receiving the uninstall task request, the base station sends the information corresponding to the uninstall task request and the current terminal device status information to the control center; The control center analyzes the current load of the system and the terminal device status according to the offloading task request, generates an offloading decision according to a predefined offloading strategy, and sends the offloading decision to the corresponding terminal device via the base station. The terminal device transmits the task data to the selected edge computing server according to the offloading decision, and the edge computing server performs the computing operation and returns the computing result to the terminal device; After completing the uninstallation operation, the terminal device sends a signal indicating that the uninstallation task is completed to the edge computing server.
3. The method according to claim 1, It is characterized in that The step of setting an experience replay pool based on the reward function, repeating the calculation until a preset number of iterations is reached, performing random sampling in the experience replay pool, and updating edge computing network parameters according to data obtained by random sampling includes: Initialize the weight parameters and network parameters in the reward function, create a computing model corresponding to the edge computing task offloading optimization environment, including state space, behavior space and reward function, and create an experience replay pool for storing experience; For the experience replay pool, in each iteration, the feedback value of the observed environment is determined according to the unloading decision, the reward value and the next state value are obtained, and the experience obtained from each interaction with the environment is stored in the experience replay pool, and the iteration is repeated until the preset stop number is reached; Random sampling is performed in the experience replay pool, an experience sample is selected, a loss value is calculated based on the selected experience sample, and the network parameters of the edge computing network are updated through back propagation.
4. An edge computing task offloading optimization system, used to implement the edge computing task offloading optimization method described in any one of claims 1 to 3. It is characterized in that include: The first unit is used to obtain information corresponding to the terminal device and the edge computing server. The terminal device sends an unloading task request to the base station through the wireless network. The base station sends the unloading task request, the terminal device and the terminal device server status to the control center and generates an unloading decision. The control center sends the unloading decision to the terminal device. According to the unloading decision, the terminal device performs a corresponding operation. After the execution is completed, the edge computing server returns a task result, and the terminal device sends a task unloading completion signal to the edge computing server; The second unit is used to define a state space according to the state of the edge computing server and the state of the terminal device after receiving the task offloading completion signal, define a behavior space based on the target server and the offloading ratio corresponding to the task offloading, determine the delay and energy consumption and define a reward function according to the state space and the behavior space, set an experience replay pool based on the reward function, repeat the calculation until a preset number of iterations is reached, perform random sampling in the experience replay pool, and update the edge computing network parameters according to the data obtained by random sampling; The third unit is used to synchronize the user-specific state context data from the current edge computing server to the target server based on the updated edge computing network parameters and the state of the service instance running on the edge computing server, initialize the service function environment in the target server, and restore the user's service instance and the corresponding edge computing task processing process in the target server, with the goal of minimizing the server switching time and maximizing the target server resource utilization, set constraints and solve them through a multi-objective optimization algorithm, and obtain an edge computing switching solution.
5. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 3.
6. A computer-readable storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 3 is implemented.
Citation Information
Patent Citations
Unloading decision optimization method based on multi-classification and distributed reinforcement learning
CN113905049A
Edge computing task unloading method based on deep reinforcement learning
CN116668439A
