Multi-level computing power network task scheduling method and device

Abstract

This application discloses a multi-level computing power network task scheduling method and apparatus. The method includes: acquiring state information of multiple user devices, multiple edge computing nodes, and a cloud server; acquiring task information of multiple tasks to be processed; and constructing a first state vector; analyzing the first state vector using a task scheduling model to obtain a target scheduling strategy including a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy. The task scheduling model is trained using a reinforcement learning algorithm, and during training, a reward function is determined based on the average cost of a first cost (task processing locally on user devices), a second cost (task offloading to edge computing nodes), and a third cost (task offloading to cloud servers); and scheduling the multiple tasks to be processed according to the target scheduling strategy. This application solves the technical problem of the lack of efficient management and coordination solutions for a large number of tasks, computing resources, and wireless resources in edge computing scenarios.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and more specifically, to a multi-level computing power network task scheduling method and apparatus. Background Technology

[0002] Multi-layered computing networks integrate technologies such as cloud computing and edge computing. They can employ different scheduling methods for users' computationally intensive or latency-sensitive tasks, making them suitable for various business scenarios. However, while multi-layered computing networks offer significant advantages in task scheduling, they also present challenges to network management, such as heterogeneity in network, resource management, and mobility management. Emerging software-defined networks (SDNs) divide the functions of network devices into control and data planes, providing a flexible centralized network management approach that benefits edge computing and caching. Simultaneously, network function virtualization (NFV) virtualizes general computer hardware, leading to the convergence of caching, computing, and communication functions. All of these factors make the efficient management and coordination of computing and wireless resources a significant challenge for computing networks.

[0003] Currently, most computing network models study computation offloading from the perspective of a single user, while research on multi-user computation offloading remains limited. Furthermore, since most wireless networks are multi-channel, a key issue in multi-user offloading scenarios is how to achieve efficient wireless access and coordination among multiple users to reduce mutual interference caused by sharing a common set of channels. In addition, existing research on task scheduling in computing networks mostly considers static models, neglecting the changes over time such as user movement and channel fading, thus making it unsuitable for today's complex mobile communication environment.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a multi-level computing power network task scheduling method and apparatus to at least solve the technical problem of lack of efficient management and coordination solutions for a large number of tasks, computing resources and wireless resources in edge computing scenarios.

[0006] According to one aspect of the embodiments of this application, a multi-level computing power network task scheduling method is provided, comprising: acquiring first state information of multiple user devices, second state information of multiple edge computing nodes, and third state information of a cloud server in a multi-level computing power network, and acquiring task information of the tasks to be processed for each user device; constructing a first state vector based on the first state information, second state information, third state information, and task information; analyzing the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy, wherein the task scheduling model is trained using a reinforcement learning algorithm, and a reward function is determined based on the target offloading cost during training, the target offloading cost being the average cost of the first cost of processing the tasks to be processed locally by the user device, the second cost of offloading the tasks to be processed to edge computing nodes for processing, and the third cost of offloading the tasks to be processed to the cloud server for processing; and scheduling multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0007] Optionally, the first state information includes at least: the computing resources of each user device and the channel parameters between each user device and the corresponding edge computing node; the second state information includes at least: the bandwidth resources of each edge computing node, the bandwidth resource allocation status, the computing resources, and the computing resource allocation status; the third state information includes at least: the computing resources of the cloud server; and the task information includes at least: the task identifier, task type, and task size of each task to be processed.

[0008] Optionally, the training process of the task scheduling model includes: constructing an initial model, wherein the initial model includes a task offloading sub-model, a bandwidth resource allocation sub-model, and a computing resource allocation sub-model; obtaining multiple training samples, wherein each training sample includes the first historical state information of multiple user devices, historical task information, the second historical state information of multiple edge computing nodes, and the third historical state information of cloud servers within a historical task time slot of a multi-layer computing power network; determining the number of iterations and the training batch size; and inputting the multiple training samples into the initial model for iterative training based on the number of iterations and the training batch size to obtain the task scheduling model.

[0009] Optionally, multiple training samples are input into the initial model for iterative training, including: for each training sample, determining a reward function based on the first historical state information, historical task information, second historical state information, and third historical state information in the training sample, and constructing a second state vector; inputting the second state vector into the task unloading sub-model to obtain the optimal unloading strategy predicted by the task unloading sub-model based on the ε-greedy algorithm and the first Q value corresponding to the optimal unloading strategy, and determining the first maximum Q value from the Q values ​​corresponding to all unloading strategies, and determining the first loss function of the task unloading sub-model based on the first Q value, the first maximum Q value, and the reward function; constructing a third state vector based on the second historical state information of each edge computing node corresponding to the optimal unloading strategy; inputting the third state vector into the bandwidth resource allocation sub-model to obtain the bandwidth resource allocation sub-model based on the ε-greedy algorithm. The optimal bandwidth resource allocation strategy predicted by the ε-greedy algorithm and the corresponding second Q-value are obtained. The second maximum Q-value is determined from all Q-values ​​corresponding to all bandwidth resource allocation strategies. The second loss function of the bandwidth resource allocation sub-model is determined based on the second Q-value, the second maximum Q-value, and the reward function. The third state vector is input into the computational resource allocation sub-model to obtain the optimal computational resource allocation strategy predicted by the ε-greedy algorithm and the corresponding third Q-value. The third maximum Q-value is determined from all Q-values ​​corresponding to all computational resource allocation strategies. The third loss function of the computational resource allocation sub-model is determined based on the third Q-value, the third maximum Q-value, and the reward function. During iterative training, the model parameters of the initial model method are adjusted based on the first, second, and third loss functions.

[0010] Optionally, the reward function is determined based on the first historical state information, historical task information, second historical state information, and third historical state information in the training samples, including: for each historical task in the historical task information, determining the latency sensitivity coefficient and energy consumption sensitivity coefficient of the historical task; determining the first processing latency and first processing energy consumption of the historical task processed locally by the user equipment, and determining the first cost of the historical task processed locally by the user equipment based on the first processing latency, latency sensitivity coefficient, first processing energy consumption, and energy consumption sensitivity coefficient; determining the second processing latency and second processing energy consumption of offloading the historical task to the edge computing node for processing, and determining the second cost of offloading the historical task to the edge computing node for processing based on the second processing latency, latency sensitivity coefficient, second processing energy consumption, and energy consumption sensitivity coefficient; determining the third processing latency and third processing energy consumption of offloading the historical task to the cloud server for processing, and determining the third cost of offloading the historical task to the cloud server for processing based on the third processing latency, latency sensitivity coefficient, third processing energy consumption, and energy consumption sensitivity coefficient; determining the average cost of the first cost, second cost, and third cost of all historical tasks; and determining the inverse value of the average cost as the reward function.

[0011] Optionally, the second processing latency includes: the communication latency of the historical task being offloaded to the edge computing node and the computing latency of the edge computing node processing the historical task; the second processing energy consumption includes: the communication energy consumption of the historical task being offloaded to the edge computing node and the computing energy consumption of the edge computing node processing the historical task; the third processing latency includes: the communication latency of the historical task being offloaded to the cloud server and the computing latency of the cloud server processing the historical task; the third processing energy consumption includes: the communication energy consumption of the historical task being offloaded to the cloud server and the computing energy consumption of the cloud server processing the historical task.

[0012] Optionally, during the training of the task scheduling model, the model prediction results are determined to meet the following constraints: tasks cannot be split, each historical task can only be processed locally, or offloaded to an edge computing node for processing, or offloaded to a cloud server for processing; the total bandwidth resources occupied by multiple historical tasks offloaded to the same edge computing node do not exceed the available bandwidth resources of the edge computing node; and the total computing resources occupied by multiple historical tasks offloaded to the same edge computing node do not exceed the available computing resources of the edge computing node.

[0013] According to another aspect of the embodiments of this application, a multi-level computing power network task scheduling device is also provided, comprising: an acquisition module, configured to acquire first state information of multiple user devices, second state information of multiple edge computing nodes, and third state information of a cloud server in a multi-level computing power network, and acquire task information of the tasks to be processed by each user device; a construction module, configured to construct a first state vector based on the first state information, second state information, third state information, and task information; an analysis module, configured to analyze the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy, wherein the task scheduling model is trained using a reinforcement learning algorithm, and a reward function is determined based on the target offloading cost during training, the target offloading cost being the average cost of the first cost of the user device processing the task to be processed locally, the second cost of offloading the task to be processed to the edge computing node for processing, and the third cost of offloading the task to be processed to the cloud server for processing; and a scheduling module, configured to schedule multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described multi-level computing network task scheduling method by running the computer program.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described multi-level computing power network task scheduling method through the computer program.

[0016] In this embodiment, the multi-layered computing network includes multiple user devices, multiple edge computing nodes, and a cloud server, making the scenario closer to a real task scheduling environment. When training the task scheduling model, a reinforcement learning algorithm is used for iterative training. Considering issues such as channel interference in dynamic environments, the reward function is determined based on the average cost of the first cost of processing tasks locally on user devices, the second cost of offloading tasks to edge computing nodes, and the third cost of offloading tasks to cloud servers. After model training, the model can predict the optimal task scheduling strategy, including task offloading strategy, bandwidth resource allocation strategy, and computing resource allocation strategy, using only the current state of the computing network and the task state as input. This achieves efficient task scheduling and effectively solves the technical problem of lacking efficient management and coordination solutions for a large number of tasks, computing resources, and wireless resources in edge computing scenarios. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of an optional multi-level computing network architecture according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of another optional multi-level computing network architecture according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the structure of an optional computer terminal according to an embodiment of this application;

[0021] Figure 4 This is a flowchart illustrating an optional multi-level computing power network task scheduling method according to an embodiment of this application;

[0022] Figure 5 This is a flowchart illustrating an optional task scheduling model training method according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of an optional multi-level computing power network task scheduling device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:

[0027] The ε-greedy algorithm is a commonly used reinforcement learning algorithm used to select the optimal action in a finite action space, where ε is a parameter between 0 and 1 used to control the balance between exploration and exploitation. The algorithm typically involves the following steps: 1) Initialize the Q-value table, setting the Q-value of all actions to 0; 2) At each time step t, select an action based on the current state, choosing a random action with probability ε and the action with the highest current Q-value with probability 1-ε; 3) Execute the selected action and observe the feedback from the environment, including rewards and the next state; 4) Update the Q-value table using the feedback, updating the Q-value of the selected action according to the update rule; 5) Repeat steps 2-4 until a stopping condition is met (e.g., reaching the maximum number of iterations or convergence).

[0028] Example 1

[0029] According to an embodiment of this application, a multi-layered computing network is first provided, the network architecture of which is as follows: Figure 1As shown, the system comprises a device layer, an edge layer, and a cloud layer. The device layer includes n user devices 11 (1 to n), the edge layer includes m edge computing nodes 12 (1 to m), and the cloud layer includes a cloud server 13. Different user devices may be covered by overlapping edge computing nodes. Within a fixed task time slot, each user device has a task to be processed. This task can be executed locally by the user device's computing resources or offloaded to the corresponding edge computing node or cloud server for execution.

[0030] Figure 2 This diagram illustrates a more detailed architecture of a multi-layered computing network. In the User Equipment Layer (UE Layer), user equipment (UEs) can be various terminal devices such as mobile phones, tablets, computers, cameras, and printers. The Edge Layer includes a Central Scheduler and multiple edge computing nodes (also known as FogNodes). These edge computing nodes can be base stations, Wi-Fi access nodes, etc., and can provide edge computing services to handle tasks offloaded by user equipment. Edge computing nodes can also forward task data to the Central Scheduler via high-speed broadband wireless communication. The Central Scheduler forwards the task data to the core network via a wired link. The core network connects to the Cloud Computing Server in the Cloud Layer. The cloud server is modeled as having large-capacity computing resources and can handle tasks offloaded by user equipment.

[0031] Typically, when multiple tasks need to be scheduled simultaneously, the central scheduler at the edge layer determines the offloading decision for tasks, the allocation of bandwidth resources, and computing resources based on system status, historical information, and task characteristics.

[0032] Specifically, this application provides a multi-level computing power network task scheduling method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] The method embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 3 A hardware block diagram of a computer terminal (or mobile device) for implementing a multi-level computing power network task scheduling method is shown. Figure 3As shown, the computer terminal 30 (or mobile device) may include one or more processors 302 (shown as 302a, 302b, ..., 302n in the figure) (processor 302 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 30 may also include... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0034] It should be noted that the aforementioned one or more processors 302 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 30 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0035] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the multi-level computing power network task scheduling method in this embodiment of the application. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, thereby implementing the above-mentioned application vulnerability detection method. The memory 304 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 304 may further include memory remotely located relative to the processor 302, and these remote memories can be connected to the computer terminal 30 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 306 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 30. In one example, the transmission device 306 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 306 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0037] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 30 (or mobile device).

[0038] Under the above operating environment, embodiments of this application provide a multi-level computing power network task scheduling method, such as... Figure 4 As shown, the method includes the following steps:

[0039] Step S402: Obtain the first status information of multiple user devices, the second status information of multiple edge computing nodes, and the third status information of the cloud server in the multi-level computing network, and obtain the task information of the tasks to be processed for each user device.

[0040] The first state information includes at least the following: computing resources of each user device, such as CPU computing power and computer hardware parameters; channel parameters between each user device and the corresponding edge computing node, such as channel gain, transmit power, and signal-to-interference-plus-noise ratio; the second state information includes at least the following: bandwidth resources, bandwidth resource allocation status, computing resources, and computing resource allocation status of each edge computing node; the third state information includes at least the following: computing resources of the cloud server; and the task information includes at least the following: task identifier, task type, and task size of each task to be processed.

[0041] Step S404: Construct a first state vector based on the first state information, the second state information, the third state information, and the task information.

[0042] Step S406: Analyze the first state vector using the pre-trained task scheduling model to obtain the target scheduling strategy. The task scheduling model is trained using a reinforcement learning algorithm. During training, the reward function is determined based on the target offloading cost. The target offloading cost is the average cost of the first cost of processing the task locally on the user device, the second cost of offloading the task to the edge computing node for processing, and the third cost of offloading the task to the cloud server for processing.

[0043] Step S408: Schedule multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: task offloading strategy, bandwidth resource allocation strategy and computing resource allocation strategy.

[0044] The following section explains each step of the multi-level computing power network task scheduling method in conjunction with the specific implementation process.

[0045] As an optional implementation method, such as Figure 5 As shown, the task scheduling model can be trained through the following steps:

[0046] Step S502: Construct an initial model, which includes: a task offloading sub-model, a bandwidth resource allocation sub-model, and a computing resource allocation sub-model.

[0047] To avoid the curse of dimensionality caused by too many model states, this application replaces the Q-table in deep learning with a deep neural network. All three sub-models can adopt a deep neural network structure, taking the state as input and estimating the value of the action through deep neural network analysis.

[0048] Step S504: Obtain multiple training samples, wherein each training sample includes the first historical state information of multiple user devices, historical task information, the second historical state information of multiple edge computing nodes, and the third historical state information of the cloud server within a historical task time slot of the multi-layer computing power network; the specific content of the historical state information obtained here can refer to the state information content obtained in the above application process.

[0049] Step S506: Determine the number of iterations and the training batch size.

[0050] Step S508: Based on the number of iterations and the training batch size, input multiple training samples into the initial model for iterative training to obtain the task scheduling model.

[0051] Optionally, when inputting multiple training samples into the initial model for iterative training, the following steps S5081-S5086 can be used:

[0052] Step S5081: For each training sample, the reward function can be determined based on the first historical state information, historical task information, second historical state information and third historical state information in the training sample, and a second state vector can be constructed.

[0053] As an optional implementation, the reward function can be determined as follows: For each historical task in the historical task information, determine the latency sensitivity coefficient and energy consumption sensitivity coefficient of the historical task; determine the first processing latency and first processing energy consumption of the historical task processed locally by the user equipment, and determine the first cost of processing the historical task locally by the user equipment based on the first processing latency, latency sensitivity coefficient, first processing energy consumption and energy consumption sensitivity coefficient; determine the second processing latency and second processing energy consumption of offloading the historical task to the edge computing node for processing, and determine the second cost of offloading the historical task to the edge computing node for processing based on the second processing latency, latency sensitivity coefficient, second processing energy consumption and energy consumption sensitivity coefficient; determine the third processing latency and third processing energy consumption of offloading the historical task to the cloud server for processing, and determine the third cost of offloading the historical task to the cloud server for processing based on the third processing latency, latency sensitivity coefficient, third processing energy consumption and energy consumption sensitivity coefficient; determine the average cost of the first cost, second cost and third cost of all historical tasks; determine the inverse value of the average cost as the reward function.

[0054] The second processing latency includes: the communication latency of historical tasks being offloaded to the edge computing node and the computing latency of the edge computing node processing historical tasks; the second processing energy consumption includes: the communication energy consumption of historical tasks being offloaded to the edge computing node and the computing energy consumption of the edge computing node processing historical tasks; the third processing latency includes: the communication latency of historical tasks being offloaded to the cloud server and the computing latency of the cloud server processing historical tasks; the third processing energy consumption includes: the communication energy consumption of historical tasks being offloaded to the cloud server and the computing energy consumption of the cloud server processing historical tasks.

[0055] The following example, using a computing network comprising N user devices and M edge computing nodes, illustrates the calculation process of the reward function:

[0056] For task n, determine its corresponding latency sensitivity coefficient. and energy consumption sensitivity coefficient

[0057] The first processing latency of user equipment n for local processing task n is calculated as follows: Among them, z n Let n be the size of the task. Let n be the CPU computing power (CPU revolutions per second); calculate the first processing energy consumption of local processing task n on user equipment n: Among them, κ n The correlation coefficient of the user equipment n hardware itself; the first cost of user equipment n locally processing historical tasks n is:

[0058]

[0059] The data transmission rate between user equipment n and edge computing node m is calculated as: R n,m =θ n B m log2(1+SINR n,m ), where B m Let θ be the coverage bandwidth of edge computing node m. n (0<θ n <1) represents the bandwidth allocation from edge computing node m to user device n, SINR n,m The signal-to-interference-to-noise ratio (SINNR) between user equipment n and edge computing node m is calculated as follows: Among them, g n,m p is the channel gain between user equipment n and edge computing node m. n,m Let n be the transmit power from user equipment n to edge computing node m. Let be the average power of Gaussian white noise in the transmit channel of user equipment n;

[0060] The communication latency for offloading computation task n to edge computing node m is: The communication energy consumption for offloading computation task n to edge computing node m is: in, The transmission power is used to calculate the computation latency of edge computing node m for processing task n. Where, β m,n The percentage of computing resources allocated to user device n by edge computing node m is given. Since the size of the returned result is much smaller than the task size itself, the return latency is negligible. The energy consumption of edge computing node m in processing task n is: in, Let the standby power of user equipment n be denoted as ; the second cost for offloading task n to edge computing node m is ultimately determined as:

[0061]

[0062] When unloading task n to cloud server c, task n needs to be unloaded to the corresponding edge computing node m first, and then unloaded to cloud server c through the central scheduler; the communication latency for unloading task n to cloud server c is: The communication energy consumption of offloading computing task n to cloud server c is: The computation latency for cloud server c to process task n is: The computing energy consumption of cloud server c in processing task n is: The third cost of offloading task n to cloud server c for processing was ultimately determined to be:

[0063]

[0064] The average cost of processing all tasks is calculated as follows:

[0065]

[0066] The optimization objective during model training is to minimize the above average cost, i.e.:

[0067] P1:

[0068] Where a is the task unloading decision vector, θ is the bandwidth resource allocation decision vector, and β is the computational resource allocation decision vector, the reward function r = -Cost can be set. t .

[0069] Optionally, a second state vector can be constructed based on the acquired historical state information: S t ={z n ,M,h n ,1- Among them, z n The size of task n (in bits) is represented by M, the number of edge nodes is h. n ={h n,1 ,h n,2 ,...,h n,M} represents the channel parameters of n to M edge computing nodes of the user equipment. This represents the remaining bandwidth resources of M edge computing nodes. This represents the remaining computing resources of the M edge computing nodes.

[0070] Step S5082: Input the second state vector into the task unloading sub-model to obtain the optimal unloading strategy predicted by the task unloading sub-model based on the ε-greedy algorithm. The first Q-value corresponding to the optimal unloading strategy is determined, and the first maximum Q-value is determined from the Q-values ​​corresponding to all unloading strategies. Based on the first Q-value, the first maximum Q-value, and the reward function, the first loss function of the task unloading sub-model is determined. The formula for the first loss function is as follows:

[0071]

[0072] Where γ is the decay factor in reinforcement learning, This indicates that the best uninstallation strategy will be implemented. The state after s n The first Q value, max a Q(s ′ n a) represents the state s after executing uninstallation strategy a. ′ n The first maximum Q value.

[0073] Step S5083: Construct a third state vector based on the second historical state information of each edge computing node corresponding to the optimal unloading strategy.

[0074] Step S5084: Input the third state vector into the bandwidth resource allocation sub-model to obtain the optimal bandwidth resource allocation strategy predicted by the bandwidth resource allocation sub-model based on the ε-greedy algorithm and the second Q value corresponding to the optimal bandwidth resource allocation strategy. Then, determine the second maximum Q value from the Q values ​​corresponding to all bandwidth resource allocation strategies. Based on the second Q value, the second maximum Q value, and the reward function, determine the second loss function of the bandwidth resource allocation sub-model. The formula for the second loss function is as follows:

[0075]

[0076] in, This indicates the implementation of the optimal bandwidth resource allocation strategy. The state after s n The second Q value, max θ Q(s ′ n ,θ)) represents the state s after executing the bandwidth resource allocation strategy θ. ′ n The second largest Q value.

[0077] Step S5085: Input the third state vector into the computational resource allocation sub-model to obtain the optimal computational resource allocation strategy predicted by the computational resource allocation sub-model based on the ε-greedy algorithm and the third Q value corresponding to the optimal computational resource allocation strategy. Then, determine the third maximum Q value from the Q values ​​corresponding to all computational resource allocation strategies. Based on the third Q value, the third maximum Q value, and the reward function, determine the third loss function of the computational resource allocation sub-model. The formula for the third loss function is as follows:

[0078]

[0079] in, This indicates the implementation of the optimal computing resource allocation strategy. The state after s n The third Q value, max β Q(s ′ n ,β)) represents the state s after executing the computational resource allocation strategy β. ′ n The third largest Q value.

[0080] Step S5086: During the iterative training process, adjust the model parameters of the initial model method according to the first loss function, the second loss function, and the third loss function.

[0081] It should be noted that during the training process of the task scheduling model, it is necessary to ensure that the model prediction results meet the following constraints:

[0082] 1) Tasks cannot be split. Each historical task can only be processed locally, or offloaded to an edge computing node for processing, or offloaded to a cloud server for processing.

[0083] 2) The total bandwidth resources occupied by multiple historical tasks offloaded to the same edge computing node shall not exceed the available bandwidth resources B of the edge computing node. max ;

[0084] 3) The total computing resources occupied by multiple historical tasks offloaded to the same edge computing node do not exceed the available computing resources f of the edge computing node. max .

[0085] In this embodiment, the multi-layered computing network includes multiple user devices, multiple edge computing nodes, and a cloud server, making the scenario closer to a real task scheduling environment. When training the task scheduling model, a reinforcement learning algorithm is used for iterative training. Considering issues such as channel interference in dynamic environments, the reward function is determined based on the average cost of the first cost of processing tasks locally on user devices, the second cost of offloading tasks to edge computing nodes, and the third cost of offloading tasks to cloud servers. After model training, the model can predict the optimal task scheduling strategy, including task offloading strategy, bandwidth resource allocation strategy, and computing resource allocation strategy, using only the current state of the computing network and the task state as input. This achieves efficient task scheduling and effectively solves the technical problem of lacking efficient management and coordination solutions for a large number of tasks, computing resources, and wireless resources in edge computing scenarios.

[0086] Example 2

[0087] According to embodiments of this application, a multi-level computing power network task scheduling apparatus for implementing the multi-level computing power network task scheduling method in Embodiment 1 is also provided, such as... Figure 6 As shown, the multi-level computing power network task scheduling device includes at least an acquisition module 61, a construction module 62, an analysis module 63, and a scheduling module 64, wherein:

[0088] The acquisition module 61 is used to acquire the first status information of multiple user devices, the second status information of multiple edge computing nodes and the third status information of cloud servers in the multi-level computing network, and to acquire the task information of the tasks to be processed for each user device.

[0089] The first state information includes at least the following: computing resources of each user device, such as CPU computing power and computer hardware parameters; channel parameters between each user device and the corresponding edge computing node, such as channel gain, transmit power, and signal-to-interference-plus-noise ratio; the second state information includes at least the following: bandwidth resources, bandwidth resource allocation status, computing resources, and computing resource allocation status of each edge computing node; the third state information includes at least the following: computing resources of the cloud server; and the task information includes at least the following: task identifier, task type, and task size of each task to be processed.

[0090] Module 62 is used to construct a first state vector based on the first state information, the second state information, the third state information, and the task information.

[0091] Analysis module 63 is used to analyze the first state vector using a pre-trained task scheduling model to obtain the target scheduling strategy. The task scheduling model is trained using a reinforcement learning algorithm. During training, the reward function is determined based on the target offloading cost. The target offloading cost is the average cost of the first cost of processing the task locally on the user device, the second cost of offloading the task to the edge computing node for processing, and the third cost of offloading the task to the cloud server for processing.

[0092] The scheduling module 64 is used to schedule multiple tasks to be processed according to a target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0093] Optionally, the multi-level computing network task scheduling device in this embodiment further includes a model training module, used to train the task scheduling model through the following steps: constructing an initial model, wherein the initial model includes: a task offloading sub-model, a bandwidth resource allocation sub-model, and a computing resource allocation sub-model; obtaining multiple training samples, wherein each training sample includes first historical state information of multiple user devices, historical task information, second historical state information of multiple edge computing nodes, and third historical state information of cloud servers within a historical task time slot of the multi-level computing network; determining the number of iterations and the training batch size; and inputting the multiple training samples into the initial model for iterative training according to the number of iterations and the training batch size to obtain the task scheduling model.

[0094] Optionally, during iterative training, the model training module, for each training sample, determines a reward function based on the first historical state information, historical task information, second historical state information, and third historical state information in the training sample, and constructs a second state vector. The second state vector is then input into the task unloading sub-model to obtain the optimal unloading strategy predicted by the task unloading sub-model based on the ε-greedy algorithm and the first Q-value corresponding to the optimal unloading strategy. A first maximum Q-value is determined from the Q-values ​​corresponding to all unloading strategies. The first loss function of the task unloading sub-model is determined based on the first Q-value, the first maximum Q-value, and the reward function. A third state vector is constructed based on the second historical state information of each edge computing node corresponding to the optimal unloading strategy. This third state vector is then input into the bandwidth resource allocation sub-model to obtain the bandwidth resource allocation sub-model based on the ε-greedy algorithm. The optimal bandwidth resource allocation strategy predicted by the y-algorithm and the corresponding second Q-value are obtained. The second maximum Q-value is determined from all Q-values ​​corresponding to all bandwidth resource allocation strategies. The second loss function of the bandwidth resource allocation sub-model is determined based on the second Q-value, the second maximum Q-value, and the reward function. The third state vector is input into the computational resource allocation sub-model to obtain the optimal computational resource allocation strategy predicted by the ε-greedy algorithm and the corresponding third Q-value. The third maximum Q-value is determined from all Q-values ​​corresponding to all computational resource allocation strategies. The third loss function of the computational resource allocation sub-model is determined based on the third Q-value, the third maximum Q-value, and the reward function. During iterative training, the model parameters of the initial model method are adjusted based on the first, second, and third loss functions.

[0095] As an optional implementation, the model training module can determine the reward function as follows: For each historical task in the historical task information, determine the latency sensitivity coefficient and energy consumption sensitivity coefficient of the historical task; determine the first processing latency and first processing energy consumption of the historical task processed locally by the user device, and determine the first cost of processing the historical task locally by the user device based on the first processing latency, latency sensitivity coefficient, first processing energy consumption and energy consumption sensitivity coefficient; determine the second processing latency and second processing energy consumption of offloading the historical task to the edge computing node for processing, and determine the second cost of offloading the historical task to the edge computing node for processing based on the second processing latency, latency sensitivity coefficient, second processing energy consumption and energy consumption sensitivity coefficient; determine the third processing latency and third processing energy consumption of offloading the historical task to the cloud server for processing, and determine the third cost of offloading the historical task to the cloud server for processing based on the third processing latency, latency sensitivity coefficient, third processing energy consumption and energy consumption sensitivity coefficient; determine the average cost of the first cost, second cost and third cost of all historical tasks; determine the inverse value of the average cost as the reward function.

[0096] The second processing latency includes: the communication latency of historical tasks being offloaded to the edge computing node and the computing latency of the edge computing node processing historical tasks; the second processing energy consumption includes: the communication energy consumption of historical tasks being offloaded to the edge computing node and the computing energy consumption of the edge computing node processing historical tasks; the third processing latency includes: the communication latency of historical tasks being offloaded to the cloud server and the computing latency of the cloud server processing historical tasks; the third processing energy consumption includes: the communication energy consumption of historical tasks being offloaded to the cloud server and the computing energy consumption of the cloud server processing historical tasks.

[0097] Optionally, during the training of the task scheduling model, the model training module determines that the model prediction results meet the following constraints: tasks cannot be split, each historical task can only be processed locally, or offloaded to an edge computing node for processing, or offloaded to a cloud server for processing; the total bandwidth resources occupied by multiple historical tasks offloaded to the same edge computing node do not exceed the available bandwidth resources of the edge computing node; and the total computing resources occupied by multiple historical tasks offloaded to the same edge computing node do not exceed the available computing resources of the edge computing node.

[0098] It should be noted that each module in the multi-level computing network task scheduling device in this application embodiment corresponds one-to-one with each implementation step of the multi-level computing network task scheduling method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0099] Example 3

[0100] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the multi-level computing network task scheduling method in Embodiment 1 by running the computer program.

[0101] Specifically, the device containing the non-volatile storage medium executes the following steps by running the computer program: acquiring first state information of multiple user devices, second state information of multiple edge computing nodes, and third state information of the cloud server in a multi-level computing network, and acquiring task information of the tasks to be processed for each user device; constructing a first state vector based on the first state information, second state information, third state information, and task information; analyzing the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy, wherein the task scheduling model is trained using a reinforcement learning algorithm, and the reward function is determined based on the target offloading cost during training, the target offloading cost being the average cost of the first cost of processing the task locally on the user device, the second cost of offloading the task to the edge computing node for processing, and the third cost of offloading the task to the cloud server for processing; scheduling multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0102] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the multi-level computing power network task scheduling method in embodiment 1 during runtime.

[0103] Specifically, the computer program executes the following steps during runtime: acquiring first state information of multiple user devices, second state information of multiple edge computing nodes, and third state information of the cloud server in a multi-layered computing network, and acquiring task information of the tasks to be processed for each user device; constructing a first state vector based on the first state information, second state information, third state information, and task information; analyzing the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy, wherein the task scheduling model is trained using a reinforcement learning algorithm, and the reward function is determined based on the target offloading cost during training, the target offloading cost being the average cost of the first cost of processing the task locally on the user device, the second cost of offloading the task to the edge computing node for processing, and the third cost of offloading the task to the cloud server for processing; scheduling multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0104] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the multi-level computing power network task scheduling method of Embodiment 1 through the computer program.

[0105] Specifically, the processor is configured to execute the following steps via a computer program: acquiring first state information of multiple user devices, second state information of multiple edge computing nodes, and third state information of the cloud server in a multi-layered computing network, and acquiring task information of the tasks to be processed for each user device; constructing a first state vector based on the first state information, second state information, third state information, and task information; analyzing the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy, wherein the task scheduling model is trained using a reinforcement learning algorithm, and the reward function is determined based on the target offloading cost during training, the target offloading cost being the average cost of the first cost of processing the task locally on the user device, the second cost of offloading the task to the edge computing node for processing, and the third cost of offloading the task to the cloud server for processing; scheduling multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

[0106] The sequence numbers of the above embodiments are for descriptive purposes only and do not indicate the superiority or inferiority of the embodiments.

[0107] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0112] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A multi-level computing power network task scheduling method, characterized in that, include: The system acquires the first state information of multiple user devices, the second state information of multiple edge computing nodes, and the third state information of cloud servers in a multi-level computing network, and acquires the task information of the tasks to be processed for each user device. A first state vector is constructed based on the first state information, the second state information, the third state information, and the task information; The first state vector is analyzed using a pre-trained task scheduling model to obtain a target scheduling strategy. The task scheduling model is trained using a reinforcement learning algorithm. During training, a reward function is determined based on the target offloading cost. The target offloading cost is the average cost of a first cost of processing the task locally on the user device, a second cost of offloading the task to an edge computing node, and a third cost of offloading the task to a cloud server. The first cost, the second cost, and the third cost are all determined based at least on the latency sensitivity coefficient and energy consumption sensitivity coefficient of the task. The task scheduling model includes a task offloading sub-model, a bandwidth resource allocation sub-model, and a computing resource allocation sub-model. The multiple tasks to be processed are scheduled according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

2. The method according to claim 1, characterized in that, The first status information includes at least: the computing resources of each user device and the channel parameters between each user device and the corresponding edge computing node; The second status information includes at least: bandwidth resources, bandwidth resource allocation status, computing resources, and computing resource allocation status of each edge computing node; The third state information includes at least: the computing resources of the cloud server; The task information includes at least: the task identifier, task type, and task size of each task to be processed.

3. The method according to claim 1, characterized in that, The training process of the task scheduling model includes: Build the initial model; Multiple training samples are obtained, wherein each training sample includes the first historical state information of multiple user devices, historical task information, the second historical state information of multiple edge computing nodes, and the third historical state information of cloud servers within a historical task time slot of the multi-level computing power network. Determine the number of iterations and the training batch size; Based on the number of iterations and the training batch size, the multiple training samples are input into the initial model for iterative training to obtain the task scheduling model.

4. The method according to claim 3, characterized in that, The multiple training samples are input into the initial model for iterative training, including: For each training sample, the reward function is determined based on the first historical state information, the historical task information, the second historical state information, and the third historical state information in the training sample, and a second state vector is constructed. The second state vector is input into the task unloading sub-model to obtain the optimal unloading strategy predicted by the task unloading sub-model based on the ε-greedy algorithm and the first Q value corresponding to the optimal unloading strategy. The first maximum Q value is determined from the Q values ​​corresponding to all unloading strategies. The first loss function of the task unloading sub-model is determined based on the first Q value, the first maximum Q value and the reward function. A third state vector is constructed based on the second historical state information of each edge computing node corresponding to the optimal unloading strategy. The third state vector is input into the bandwidth resource allocation sub-model to obtain the optimal bandwidth resource allocation strategy predicted by the bandwidth resource allocation sub-model based on the ε-greedy algorithm and the second Q value corresponding to the optimal bandwidth resource allocation strategy. The second maximum Q value is determined from the Q values ​​corresponding to all bandwidth resource allocation strategies. The second loss function of the bandwidth resource allocation sub-model is determined based on the second Q value, the second maximum Q value and the reward function. The third state vector is input into the computing resource allocation sub-model to obtain the optimal computing resource allocation strategy predicted by the computing resource allocation sub-model based on the ε-greedy algorithm and the third Q value corresponding to the optimal computing resource allocation strategy. The third maximum Q value is determined from the Q values ​​corresponding to all computing resource allocation strategies. The third loss function of the computing resource allocation sub-model is determined based on the third Q value, the third maximum Q value and the reward function. During iterative training, the model parameters of the initial model method are adjusted based on the first loss function, the second loss function, and the third loss function.

5. The method according to claim 4, characterized in that, Determining the reward function based on the first historical state information, the historical task information, the second historical state information, and the third historical state information in the training samples includes: For each historical task in the historical task information, determine the latency sensitivity coefficient and energy consumption sensitivity coefficient of the historical task; A first processing latency and a first processing power consumption for the user equipment to locally process the historical task are determined, and a first cost for the user equipment to locally process the historical task is determined based on the first processing latency, the latency sensitivity coefficient, the first processing power consumption, and the power consumption sensitivity coefficient. The second processing latency and the second processing energy consumption for offloading the historical task to the edge computing node are determined, and the second cost for offloading the historical task to the edge computing node is determined based on the second processing latency, the latency sensitivity coefficient, the second processing energy consumption and the energy consumption sensitivity coefficient. The third processing latency and third processing energy consumption of offloading the historical task to the cloud server are determined, and the third cost of offloading the historical task to the cloud server is determined based on the third processing latency, the latency sensitivity coefficient, the third processing energy consumption and the energy consumption sensitivity coefficient. Determine the average cost of the first cost, the second cost, and the third cost for all the historical tasks; The inverse value of the average cost is determined to be the reward function.

6. The method according to claim 5, characterized in that, The second processing latency includes: the communication latency of the historical task being offloaded to the edge computing node and the computing latency of the edge computing node processing the historical task; The second processing energy consumption includes: the communication energy consumption of the historical task being offloaded to the edge computing node and the computing energy consumption of the edge computing node processing the historical task; The third processing latency includes: the communication latency of the historical task being unloaded to the cloud server and the computation latency of the cloud server processing the historical task; The third processing energy consumption includes: the communication energy consumption of the historical task being offloaded to the cloud server and the computing energy consumption of the cloud server processing the historical task.

7. The method according to claim 3, characterized in that, The method further includes: During the training process of the task scheduling model, the model prediction results are determined to satisfy the following constraints: Tasks cannot be split; each historical task can only be processed locally, or offloaded to an edge computing node for processing, or offloaded to a cloud server for processing. The total bandwidth resources occupied by multiple historical tasks offloaded to the same edge computing node shall not exceed the available bandwidth resources of the edge computing node. The total computing resources occupied by multiple historical tasks offloaded to the same edge computing node shall not exceed the available computing resources of the edge computing node.

8. A multi-level computing power network task scheduling device, characterized in that, include: The acquisition module is used to acquire the first status information of multiple user devices, the second status information of multiple edge computing nodes and the third status information of cloud servers in the multi-level computing network, and to acquire the task information of the tasks to be processed for each user device. The construction module is used to construct a first state vector based on the first state information, the second state information, the third state information, and the task information; The analysis module is used to analyze the first state vector using a pre-trained task scheduling model to obtain a target scheduling strategy. The task scheduling model is trained using a reinforcement learning algorithm. During training, a reward function is determined based on the target offloading cost. The target offloading cost is the average cost of a first cost of processing the task locally on the user device, a second cost of offloading the task to an edge computing node, and a third cost of offloading the task to a cloud server. The first cost, the second cost, and the third cost are all determined based at least on the latency sensitivity coefficient and energy consumption sensitivity coefficient of the task. The task scheduling model includes a task offloading sub-model, a bandwidth resource allocation sub-model, and a computing resource allocation sub-model. The scheduling module is used to schedule multiple tasks to be processed according to the target scheduling strategy, wherein the target scheduling strategy includes: a task offloading strategy, a bandwidth resource allocation strategy, and a computing resource allocation strategy.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the multi-level computing network task scheduling method according to any one of claims 1 to 7 by running the computer program.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the multi-level computing network task scheduling method according to any one of claims 1 to 7 through the computer program.

Images