Task offloading method and system

Abstract

The present disclosure relates to the technical field of communications, and provides a task offloading method and system. The method comprises: obtaining a first state of an offloading environment, the offloading environment comprising a plurality of nodes, and each node being one of a terminal, an edge server, and a cloud server; and determining first information on the basis of the first state, the first information being used to indicate a strategy for the terminal to perform task offloading. According to the task offloading solution provided by the embodiments of the present disclosure, tasks can be systematically assigned to nodes having different computing capabilities, so as to jointly improve overall network efficiency.

Description

Task offloading method and system TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of communication, and particularly relates to a task offloading method and system. BACKGROUND

[0002] In the field of communication, the cooperation between cloud computing and edge computing occupies the center stage and is expected to provide an enhanced and responsive connection experience.

[0003] SUMMARY

[0004] The present disclosure provides a task offloading method and system, which can systematically allocate tasks to nodes with different computing capabilities to improve the overall efficiency of the network.

[0005] The first aspect embodiment of the present disclosure provides a task offloading method, executed by a terminal side, comprising: acquiring a first state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determining first information according to the first state, the first information being used to indicate a strategy of the terminal executing task offloading.

[0006] The second aspect embodiment of the present disclosure provides a task offloading method, executed by an edge server side, comprising: acquiring a second state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determining second information according to the second state, the second information being used to indicate a strategy of the edge server executing task offloading.

[0007] The third aspect embodiment of the present disclosure provides a task offloading method, executed by a cloud server side, comprising: acquiring a third state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determining third information according to the third state, the third information being used to indicate a strategy of the cloud server executing task offloading.

[0008] The fourth aspect embodiment of the present disclosure provides a terminal, comprising: a processing module configured to acquire a first state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determine first information according to the first state, the first information being used to indicate a strategy of the terminal executing task offloading.

[0009] In a fifth aspect, an embodiment of the present disclosure provides an edge server, comprising: a processing module configured to acquire a second state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine second information according to the second state, the second information being used to indicate a strategy of the edge server performing task offloading.

[0010] In a sixth aspect, an embodiment of the present disclosure provides a cloud server, comprising: a processing module configured to acquire a third state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine third information according to the third state, the third information being used to indicate a strategy of the cloud server performing task offloading.

[0011] In a seventh aspect, an embodiment of the present disclosure provides a communication device, comprising: one or more processors; wherein the processor is configured to execute the method in the first aspect, or execute the method in the second aspect, or execute the method in the third aspect.

[0012] In an eighth aspect, an embodiment of the present disclosure provides a task offloading system, comprising: a terminal, an edge server, and a cloud server; the terminal executes the method in the first aspect, the edge server executes the method in the second aspect, and the cloud server executes the method in the third aspect.

[0013] In a ninth aspect, an embodiment of the present disclosure provides a computer storage medium, wherein the computer storage medium stores computer executable instructions; the computer executable instructions are executed by a processor to implement the method in the first aspect, or the method in the second aspect, or the method in the third aspect.

[0014] In a tenth aspect, an embodiment of the present disclosure provides a computer program product, wherein the computer program product stores a computer program; the computer program is executed by a processor to implement the method in the first aspect, or the method in the second aspect, or the method in the third aspect.

[0015] Additional aspects and advantages of the present disclosure will be made apparent by the following description of embodiments, given as a non-restrictive example, with reference to the attached drawings wherein: BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and / or additional aspects and advantages of the present disclosure will become apparent and be made clear to those skilled in the art from the following description of embodiments, given as a non-restrictive example, with reference to the attached drawings, wherein:

[0017] FIG. 1 is a schematic diagram of an architecture of a task offloading system according to an embodiment of the present disclosure;

[0018] FIG. 2 is a flow diagram of a task offloading method according to an embodiment of the present disclosure;

[0019] FIG. 3 is a flow diagram of a task offloading method according to an embodiment of the present disclosure;

[0020] FIG. 4 is a flow diagram of a task offloading method according to an embodiment of the present disclosure;

[0021] FIG. 5 is a schematic diagram of an example according to an embodiment of the present disclosure;

[0022] FIG. 6 is a schematic diagram of an example according to an embodiment of the present disclosure;

[0023] FIG. 7 is a block diagram of a terminal according to an embodiment of the present disclosure;

[0024] FIG. 8 is a block diagram of an edge server according to an embodiment of the present disclosure;

[0025] FIG. 9 is a block diagram of a cloud server according to an embodiment of the present disclosure;

[0026] FIG. 10 is a structural schematic diagram of a communication device according to an embodiment of the present disclosure;

[0027] FIG. 11 is a structural schematic diagram of a chip according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as limiting the present disclosure. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] For ease of understanding, first introduce the terms related to the embodiments of the present disclosure.

[0030] 1. Cloud server

[0031] The cloud server can provide cloud computing services. Cloud computing is a mode of providing computing resources and services over the Internet, allowing users to access data, applications and computing power on remote servers without having or maintaining physical hardware and software locally. The core concept of cloud computing is to distribute computing tasks across a large number of computers (servers) in the network to form a resource pool, and users can obtain the required computing power, storage space or specific services from this resource pool according to their needs.

[0032] 2. Edge server

[0033] Edge Server refers to a server deployed at the edge of a network, i.e., closer to end users or data sources. Unlike traditional centralized servers, edge servers aim to reduce the latency of data transmission, improve the speed of data processing, and enhance user experience. They are usually located in Internet Service Provider (ISP) facilities or within geographically dispersed data centers to be closer to users or data generation points.

[0034] Embodiments of the present disclosure provide a task offloading method and system.

[0035] In a first aspect, embodiments of the present disclosure provide a task offloading method, executed by a terminal side, comprising: obtaining a first state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; determining first information according to the first state, the first information being used to indicate a strategy of the terminal executing task offloading.

[0036] By applying the technical solutions of the present disclosure, tasks can be systematically allocated to nodes with different computing capabilities to collectively improve the overall efficiency of the network.

[0037] In combination with some embodiments of the first aspect, the first information comprises at least one of:

[0038] An offloading destination of the task; and an offloading percentage of the task.

[0039] In combination with some embodiments of the first aspect, the offloading destination comprises at least one of:

[0040] An edge server; a cloud server; and a terminal.

[0041] In combination with some embodiments of the first aspect, the offloading percentage comprises at least one of:

[0042] A percentage of the task offloaded to the edge server; a percentage of the task offloaded to the cloud server; and a percentage of the task processed locally by the terminal.

[0043] In combination with some embodiments of the first aspect, the first state comprises at least one of:

[0044] Task information; locally available computing resources of the terminal; local computing capability of the terminal; a percentage of tasks that can be processed locally by the terminal; computing capability of the edge server; a percentage of tasks that can be offloaded to the edge server; computing capability of the cloud server; a percentage of tasks that can be offloaded to the cloud server; and a communication rate between nodes.

[0045] In some embodiments of the first aspect, determining the first information according to the first state comprises: inputting the first state into a first model, and determining the first information according to an output result of the first model; wherein the first model is trained by using a multi-agent deep deterministic policy gradient (MADDPG) algorithm.

[0046] In some embodiments of the first aspect, each of the plurality of nodes has an agent; and inputting the first state into the first model comprises: inputting the first state into the first model corresponding to the agent of the terminal.

[0047] In some embodiments of the first aspect, a model parameter of the first model is determined according to a model training result of each agent.

[0048] In some embodiments of the first aspect, a reward index in the MADDPG algorithm is determined according to a total time and a total energy of each agent for processing a task, and the reward index is used to determine a loss function corresponding to the first model.

[0049] In some embodiments of the first aspect, the total time includes a task transmission time between agents and a calculation time of the agents for the task.

[0050] In a second aspect, the embodiments of the present disclosure provide a task offloading method, executed by an edge server side, comprising: obtaining a second state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; determining second information according to the second state, the second information being used to indicate a strategy of the edge server for performing task offloading.

[0051] By applying the technical solutions of the present disclosure, tasks can be systematically allocated to nodes with different computing capabilities to improve the overall efficiency of the network.

[0052] In some embodiments of the second aspect, the second information comprises at least one of:

[0053] An offloading destination of the task; and an offloading percentage of the task.

[0054] In some embodiments of the second aspect, the offloading destination comprises at least one of:

[0055] A cloud server; a first edge server; and a local edge server.

[0056] In some embodiments of the second aspect, the offloading percentage comprises at least one of:

[0057] Percentage of tasks offloaded to cloud servers; percentage of tasks offloaded to first edge servers; percentage of tasks processed locally by edge servers.

[0058] In conjunction with some embodiments of the second aspect, the second state includes at least one of the following:

[0059] Task information; available computing resources on the edge server; local computing power of the edge server; percentage of tasks that the edge server can process locally; computing power of the cloud server; percentage of tasks that can be offloaded to the cloud server; computing power of the first edge server; percentage of tasks that can be offloaded to the first edge server; communication rate between nodes.

[0060] In conjunction with some embodiments of the second aspect, determining the second information based on the second state includes: inputting the second state into a second model and determining the second information based on the output of the second model; wherein the second model is trained using the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm.

[0061] In some embodiments of the second aspect, each of the plurality of nodes has an agent; inputting the second state into the second model includes: inputting the second state into the second model corresponding to the agent of the edge server.

[0062] In conjunction with some embodiments of the second aspect, the model parameters of the second model are determined based on the model training results of each agent.

[0063] In conjunction with some embodiments of the second aspect, the reward metric in the MADDPG algorithm is determined based on the total time and total energy of each agent in processing the task, and the reward metric is used to determine the loss function corresponding to the second model.

[0064] In conjunction with some embodiments of the second aspect, the total time includes the task transmission time between agents and the computation time of agents for the task.

[0065] Thirdly, this disclosure proposes a task offloading method executed by a cloud server. The method includes: obtaining a third state of the offloading environment, the offloading environment including multiple nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determining third information based on the third state, the third information being used to instruct the cloud server to execute a task offloading strategy.

[0066] By applying the technical solution disclosed herein, tasks can be systematically assigned to nodes with different computing capabilities to jointly improve the overall efficiency of the network.

[0067] In some embodiments of the third aspect, the third information comprises at least one of:

[0068] a destination of offloading the task; a percentage of offloading the task.

[0069] In some embodiments of the third aspect, the destination of offloading comprises at least one of:

[0070] a first cloud server; a local of the cloud server.

[0071] In some embodiments of the third aspect, the percentage of offloading comprises at least one of:

[0072] a percentage of offloading the task to the first cloud server; a percentage of processing the task by the local of the cloud server.

[0073] In some embodiments of the third aspect, the third state comprises at least one of:

[0074] task information; a computing resource available locally of the cloud server; a computing capability of the local of the cloud server; a percentage of the task processable locally of the edge server; a computing capability of the first cloud server; a percentage of the task offloadable to the first cloud server; a communication rate between nodes.

[0075] In some embodiments of the third aspect, determining the third information according to the third state comprises: inputting the third state into a third model, and determining the third information according to an output result of the third model; wherein the third model is obtained by training using a multi-agent deep deterministic policy gradient (MADDPG) algorithm.

[0076] In some embodiments of the third aspect, each of the plurality of nodes has an agent; and inputting the third state into the third model comprises: inputting the third state into the third model corresponding to the agent of the cloud server.

[0077] In some embodiments of the third aspect, a model parameter of the third model is determined according to a model training result of each agent.

[0078] In some embodiments of the third aspect, a reward index in the MADDPG algorithm is determined according to a total time and a total energy of each agent for processing the task, and the reward index is used to determine a loss function corresponding to the third model.

[0079] In some embodiments of the third aspect, the total time comprises a task transmission time between the agents, and a computing time of the agents for the task.

[0080] In a fourth aspect, the embodiments of the present disclosure provide a terminal, which comprises: a processing module configured to acquire a first state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine first information according to the first state, the first information being used to indicate a strategy of the terminal performing task offloading.

[0081] In a fifth aspect, the embodiments of the present disclosure provide an edge server, which comprises: a processing module configured to acquire a second state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine second information according to the second state, the second information being used to indicate a strategy of the edge server performing task offloading.

[0082] In a sixth aspect, the embodiments of the present disclosure provide a cloud server, which comprises: a processing module configured to acquire a third state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine third information according to the third state, the third information being used to indicate a strategy of the cloud server performing task offloading.

[0083] In a seventh aspect, the embodiments of the present disclosure provide a communication device, which can be a terminal, an edge server, or a cloud server, comprising: one or more processors; wherein the processor of the terminal is configured to execute the method of the first aspect, the processor of the edge server is configured to execute the method of the second aspect, and the processor of the cloud server is configured to execute the method of the third aspect.

[0084] In an eighth aspect, the embodiments of the present disclosure provide a task offloading system, which comprises: a terminal, an edge server, and a cloud server; the terminal executes the method of the first aspect, the edge server executes the method of the second aspect, and the cloud server executes the method of the third aspect.

[0085] In a ninth aspect, the embodiments of the present disclosure provide a computer storage medium, wherein the computer storage medium stores computer executable instructions; the computer executable instructions are executed by a processor to implement the method of the first aspect, or the method of the second aspect, or the method of the third aspect.

[0086] In a tenth aspect, the embodiments of the present disclosure provide a computer program product, which comprises a computer program; the computer program is executed by a processor to implement the method of the first aspect, or the method of the second aspect, or the method of the third aspect.

[0087] In a 11th aspect, the embodiments of the present disclosure provide a computer program, which, when executed on a computer, causes the computer to perform the method according to the embodiments of the first aspect, or the method according to the embodiments of the second aspect, or the method according to the embodiments of the third aspect.

[0088] In a 12th aspect, the embodiments of the present disclosure provide a chip or a chip system. The chip or the chip system includes processing circuitry configured to perform the method according to the embodiments of the first aspect, or the method according to the embodiments of the second aspect, or the method according to the embodiments of the third aspect.

[0089] It can be understood that the terminal, the edge server, the cloud server, the task offloading system, the computer storage medium, and the like are used to execute the method according to the embodiments of the present disclosure. Therefore, the beneficial effects achieved by the terminal, the edge server, the cloud server, the task offloading system, the computer storage medium, and the like can refer to the beneficial effects in the corresponding method, which will not be described here.

[0090] The embodiments of the present disclosure provide a task offloading method and system. In some embodiments, the terms of the task offloading method and the task processing method, the information processing method, the information offloading method, the information receiving method, the information sending method, and the like can be replaced with each other, and the terms of the task offloading system and the task processing system, the information processing system, the communication system, the information sending system, the information receiving system, and the like can be replaced with each other.

[0091] The embodiments of the present disclosure are not exhaustive, but are only a part of the embodiments, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in an embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily, for example, the scheme after removing some steps in an embodiment can also be implemented as an independent embodiment, and the order of the steps in an embodiment can be exchanged arbitrarily, in addition, the optional implementation manners in an embodiment can be combined arbitrarily; in addition, the embodiments can be combined arbitrarily, for example, the steps of different embodiments or part or all of the steps of different embodiments can be combined arbitrarily, and an embodiment can be combined with the optional implementation manners of other embodiments.

[0092] In the embodiments of the present disclosure, the terms and / or descriptions between the embodiments are consistent if there is no special description and logical conflict, and can be referred to each other, and the technical features in different embodiments can be combined to form a new embodiment according to the logical relationship between them.

[0093] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and not as a limitation on the present disclosure.

[0094] In the embodiments of the present disclosure, an element represented in singular form, such as "a", "an", "the", "said", "the aforementioned", "the foregoing", "this", and the like, unless otherwise specified, can represent "one and only one", or can represent "one or more", "at least one", and the like. For example, in the case of using an article such as "a", "an", "the" in English, the noun after the article can be understood as a singular expression, or can be understood as a plural expression.

[0095] In the embodiments of the present disclosure, "plurality" refers to two or more.

[0096] In some embodiments, the terms "at least one of", "at least one of", "at least one of", "one or more", "a plurality of", "multiple", and the like can be replaced with each other.

[0097] In the embodiments of the present disclosure, the description manner such as "at least one of A, B, C, and the like", "A and / or B and / or C, and the like" includes any one of A, B, C, and the like existing alone, and also includes any combination of any multiple of A, B, C, and the like, each of which can exist alone; for example, "at least one of A, B, C" includes the cases of A alone, B alone, C alone, A and B in combination, A and C in combination, B and C in combination, A and B and C in combination; for example, A and / or B includes the cases of A alone, B alone, and the combination of A and B.

[0098] In some embodiments, the description manner such as "A in one case, and B in another case", "in response to A in one case, and in response to B in another case", and the like, according to the case, can include the following technical solutions: A is executed regardless of B, that is, A in some embodiments; B is executed regardless of A, that is, B in some embodiments; A and B are selectively executed, that is, A and B are selected from A and B to be executed in some embodiments; A and B are both executed, that is, A and B in some embodiments. When there are more branches such as A, B, C, and the like, it is similar to the above.

[0099] The prefix words of "first", "second" and the like in the embodiments of the present disclosure are merely used to distinguish different description objects, and do not constitute limitation on the position, order, priority, quantity or content of the description objects. The description objects are described in the claims or embodiments, and should not be construed as redundant limitation because of the use of the prefix words. For example, the description object is "field", and the ordinal words before "field" in "first field" and "second field" do not limit the position or order between "fields", and "first" and "second" do not limit whether the "fields" modified thereby are in the same message or not, nor limit the order of "first field" and "second field". For another example, the description object is "level", and the ordinal words before "level" in "first level" and "second level" do not limit the priority between "levels". For another example, the quantity of the description object is not limited by the ordinal words, and can be one or more. For example, "first device", wherein the quantity of "device" can be one or more. In addition, the objects modified by different prefix words can be the same or different, for example, the description object is "device", and "first device" and "second device" can be the same device or different devices, and the types thereof can be the same or different. For another example, the description object is "information", and "first information" and "second information" can be the same information or different information, and the contents thereof can be the same or different.

[0100] In some embodiments, "including A", "containing A", "for indicating A", "carrying A" can be interpreted as directly carrying A, or indirectly indicating A.

[0101] In some embodiments, the terms of "in response to", "in response to determining", "in the case of", "when", "when", "if", "if" and the like can be replaced with each other.

[0102] In some embodiments, the terms of "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not lower than", "above" and the like can be replaced with each other, and the terms of "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", "below" and the like can be replaced with each other.

[0103] In some embodiments, an apparatus or the like can be interpreted as an entity, and can also be interpreted as virtual, and the name thereof is not limited to the name described in the embodiments, and the terms "apparatus", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "subject" and the like can be replaced with each other.

[0104] In some embodiments, a "network" can be interpreted as an apparatus (for example, an access network device, a core network device, and the like) included in the network.

[0105] In some embodiments, the terms "access network device (AN device)", "radio access network device (RAN device)", "base station (BS)", "radio base station", "fixed station", "node", "access point", "transmission point (TP)", "reception point (RP)", "transmission / reception point (TRP)", "panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "carrier", "component carrier", "bandwidth part (BWP)" and the like can be replaced with each other.

[0106] In some embodiments, the terms "terminal," "terminal device," "user equipment (UE)," "user terminal," "mobile station (MS)," "mobile terminal (MT)," "subscriber station," "mobile unit," "subscriber unit," "wireless unit," "remote unit," "mobile device," "wireless device," "wireless communication device," "remote device," "mobile subscriber station," "access terminal," "mobile terminal," "wireless terminal," "remote terminal," "handset," "user agent," "mobile client," "client," "NB-IoT (Narrow Band-Internet of Things) device," and the like can be replaced with each other.

[0107] In some embodiments, an access network device, a core network device, or a network device can be replaced with a terminal. For example, for a structure in which communication between an access network device, a core network device, or a network device and a terminal is replaced with communication between a plurality of terminals (for example, also referred to as device-to-device (D2D), vehicle-to-everything (V2X), and the like), embodiments of the present disclosure can also be applied. In this case, a structure in which a terminal has all or part of the functions of an access network device can also be provided. Furthermore, the language of "uplink," "downlink," and the like can also be replaced with language corresponding to communication between terminals (for example, "side"). For example, an uplink channel, a downlink channel, and the like can be replaced with a side channel, and an uplink, a downlink, and the like can be replaced with a sidelink.

[0108] In some embodiments, a terminal can be replaced with an access network device, a core network device, or a network device. In this case, a structure in which an access network device, a core network device, or a network device has all or part of the functions of a terminal can also be provided.

[0109] In some embodiments, the data, information, etc. can be obtained in compliance with the laws and regulations of the country where the location is situated.

[0110] In some embodiments, the data, information, etc. can be obtained after obtaining the consent of the user.

[0111] In some embodiments, the threshold mentioned in the embodiments can be a numerical value, a constant, or some fixed value, etc.

[0112] In addition, each element, each row, or each column in the table of the embodiments of the disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.

[0113] The corresponding relationship shown in each table in the disclosure can be configured or predefined. The values of the information in each table are only examples, and other values can be configured, and the disclosure is not limited. When configuring the corresponding relationship of the information and each parameter, it is not necessarily required to configure all the corresponding relationships shown in each table. For example, the corresponding relationship shown in some rows in the table in the disclosure can not be configured. For another example, the above table can be appropriately deformed and adjusted, for example, split, merged, etc. The name of the parameter shown in the title of each table in the above can also use other names that can be understood by the communication device, and the value or representation of the parameter can also use other values or representations that can be understood by the communication device. When implementing each table, other data structures can also be used, for example, an array, a queue, a container, a stack, a linear table, a pointer, a linked list, a tree, a graph, a structure, a class, a heap, a hash table, etc.

[0114] The predefinition in the disclosure can be understood as definition, predefinition, storage, pre-storage, pre-negotiation, pre-configuration, solidification, or pre-burning.

[0115] The task offloading method and system provided by the disclosure will be described in detail below with reference to the accompanying drawings.

[0116] FIG. 1 shows a structure diagram of a task offloading system according to an embodiment of the disclosure. As shown in FIG. 1, the system architecture can include a terminal 11, an edge server 12, and a cloud server 13.

[0117] In some embodiments, the terminal 11 can be referred to as a terminal device, a user equipment, a mobile station (MS), a mobile terminal (MT), an NB-IoT terminal, etc. The terminal 11 can also be a car with communication function, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality device, an augmented reality device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, etc. Embodiments of the present disclosure do not limit the specific technology and specific device form of the terminal 11.

[0118] In some embodiments, the edge server 12 can be a kind of network device, which can be deployed at the edge of the network, i.e., a server closer to the end user or data source.

[0119] In some embodiments, the cloud server 13 can be a kind of network device, which can provide cloud computing services.

[0120] In some examples, the network device can be an entity for transmitting or receiving signals on the network side. For example, the network device can be a communication satellite, an evolved NodeB (eNB), a transmission reception point (TRP), a next generation NodeB (gNB) in an NR system, a base station in other future mobile communication systems, or an access node in a wireless fidelity (WiFi) system, etc. Embodiments of the present disclosure do not limit the specific technology and specific device form of the network device. The network device provided by the embodiments of the present disclosure can be composed of a central unit (CU) and a distributed unit (DU), wherein the CU can also be referred to as a control unit (control unit). The CU-DU structure can split the protocol layer of the network device, e.g., a base station, and the functions of part of the protocol layer are controlled by the CU, and the functions of the remaining part or all of the protocol layer are distributed in the DU and controlled by the CU.

[0121] It can be understood that the task offloading system described in the embodiments of the present disclosure is for more clearly illustrating the technical solutions of the embodiments of the present disclosure, and does not constitute a limitation on the technical solutions proposed by the embodiments of the present disclosure. Those skilled in the art can know that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions proposed by the embodiments of the present disclosure are also applicable to similar technical problems.

[0122] The following embodiments of the present disclosure can be applied to the task offloading system shown in FIG. 1, or part of the subjects, but are not limited thereto. The subjects shown in FIG. 1 are exemplary, and the communication processing system can include all or part of the subjects in FIG. 1, or other subjects other than FIG. 1. The number and form of each subject is arbitrary, and the connection relationship between the subjects is exemplary. The subjects can be connected or not connected, and the connection can be in any way, can be direct connection or indirect connection, can be wired connection or wireless connection.

[0123] Embodiments of the present disclosure can be applied to satellite communication, Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G NR, 6th generation mobile communication technology (6G), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (Bluetooth (registered trademark)), Public Land Mobile Network (PLMN) network, Device-to-Device (D2D) system, Machine to Machine (M2M) system, Internet of Things (IoT) system, Vehicle-to-Everything (V2X), system using other task offloading methods, next-generation system expanded based on them, and the like. Further, a plurality of systems can be applied in combination (for example, combination of LTE or LTE-A and 5G, and the like).

[0124] The collaboration between cloud computing and edge computing will redefine resource management. Cloud services support dynamic resource allocation, ensuring optimal performance for different applications. This collaboration meets the dynamic needs of the evolving digital environment. The collaboration between cloud computing and edge computing aims to enhance user experience. In future communication technologies, this partnership is expected to achieve a seamless fusion of centralized power and decentralized agility, resulting in a powerful and responsive network. The collaboration between cloud and edge computing promises a future of unparalleled connectivity, efficiency, and transformative user experience. Cloud computing effectively manages data resources, enabling global connectivity and supporting emerging technologies. Meanwhile, edge computing operates at the network edge, focusing on low-latency processing, real-time experiences, and custom network slicing. This powerful combination promises lightning-fast connectivity, efficient resource management, and a seamless fusion of centralized power and decentralized agility, ultimately shaping a future of technology beyond boundaries, providing unprecedented user experience, and promoting global innovation.

[0125] As the communication network technology continues to evolve, a wide variety of application tasks are steadily increasing, intensifying the computing pressure. To address this issue, the task offloading scheme provided by the embodiments of the present disclosure can systematically allocate tasks to nodes with different computing capabilities to collectively improve the overall efficiency of the network.

[0126] Further, to illustrate the specific execution process of each part in the above task offloading system, FIG. 2 shows a schematic diagram of a task offloading method according to an embodiment of the present disclosure. The method can be executed by the terminal side, as shown in FIG. 2, and can include the following steps:

[0127] Step S201, obtaining a first state of an offloading environment.

[0128] In some embodiments, the offloading environment includes a plurality of nodes, where the nodes can be one of a terminal, an edge server, a cloud server, etc. The nodes in the offloading environment can provide task offloading services, such as offloading tasks of a terminal to an edge server or a cloud server, offloading tasks of an edge server to a cloud server, etc.

[0129] In some examples, these nodes such as terminals, edge servers, and cloud servers have computing capabilities and can perform task computing processing, such as data cleaning, data analysis, image processing, physical simulation, bioinformatics analysis, data encryption, data decryption, etc.

[0130] In some examples, tasks can be generated by a terminal. In some examples, tasks can also be passed over by other nodes.

[0131] In some examples, the nodes in the offloading environment can interact with each other to achieve the allocation of tasks to nodes with different computing capabilities to improve the overall efficiency of the network.

[0132] In some embodiments, the first state can be the state of each node obtained by the terminal from the offloading environment, which can include the state of the terminal itself, the state of the edge server, the state of the cloud server, etc.

[0133] In some examples, the terminal can request to obtain the state of the edge server and the state of the cloud server, and then the state of the edge server and the state of the cloud server are sent to the terminal, the terminal receives the state of the edge server and the state of the cloud server, and obtains the first state in combination with the state of the terminal itself.

[0134] In some examples, the nodes in the offloading environment can periodically or non-periodically notify the state, such as that the terminal can obtain the state of the edge server and the state of the cloud server every certain period of time, and obtain the first state in combination with the state of the terminal itself.

[0135] In some embodiments, the first state includes at least one of the following A1 to I1:

[0136] A1, task information, such as the task size, task type, CPU cycle number required for task computation, time limit for obtaining task computation result, etc. of the terminal task.

[0137] B1, computing resources available locally to the terminal, such as CPU resources, GPU resources, RAM resources, storage resources, etc. available locally to the terminal, which can be used for processing of the task.

[0138] C1, computing capability locally to the terminal, such as CPU computing rate, memory speed, storage read-write speed, etc. locally to the terminal.

[0139] D1, percentage of tasks that can be processed locally to the terminal, such as the percentage of tasks that can be processed locally to the terminal, which can be determined according to A1, B1, and C1.

[0140] E1, computing capability of the edge server, such as CPU computing rate, memory speed, storage read-write speed, etc. of the edge server.

[0141] F1, percentage of tasks that can be offloaded to the edge server, such as the percentage of tasks that can be offloaded to the edge server for processing, which can be determined according to D1.

[0142] G1, computing capability of the cloud server, such as CPU computing rate, memory speed, storage read-write speed, etc. of the cloud server.

[0143] H1, a percentage of tasks that can be offloaded to a cloud server, such as a percentage of tasks that can be offloaded to a cloud server for processing, can be determined according to D1.

[0144] I1, a communication rate between nodes, such as a data transmission rate between a terminal, an edge server, and a cloud server.

[0145] Step S202, determining first information according to the first state.

[0146] In some embodiments, the first information can be used to indicate a strategy for the terminal to perform task offloading. In some examples, the terminal performs task offloading according to the strategy indicated by the first information.

[0147] In some embodiments, the first information can include at least one of the following A2 to B2:

[0148] A2, an offloading destination of a task, used to determine a target node to which the task is offloaded.

[0149] B2, an offloading percentage of a task, used to determine a percentage of task offloading.

[0150] In some examples, the offloading destination of a task can include at least one of the following:

[0151] An edge server; a cloud server; a terminal.

[0152] For example, the terminal offloads a task to an edge server for processing, the edge server receives the offloaded task data, performs task computation using the computing resources locally available to the edge server, and then returns the task computation result to the terminal.

[0153] In some examples, the offloading percentage of a task can include at least one of the following:

[0154] A percentage of tasks offloaded to an edge server;

[0155] A percentage of tasks offloaded to a cloud server;

[0156] A percentage of tasks processed locally by a terminal.

[0157] In some examples, the offloading destination of a task can also include another terminal other than the current terminal, such as a first terminal. In some scenarios, if the current terminal has limited computing resources available and the first terminal has corresponding available computing resources and computing capabilities, the task can be offloaded to the first terminal for processing. Correspondingly, the offloading percentage of a task can also include a percentage of tasks offloaded to the first terminal, etc.

[0158] In some embodiments, determining the first information according to the first state comprises: inputting the first state into a first model, and determining the first information according to an output result of the first model; wherein the first model is trained by using a multi-agent deep deterministic policy gradient (MADDPG) algorithm.

[0159] The embodiment determines an efficient algorithm based on MADDPG to observe the states of the terminal, the edge server and the cloud server, and determines the strategy of the nodes to perform task offloading according to the states, and then allocates tasks to nodes with different computing capabilities to improve the overall efficiency of the network, thereby reducing the delay and energy consumption.

[0160] In some embodiments, each node in the plurality of nodes of the offloading environment has an agent (or can be referred to as an agent), and each agent has a model, which can be used to input the state of each node in the offloading environment to obtain the execution strategy of the current node for task offloading; accordingly, the above inputting the first state into the first model can comprise: inputting the first state into the first model corresponding to the agent of the terminal, and then determining the first information according to the output result of the first model.

[0161] In some embodiments, the model parameters of the first model can be determined according to the training results of the model of each agent. For example, the model parameters of the first model corresponding to the agent of the terminal can be determined according to the training results of the model of the agent of the edge server and the training results of the model of the agent of the cloud server. For example, each agent's model is based on an actor-critic algorithm, including a critic network and an actor network. When training the parameters of the critic network and the actor network of the first model, the execution actions of the actor can be evaluated by multiple critics, which come from the first model and the models of other agents.

[0162] In some embodiments, the reward indicator in the above MADDPG algorithm is determined according to the total time and the total energy consumed by each agent for task processing, and the reward indicator is used to determine the loss function corresponding to the first model. In some examples, the total time for task processing by each agent can include the task transmission time between agents, and the calculation time of the agent for the task, etc.

[0163] For example, the critic network can evaluate the task offloading actions made by the actor network according to the total time for task processing and the total energy consumed, the goal being to minimize the total time and minimize the total energy consumed, and to give a corresponding reward according to the degree of compliance. Through this process, the task offloading actions are continuously optimized to obtain the optimal strategy for executing task offloading.

[0164] In some examples, a 6G End-Edge-Cloud network consisting of U terminals, E edge servers, and C cloud servers is considered as an example. The task is generated from a resource-limited terminal. The demand exceeds the capacity of the terminal resources, and the terminal's demand may not be met. Therefore, it is necessary to coordinate the computing resources of edge servers and cloud servers to accommodate the demand. For the characteristics of 6G high bandwidth and low latency, the optimization framework and algorithm are determined to determine the appropriate offloading method for the 6G network. A distributed Markov process is used to convert the problem, and each node in the End-Edge-Cloud architecture is considered as an agent (or called as a proxy). A suitable multi-agent environment is created, and 6G scenario indicators such as task size, CPU cycles, computing power of each node, percentage of offloaded tasks, and communication rate are considered. This method allows effective collaboration and decision-making between agents and dynamic adjustment of operations based on changing states and system conditions. The multi-agent deep deterministic policy gradient (MADDPG) algorithm is optimized, and the action space and state space are optimized to better adapt to the environment. For the 6G network, a more fine-grained resource allocation operation (percentage allocation to cloud servers and edge servers) is designed. This helps agents better utilize the performance of the 6G network. This method improves the efficiency and effectiveness of the entire system while maintaining scalability and adaptability to changing conditions.

[0165] The embodiment proposes an algorithm based on deep reinforcement learning, which utilizes the multi-agent method in the 6G network terminal-edge-cloud architecture. Thus, the task offloading problem can be converted into a decentralized partially observable Markov decision process. An efficient MADDPG-based algorithm is designed to observe the states of the terminal, edge server, and cloud server, and the task can be systematically allocated to nodes with different computing capabilities to collectively improve the overall efficiency of the network, thereby reducing offloading delay and energy consumption.

[0166] FIG. 3 shows a flowchart of a task offloading method according to an embodiment of the present disclosure. It can be performed by the edge server side and can include the following steps.

[0167] Step S301, a second state of the offloading environment is obtained.

[0168] In some embodiments, the offloading environment includes a plurality of nodes, which can be one of a terminal, an edge server, a cloud server, etc.

[0169] In some examples, these nodes such as terminals, edge servers, and cloud servers have computing capabilities and can perform task computation processing.

[0170] In some examples, the task can be generated by the terminal and sent to the edge server by the terminal, or a task sent by other nodes. In some examples, the task can also be generated locally by the edge server according to actual needs, etc.

[0171] In some embodiments, the second state can be the state of each node obtained by the edge server from the offloading environment, can include the state of the edge server itself, and the state of the cloud server, etc.

[0172] In some embodiments, the second state includes at least one of the following A3 to I3:

[0173] A3, task information, such as the task size, task type, central processing unit (CPU) cycle number required for task calculation, time limit for obtaining task calculation result, etc. of the edge server task.

[0174] B3, locally available computing resources of the edge server, such as CPU resources, graphics processing unit (GPU) resources, random access memory (RAM) resources, storage resources, etc. available locally to the edge server, which can be used for processing of the task.

[0175] C3, computing power locally to the edge server, such as CPU computing rate, memory speed, storage read-write speed, etc. locally to the edge server.

[0176] D3, percentage of tasks that can be processed locally to the edge server, such as percentage of tasks that can be processed locally to the edge server, which can be determined according to A3, B3, and C3.

[0177] E3, computing power of the cloud server, such as CPU computing rate, memory speed, storage read-write speed, etc. of the cloud server.

[0178] F3, percentage of tasks that can be offloaded to the cloud server, such as percentage of tasks that can be offloaded to the cloud server for processing.

[0179] G3, computing power of the first edge server, which can be another edge server different from the current edge server, and the task can also be offloaded to the first edge server for calculation according to actual needs.

[0180] H3, percentage of tasks that can be offloaded to the first edge server.

[0181] I3, communication rate between nodes.

[0182] Step S302, determining the second information according to the second state.

[0183] In some embodiments, the second information is used to indicate a policy for the edge server to perform task offloading. In some examples, the edge server performs task offloading according to the policy indicated by the second information.

[0184] In some embodiments, the second information can include at least one of the following A4 to B4:

[0185] A4, an offloading destination of the task, used to determine a target node to which the task is offloaded.

[0186] B4, an offloading percentage of the task, used to determine a percentage of task offloading.

[0187] In some embodiments, the offloading destination of the task can include at least one of the following:

[0188] a cloud server; a first edge server; an edge server locally.

[0189] In some embodiments, the offloading percentage of the task can include at least one of the following:

[0190] a percentage of the task offloaded to the cloud server;

[0191] a percentage of the task offloaded to the first edge server;

[0192] a percentage of the task processed by the edge server locally.

[0193] In some embodiments, determining the second information according to the second state includes: inputting the second state into a second model, and determining the second information according to an output result of the second model; wherein the second model is obtained by training using a MADDPG algorithm.

[0194] The embodiment determines an efficient algorithm based on MADDPG to observe the states of the terminal, the edge server and the cloud server, and determines the policy for these nodes to perform task offloading according to the states, and then allocates tasks to nodes with different computing capabilities to improve the overall efficiency of the network, thereby reducing delay and energy consumption.

[0195] In some embodiments, each node of the plurality of nodes has an agent (or can be referred to as an agent); each agent has a model respectively, which can be used to input the state of each node in the offloading environment to obtain the execution policy of the current node for task offloading; correspondingly, the above inputting the second state into the second model can include: inputting the second state into the second model corresponding to the agent of the edge server, and then determining the second information according to the output result of the second model.

[0196] In some embodiments, the model parameters of the second model can be determined according to the training results of the model of each agent.

[0197] In some embodiments, the reward indicator in the MADDPG algorithm described above is determined according to a total time and a total energy of each agent for processing a task, and the reward indicator is used to determine a loss function corresponding to the second model. In some examples, the total time includes a task transmission time between agents and a calculation time of the agent for the task.

[0198] For specific descriptions of some examples in this embodiment, refer to the corresponding descriptions of the embodiments in FIG. 2, which are not repeated here.

[0199] Through the task offloading scheme provided in this embodiment, tasks can be systematically allocated to nodes with different computing capabilities to improve the overall efficiency of the network.

[0200] FIG. 4 shows a flowchart of a task offloading method according to an embodiment of the present disclosure. As shown in FIG. 4, the method can be performed by a cloud server side and can include the following steps.

[0201] In step S401, a third state of an offloading environment is obtained.

[0202] In some embodiments, the offloading environment includes a plurality of nodes, and the nodes are one of a terminal, an edge server, a cloud server, and the like.

[0203] In some examples, these nodes such as the terminal, the edge server, and the cloud server have computing capabilities and can perform task calculation processing.

[0204] In some examples, the task can be generated by the terminal and sent to the cloud server by the terminal, or sent to the cloud server by the edge server after being sent to the edge server by the terminal, or sent by other nodes such as the cloud server. In some examples, the task can also be generated locally by the cloud server according to actual needs, etc.

[0205] In some embodiments, the third state can be a state of each node obtained by the cloud server from the offloading environment, which can include a state of the cloud server itself, a state of other cloud servers, and the like, in addition to a state of the edge server and a state of the terminal, etc.

[0206] In some embodiments, the third state includes at least one of the following A5 to G5:

[0207] A5, task information; such as a task size, a task type, a number of central processing unit (CPU) cycles required for task calculation, a time limit for obtaining a task calculation result, and the like of a cloud server task.

[0208] B5, computing resources available locally to the cloud server, such as CPU resources, graphics processing unit (GPU) resources, memory (RAM) resources, storage resources, and the like available locally to the cloud server, which can be used for processing of the task.

[0209] C5, computing power local to the cloud server, such as CPU computing rate, memory speed, storage read-write speed, and the like local to the cloud server.

[0210] D5, percentage of tasks that can be processed locally by the edge server.

[0211] E5, computing power of a first cloud server, which can be another cloud server different from the current cloud server, and according to actual needs, the task can be offloaded to the first cloud server for computation.

[0212] F5, percentage of tasks that can be offloaded to the first cloud server.

[0213] G5, communication rate between nodes.

[0214] Step S402, determining third information according to the third state.

[0215] In some embodiments, the third information is used to indicate a strategy for the cloud server to perform task offloading. In some examples, the cloud server performs task offloading according to the strategy indicated by the third information.

[0216] In some embodiments, the third information includes at least one of the following A6 to B6:

[0217] A6, offloading destination of the task;

[0218] B6, offloading percentage of the task.

[0219] In some embodiments, the offloading destination of the task includes at least one of the following:

[0220] the first cloud server; and the cloud server locally.

[0221] In some embodiments, the offloading percentage of the task includes at least one of the following:

[0222] percentage of tasks offloaded to the first cloud server;

[0223] percentage of tasks processed locally by the cloud server.

[0224] In some embodiments, determining the third information according to the third state includes: inputting the third state into a third model, and determining the third information according to an output result of the third model; wherein the third model is trained using a MADDPG algorithm.

[0225] The embodiment determines an efficient algorithm based on MADDPG to observe the states of the terminal, edge server and cloud server, and determines the task offloading strategy of the nodes according to the states, and then allocates tasks to nodes with different computing capabilities to improve the overall efficiency of the network, thereby reducing the delay and energy consumption.

[0226] In some embodiments, each node in the plurality of nodes has an agent (or can be referred to as an agent), and each agent has a model, which can be used to input the state of each node in the offloading environment to obtain the execution strategy of the current node for task offloading; correspondingly, the above inputting the third state into the third model can include inputting the third state into the third model corresponding to the agent of the cloud server.

[0227] In some embodiments, the model parameters of the third model can be determined according to the model training results of each agent.

[0228] In some embodiments, the reward index in the above MADDPG algorithm is determined according to the total time and total energy of each agent for task processing, and the reward index is used to determine the loss function corresponding to the third model. In some examples, the total time includes the task transmission time between agents and the calculation time of the agent for the task.

[0229] The specific examples in the embodiment can be referred to the corresponding description of the embodiments in FIGS. 2-3, which will not be repeated here.

[0230] Through the task offloading scheme provided by the embodiment, tasks can be systematically allocated to nodes with different computing capabilities to improve the overall efficiency of the network.

[0231] The following is an exemplary introduction to the above-mentioned embodiment method, which can be applied to the following scenarios, but is not limited thereto:

[0232] The embodiment of the present disclosure proposes an algorithm based on deep reinforcement learning, which utilizes the multi-agent method in the 6G network terminal-edge-cloud architecture. Thus, the task offloading problem can be converted into a decentralized partially observable Markov decision process. An efficient algorithm based on MADDPG is designed to observe the state of the terminal, edge server and cloud server, thereby reducing the offloading delay and energy consumption. The method of the present embodiment aims to improve the overall efficiency of the network by systematically assigning tasks to nodes with different computing capabilities. That is, if the task cannot be completed locally on the terminal, the terminal will offload the task to the edge server or cloud server according to the selected strategy. Consider a 6G End-Edge-Cloud network composed of U terminals, E edge servers and C cloud servers. The task is generated from a resource-limited terminal. The demand exceeds the capacity of the terminal resources, and the demand of the terminal may not be met. Therefore, it is necessary to coordinate the computing resources of the edge server and the cloud server to adapt to the demand.

[0233] The present example is outlined as follows:

[0234] 1. For the characteristics of 6G high bandwidth and low latency, optimize the framework and algorithm, and explore the offloading method suitable for 6G network.

[0235] Use a distributed Markov process to convert the problem, and consider each node in the end-edge-cloud architecture as an agent.

[0236] 2. Create a suitable multi-agent environment and consider 6G scenario indicators such as task size, CPU cycles, computing power, offloaded task percentage and communication rate. This approach allows effective collaboration and decision-making between agents, as well as dynamic adjustment of operations based on changing states and system conditions.

[0237] 3. Optimize the MADDPG algorithm, optimize the action space and state space, and make it better adapt to the environment. For 6G networks, a more fine-grained resource allocation operation is designed (allocate to cloud servers and edge servers by percentage). This helps agents better utilize the performance of 6G networks. This approach improves the efficiency and effectiveness of the entire system while maintaining scalability and adaptability to changing conditions.

[0238] In some examples, as shown in FIG. 5, one of the scenarios involved in the present embodiment involves 6G task offloading involving a plurality of terminals (User Ends), a plurality of edge servers (Edge Servers) and a plurality of cloud servers (Cloud Servers).

[0239] For terminal-local computing: the computing rate of the terminal is quantified by the frequency of CPU cycles, denoted by f ldenotes the computation rate of the terminal CPU. The computation task can be done locally at the terminal with a local computation time t l = M CPU / f l , M CPU denotes the number of CPU cycles needed for the task to complete. The computation task will consume energy, which is given by: e l = a l * t l , where a l denotes the energy consumption of the terminal per second, which can be obtained by measurement.

[0240] For computation at the edge server: The computation task can be processed at the edge server with an edge computation time t e = M CPU / f e , where f e is the computation rate of the edge server CPU, which is determined by the performance of the edge server. Similarly, the edge server also has an energy cost e e = a e * t e , where a e denotes the energy consumption of the edge server per second. When the task is offloaded to the edge server, the terminal is considered to have no energy cost.

[0241] For computation at the cloud server: Unlike the terminal and the edge server, the cloud server has advanced computing performance. Therefore, the cloud server is considered to be able to process an unlimited number of tasks in parallel, and the computation time t c = M CPU / f c , where f c is the computation rate of the cloud server CPU. If the task is offloaded to the cloud server, the terminal and the edge server are also considered to have no energy consumption.

[0242] For transmission of task data: Time Division Multiple Access (TDMA) is used to complete the transmission of offloaded tasks. The transmission rate is defined as R 6G , which is defined as: R 6G = B t log2(1 + P i,j / N), in which B t is the bandwidth at time t, P i,j is the transmission power of node i to node j, and N is the background noise power.

[0243] The communication time cost from the terminal to the edge server is: t u,e = M u,e / R 6G , where M u,eM is the data that offloads the task from the terminal to the edge server. The communication time cost from the terminal to the cloud server is: t u,c = M u,e / R 6G + M e,c / R wired where M u,e is the data that offloads the task from the terminal to the edge server, M e,c represents the data that needs to be transmitted from the edge server to the cloud server. R wired represents the wired data transmission rate.

[0244] In the 6G scenario, a large number of tasks need to be executed in time, resulting in a large amount of computation that must be completed quickly. Therefore, time is the most critical factor, which can be divided into computation time and transmission time. In addition, the energy cost generated by the terminal and the edge server must also be considered in the system. The resource allocation problem in the 6G network mainly focuses on two key factors: 1) total time and 2) total energy.

[0245] 1) Total time T is a key indicator of offloading system performance, and lower time means better performance.

[0246] P1: Min T = ∑(t l + t e + t c ) + 2*∑(t u,e + t e,c )

[0247] t l represents the computation time of the terminal, t e represents the computation time of the edge server, t c represents the computation time of the cloud server, ∑(t l + t e + t c ) represents the sum of the computation times of the nodes in the offloading environment; t u,e represents the communication time from the terminal to the edge server, t e,c represents the communication time from the edge server to the cloud server, 2*∑(t u,e + t e,c ) represents the sum of the communication times between the nodes in the offloading environment. The goal of P1 is to achieve the minimum total time T.

[0248] On the other hand, total energy E represents the cost required to achieve such performance.

[0249] P2: Min E = ∑(e l + e e )

[0250] e ldenotes the energy consumed by the terminal computing task, e e denotes the energy consumed by the edge server computing task, ∑(e l +e e ) denotes the sum of the energy consumed by the computing tasks of each node in the offloading environment, where the energy consumed by the cloud server can be ignored, and P2 aims to minimize the total energy E.

[0251] In 6G networks, it is crucial to optimize these two objectives to ensure efficient resource utilization. Task offloading in the End-Edge-Cloud network is a major challenge, as it requires choosing between edge servers or cloud servers to collaboratively complete tasks while optimizing time and energy consumption. To address this issue, the present embodiment can employ a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm: MADDPG. How to use MADDPG to effectively solve the optimization problem of resource allocation in 6G networks will be introduced below.

[0252] First, the problem is converted into a decentralized Markov Decision Process (MDP), and second, the detailed logic of the MADDPG algorithm, which will be used to optimize task offloading, is introduced. However, in the End-Edge-Cloud offloading scenario, tasks generated by different terminals must be completed simultaneously. Therefore, MADRL is a better choice. In a multi-agent environment, agents make decisions based on the joint behavior of all agents, resulting in a globally optimal decision rather than a locally optimal decision. The resource allocation scenario involves time-varying parameters and a series of possible operations that can be executed continuously. Therefore, the optimization of the resource allocation problem can be restructured as a decentralized Partially Observable Markov Decision Process (POMDP) represented by the tuple (M, S, O, A, P, R). Here, M refers to the multiple agents involved, such as the agents of terminals, edge servers, and cloud servers. The set S represents the offloading state, which contains all agents. O represents the observation space, which contains several features of the agents. A is the set of actions that agents can use to allocate tasks. The joint transition is represented by P, and the joint reward is represented by R. Detailed descriptions of the components of the decentralized POMDP are provided below, with a focus on task offloading and resource allocation.

[0253] Agent: In the End-Edge-Cloud scenario, each terminal has an agent that is responsible for making decisions on task allocation and target selection. The MADDPG algorithm is used to train the models corresponding to these agents. In this approach, each agent has its own observation and action space. Nonetheless, agents must work together to achieve good rewards. They can get updates on policies from both locally and other agents.

[0254] State: The state space has the characteristics of all the agents, such as where Ta represents the computational resources required to generate a task, CPU cycle is the number of CPU cycles consumed by a task. The variable represents the percentage of tasks that are offloaded to the edge server, represents the percentage of tasks that are offloaded to the cloud server, represents the percentage of tasks that are processed locally on the terminal without offloading.

[0255] Observations: Each agent must obtain the necessary information to make informed decisions. This includes data such as task size, CPU cycles, computational capacity, percentage of offloaded tasks, and communication rates, among others, as outlined in at least one of the aforementioned A1 to I1. Based on this information, the agent generates observations and iterations to facilitate the decision-making process. Define

[0256] Action: The terminal should select an action from the action space by interacting with the environment to execute the offloading decision, then use the data generated from the interaction to train the model. Define where Target refers to the destination of the offloaded task. In the 6G offloading scenario, the agent is given three targets: 1) edge server; 2) cloud server; 3) terminal locally. Due to the large size of the task, the task will not be offloaded in a binary manner. Therefore, the offloading operation needs to specify the target and percentage of the task to be offloaded.

[0257] Reward: The goal is to minimize the total time and energy consumption. To fit the decentralized POMDP, the problem is reformulated as a Q function: MAX Q = θ i / T + (1 - θ i) / E, i.e., the optimization objective of the Q function maximization. This function consists of two parts: one represents the time cost, and the other represents the energy cost. In order to meet the requirements of deep reinforcement learning, the inverse of the Q function is multiplied by a coefficient θ i , which ranges from 0 to 1.

[0258] Deep Deterministic Policy Gradient (DDPG) algorithm and Deep Q-Learning (DQN) algorithm can be used for 6G task offloading, but these algorithms may not be suitable for multi-objective joint optimization. Multi-agent method is a good solution for joint optimization and can take advantage of DDPG, which has continuous action and state space. MADDPG is an effective and suitable algorithm for 6G resource allocation and task offloading. As shown in FIG. 6, the offloading algorithm based on MADDPG can be divided into two categories: environment and agent.

[0259] As shown in FIG. 6, multiple DDPG agents are used to work together to improve the performance of the entire system by interacting with the environment. The agent obtains observations from the environment and decides actions. The agent part can be divided into two parts: 1) critic network and 2) actor network. The critic-actor method is to update the policy, which is equivalent to each actor, and multiple critics judge it, which is equivalent to multiple judges. The actor network can learn and optimize the policy independently, and the critic network estimates the Q function according to the reward of the agent in the current state. The experience replay mechanism is an important design of MADDPG, and the experience obtained by the agent during model training is stored in a replay buffer, and the agent can sample from the replay buffer to improve learning efficiency, speed up convergence, and prevent boundary overfitting. In particular, this algorithm uses multi-agent experience replay, which means that the experience of all agents can be used to improve the learning efficiency of the entire system.

[0260] The loss function of the model can be divided into two parts: 1) actor network and 2) critic network. The actor network is responsible for updating the policy, and the critic network is responsible for optimizing the Q function.

[0261] The loss function of the actor network can be as shown in Equation One:

[0262] In Equation One, N represents the number of samples, and Q(o, a) represents the expected return (or Q value) of taking action (a) under given observation (o).

[0263] The loss function of the critic network can be shown in Equation 2:

[0264] In Equation 2, N represents the number of samples, Q(o, a) represents the expected return (or Q value) of taking action (a) under a given observation (o), and y i represents the target value.

[0265] The total loss function can be shown in Equation 3: Loss = Loss actor + Loss critic (Equation 3)

[0266] The MADRL scheme and algorithm proposed in the embodiments of the present disclosure can significantly improve the delay and energy consumption. Based on the scheme proposed in the embodiments of the present disclosure, in order to better allocate resources, a task offloading scheme in a 6G network scenario is determined. A combined optimization problem of minimizing the transmission delay and energy at the same time is formulated as a POMDP optimization problem. A MADRL scheme is proposed to realize efficient interaction between the cloud server, the edge server and the terminal, accelerate the convergence speed of the model, and optimize the performance.

[0267] The embodiments of the present disclosure also propose an apparatus for implementing any of the above methods, for example, an apparatus including units or modules for implementing each step performed by the terminal in any of the above methods. For another example, another apparatus is also proposed, including units or modules for implementing each step performed by the edge server in any of the above methods. For another example, still another apparatus is also proposed, including units or modules for implementing each step performed by the cloud server in any of the above methods.

[0268] It should be understood that the division of each unit or module in the above apparatus is only a logical function division, and all or part of them can be integrated into a physical entity or physically separated in actual implementation. In addition, the units or modules in the apparatus can be implemented in the form of processor calling software: for example, the apparatus includes a processor connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit or module of the above apparatus, wherein the processor is, for example, a general processor such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the apparatus or a memory outside the apparatus. Alternatively, the units or modules in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units or modules can be realized by the design of the hardware circuit. The above hardware circuit can be understood as one or more processors; for example, in one implementation, the above hardware circuit is an application-specific integrated circuit (ASIC), and the functions of part or all of the above units or modules are realized by the design of the logical relationship of the elements in the circuit; for example, in another implementation, the above hardware circuit is a programmable logic device (PLD), and a field programmable gate array (FPGA) is taken as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the above units or modules. All units or modules of the above apparatus can be implemented in the form of processor calling software, or all units or modules can be implemented in the form of hardware circuit, or part of the units or modules are implemented in the form of processor calling software, and the remaining part is implemented in the form of hardware circuit.

[0269] In the embodiments of the present disclosure, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processor can implement certain functions through a logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the configuration of the hardware circuit. It can be understood that the processor loads instructions to implement the functions of part or all of the units or modules described above. In addition, the hardware circuit can also be designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like.

[0270] FIG. 7 is a structural schematic diagram of a terminal according to an embodiment of the present disclosure. As shown in FIG. 7, the terminal can include a processing module 51. In some embodiments, the processing module 51 is configured to perform at least one of the communication steps performed by the terminal in any of the above methods (for example, steps S201 to S202, but not limited thereto), and details are not described herein again.

[0271] FIG. 8 is a structural schematic diagram of an edge server according to an embodiment of the present disclosure. As shown in FIG. 8, the edge server can include a processing module 61. In some embodiments, the processing module 61 is configured to perform at least one of the communication steps performed by the edge server in any of the above methods (for example, steps S301 to S302, but not limited thereto), and details are not described herein again.

[0272] FIG. 9 is a structural schematic diagram of a cloud server according to an embodiment of the present disclosure. As shown in FIG. 9, the edge server can include a processing module 71. In some embodiments, the processing module 71 is configured to perform at least one of the communication steps performed by the cloud server in any of the above methods (for example, steps S401 to S402, but not limited thereto), and details are not described herein again.

[0273] In some embodiments, the processing module can be one module, or can include multiple sub-modules. Optionally, the multiple sub-modules perform all or part of the steps required to be performed by the processing module, respectively. Optionally, the processing module can be mutually replaced with the processor.

[0274] FIG. 10 is a structural schematic diagram of a communication device 8100 according to an embodiment of the present disclosure. The communication device 8100 can be a network device such as an edge server or a cloud server, or a terminal such as a user equipment, or a chip, chip system, or processor supporting the implementation of the above-mentioned methods by the network device or terminal, and can also be a chip, chip system, or processor supporting the implementation of the above-mentioned methods by the terminal. The communication device 8100 can be used to implement the methods described in the above method embodiments, and specific implementation can be referred to the descriptions in the above method embodiments.

[0275] As shown in FIG. 10, the communication device 8100 includes one or more processors 8101. The processor 8101 can be a general-purpose processor or a special-purpose processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute programs, and process data of the programs. Optionally, the communication device 8100 is configured to perform any of the above methods. Optionally, the one or more processors 8101 are configured to invoke instructions to cause the communication device 8100 to perform any of the above methods.

[0276] In some embodiments, the communication device 8100 further includes one or more transceivers 8102. When the communication device 8100 includes one or more transceivers 8102, the transceiver 8102 performs the communication steps such as transmitting and / or receiving in the above methods, and the processor 8101 performs at least one of the other steps. In optional embodiments, the transceiver can include a receiver and / or a transmitter, which can be separate or integrated together. Optionally, the terms transceiver, transceiving unit, transceiver, transceiving circuit, interface circuit, interface, etc. can be mutually replaced, and the terms transmitter, transmitting unit, transmitter, transmitting circuit, etc. can be mutually replaced, and the terms receiver, receiving unit, receiver, receiving circuit, etc. can be mutually replaced.

[0277] In some embodiments, the communication device 8100 further includes one or more memories 8103 for storing data. Alternatively, all or part of the memories 8103 can be external to the communication device 8100. In optional embodiments, the communication device 8100 can include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102, and the interface circuit 8104 can be used to receive data from the memory 8102 or other devices, and can be used to send data to the memory 8102 or other devices. For example, the interface circuit 8104 can read data stored in the memory 8102 and send the data to the processor 8101.

[0278] The communication device 8100 described in the above embodiments can be a network device or a terminal, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 can not be limited by Figure 10. The communication device can be a standalone device or can be part of a larger device. For example, the communication device can be: 1) a standalone integrated circuit (IC), or a chip, or a chip system or subsystem; (2) a set of one or more ICs, which can optionally include a storage component for storing data, programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, a smart terminal device, a cellular phone, a wireless device, a handset, a mobile unit, a vehicle-mounted device, a network device, a cloud device, an artificial intelligence device, etc.; (6) other devices, etc.

[0279] Figure 11 is a structural schematic diagram of a chip 8200 according to an embodiment of the present disclosure. For the case where the communication device 8100 is a chip or a chip system, the structural schematic diagram of the chip 8200 shown in Figure 11 can be referred to, but is not limited thereto.

[0280] The chip 8200 includes one or more processors 8201. The chip 8200 is configured to execute any of the above methods.

[0281] In some embodiments, the chip 8200 further includes one or more interface circuits 8202. Optionally, the terms interface circuit, interface, transceiver pin, etc. can be replaced by each other. In some embodiments, the chip 8200 further includes one or more memories 8203 for storing data. Optionally, all or part of the memories 8203 can be external to the chip 8200. Optionally, the interface circuit 8202 is connected to the memory 8203, and the interface circuit 8202 can be used to receive data from the memory 8203 or other devices, and the interface circuit 8202 can be used to send data to the memory 8203 or other devices. For example, the interface circuit 8202 can read data stored in the memory 8203 and send the data to the processor 8201.

[0282] In some embodiments, the interface circuit 8202 performs at least one of the communication steps such as transmitting and / or receiving in the above-described methods. The interface circuit 8202 performing the communication steps such as transmitting and / or receiving in the above-described methods refers to, for example, the interface circuit 8202 performing data interaction between the processor 8201, the chip 8200, the memory 8203, or a transceiver device. In some embodiments, the processor 8201 performs at least one of the above-described task offloading method steps.

[0283] The modules and / or devices described in each of the embodiments of the virtual device, the physical device, the chip, etc. can be combined or separated according to the situation. Optionally, part or all of the steps can also be performed by a plurality of modules and / or devices in cooperation, which is not limited here.

[0284] The disclosure further proposes a storage medium having instructions stored thereon, which, when executed on the communication device 8100, causes the communication device 8100 to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited to this, and it can also be a storage medium readable by other devices. Optionally, the storage medium can be a non-transitory storage medium, but is not limited to this, and it can also be a transitory storage medium.

[0285] The disclosure further proposes a program product which, when executed by the communication device 8100, causes the communication device 8100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0286] The disclosure further proposes a computer program which, when executed on a computer, causes the computer to perform any of the above methods.

Claims

A task offloading method characterized by comprising: The method is executed by a terminal side, and comprises: obtaining a first state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determining first information according to the first state, the first information being used to indicate a strategy of the terminal for performing task offloading. The method of claim 1, wherein The first information comprises at least one of: an offloading destination of a task; an offloading percentage of the task. The method according to claim 2, characterized in that The offloading destination comprises at least one of: an edge server; a cloud server; a terminal. The method according to any one of claims 2 to 3, characterized in that The offloading percentage comprises at least one of: a percentage of the task offloaded to the edge server; a percentage of the task offloaded to the cloud server; a percentage of the task processed locally by the terminal. The method according to any one of claims 1 to 4, characterized in that The first state comprises at least one of: task information; computing resources available locally to the terminal; computing capability of the terminal locally; a percentage of tasks processable locally to the terminal; computing capability of the edge server; a percentage of tasks offloadable to the edge server; computing capability of the cloud server; a percentage of tasks offloadable to the cloud server; a communication rate between the nodes. The method according to any one of claims 1 to 5, characterized in that Determining the first information according to the first state comprises: inputting the first state into a first model, and determining the first information according to an output result of the first model; wherein the first model is trained by using a multi-agent deep deterministic policy gradient (MADDPG) algorithm. The method according to claim 6, characterized in that Each of the plurality of nodes has an agent; inputting the first state into the first model comprises: inputting the first state into the first model corresponding to the agent of the terminal. The method of claim 7, wherein Model parameters of the first model are determined according to model training results of each agent. The method according to any one of claims 7 to 8, characterized in that A reward index in the MADDPG algorithm is determined according to a total time and a total energy of each agent for processing a task, and the reward index is used to determine a loss function corresponding to the first model. The method of claim 9, wherein The total time comprises a task transmission time between the agents and a computing time of the agents for the task. A task offloading method characterized by comprising: The method is executed by an edge server side, and comprises: obtaining a second state of an offloading environment, the offloading environment comprising a plurality of nodes, the nodes being one of a terminal, an edge server and a cloud server; determining second information according to the second state, the second information being used to indicate a strategy of the edge server for performing task offloading. The method of claim 11, wherein The second information comprises at least one of: an offloading destination of a task; an offloading percentage of the task. The method of claim 12, wherein The offloading destination comprises at least one of: a cloud server; a first edge server; the edge server locally. The method according to any one of claims 12 to 13, characterized in that The offloading percentage comprises at least one of: a percentage of the task offloaded to the cloud server; a percentage of the task offloaded to the first edge server; a percentage of the task processed locally by the edge server. The method according to any one of claims 11 to 14, characterized in that The second state comprises at least one of: task information; computing resources available locally to the edge server; computing capability of the edge server locally; a percentage of tasks processable locally to the edge server; computing capability of the cloud server; a percentage of tasks offloadable to the cloud server; computing capability of the first edge server; a percentage of tasks offloadable to the first edge server; a communication rate between the nodes. The method according to any one of claims 11 to 15, characterized in that The second information is determined according to the second state, including: The second state is input into a second model, and the second information is determined according to an output result of the second model; The second model is obtained by training using a multi-agent deep deterministic policy gradient (MADDPG) algorithm. The method of claim 16, wherein Each of the plurality of nodes has an agent; The second state is input into the second model corresponding to the agent of the edge server. The model parameters of the second model are determined according to the model training result of each agent. The method of claim 17, wherein The reward index in the MADDPG algorithm is determined according to the total time and total energy of each agent for processing the task, and the reward index is used to determine a loss function corresponding to the second model. The method according to any one of claims 17 to 18, characterized in that The total time includes the task transmission time between agents and the calculation time of the agent for the task. The method of claim 19, wherein The method is executed by a cloud server side, and the method includes: A task offloading method characterized by comprising: A third state of an offloading environment is obtained, the offloading environment includes a plurality of nodes, and the nodes are one of a terminal, an edge server, and a cloud server. Third information is determined according to the third state, and the third information is used to indicate a strategy of the cloud server for performing task offloading. The third information includes at least one of the following: The method of claim 21, wherein An offloading destination of the task; An offloading percentage of the task. The offloading destination includes at least one of the following: The method of claim 22, wherein A first cloud server; The cloud server locally. The offloading percentage includes at least one of the following: The method according to any one of claims 22 to 23, characterized in that A percentage of the task offloaded to the first cloud server; A percentage of the task processed by the cloud server locally. The third state includes at least one of the following: The method according to any one of claims 21 to 24, characterized in that Task information; Computing resources available to the cloud server locally; Computing capability of the cloud server locally; A percentage of tasks that can be processed by the edge server locally; Computing capability of the first cloud server; A percentage of tasks that can be offloaded to the first cloud server; A communication rate between nodes. The third information is determined according to the third state, including: The method according to any one of claims 21 to 25, characterized in that The third state is input into a third model, and the third information is determined according to an output result of the third model; The third model is obtained by training using a multi-agent deep deterministic policy gradient (MADDPG) algorithm. Each of the plurality of nodes has an agent; The method of claim 26, wherein The third state is input into the third model corresponding to the agent of the cloud server. The model parameters of the third model are determined according to the model training result of each agent. The reward index in the MADDPG algorithm is determined according to the total time and total energy of each agent for processing the task, and the reward index is used to determine a loss function corresponding to the third model. The method of claim 27, wherein The total time includes the task transmission time between agents and the calculation time of the agent for the task. The method according to any one of claims 27 to 28, characterized in that The method includes: The method of claim 29, wherein A processing module is configured to obtain a first state of an offloading environment, the offloading environment includes a plurality of nodes, and the nodes are one of a terminal, an edge server, and a cloud server; and determine first information according to the first state, the first information is used to indicate a strategy of the terminal for performing task offloading. A terminal, characterized by comprising: The method includes: ​ An edge server, characterized in that ​ The processing module is configured to acquire a second state of an offloading environment, the offloading environment including a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine second information according to the second state, the second information being used to indicate a strategy of the edge server performing task offloading. A cloud server, characterized in that, Comprise: The processing module is configured to acquire a third state of an offloading environment, the offloading environment including a plurality of nodes, the nodes being one of a terminal, an edge server, and a cloud server; and determine third information according to the third state, the third information being used to indicate a strategy of the cloud server performing task offloading. A task offloading system characterized by, Comprise: The terminal is configured to implement the method in any one of claims 1 to 10; The edge server is configured to implement the method in any one of claims 11 to 20; The cloud server is configured to implement the method in any one of claims 21 to 30. A communication device characterized by comprising: Comprise: One or more processors; The processor is used to execute the method in any one of claims 1 to 30. A computer storage medium, wherein, The computer storage medium stores computer executable instructions; the computer executable instructions are executed by the processor, and can implement the method in any one of claims 1 to 30. A computer program product comprises a computer program, the computer program is executed by the processor, and can implement the method in any one of claims 1 to 30.

Images