Task offloading method and apparatus, and storage medium

Abstract

A task offloading method, the method comprising: acquiring task information of a task to be processed of a terminal device and location information of the terminal device; on the basis of the location information and mobility of the terminal device, determining a target network device, the target network device being configured with an edge server; and allocating the task to be processed to the terminal device and the edge server to undergo processing.

Description

Task unloading methods, devices and storage media

[0001] Cross-reference of related applications

[0002] This application claims priority to Chinese Patent Application No. 202510228950.8, filed on February 27, 2025, entitled “Task Unloading Method, Apparatus and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of information technology, and in particular to a task unloading method, apparatus and storage medium. Background Technology

[0004] With the continuous advancement of information technology, the amount of data that terminal devices need to process has increased dramatically. However, most terminal devices have limited computing power. Relying solely on the local computing resources of the terminal device to execute all tasks will result in significant execution latency, which cannot meet latency requirements. Summary of the Invention

[0005] The technical solution adopted in this application is as follows:

[0006] In a first aspect, this application provides a task offloading method, which includes: obtaining task information of a task to be processed by a terminal device and location information of the terminal device; determining a target network device based on the location information and mobility of the terminal device, wherein the target network device is configured with an edge server; and allocating the task to be processed to the terminal device and the edge server for processing.

[0007] In one possible implementation, the task information includes the task deadline delay; wherein, the processing time of the terminal device is determined based on the terminal device's computing speed and computing frequency; and the processing time of the edge server is determined based on the edge server's computing speed, the computing resources allocated by the edge server to the task to be processed, and the data transmission rate between the terminal device and the target network device.

[0008] In one possible implementation, the data transmission rate between the terminal device and the target network device is determined based on the channel bandwidth of the edge server, the transmission power of the terminal device, the channel gain between the terminal device and the target network device, and the distance between the terminal device and the target network device.

[0009] In one possible implementation, the data transmission rate between the terminal device and the target network device satisfies the following formula:

[0010] Where W represents the channel bandwidth of the edge server, P i，trans h represents the transmission power of the terminal device.i Dis represents the channel gain between the terminal device and the target network device. ij β represents the distance between the terminal device and the target network device, N0 represents the Gaussian white noise power, and I represents the interference power.

[0011] In one possible implementation, there are multiple terminal devices, and the method further includes: for each terminal device, determining the task processing time corresponding to each terminal device, wherein the task processing time is the maximum time for the edge server to process the pending tasks of the terminal device; and based on the task processing time corresponding to the terminal device, determining the minimum computing resources required for the edge server to process the pending tasks of the terminal device.

[0012] In one possible implementation, the method further includes: if the sum of the minimum computing resources required by the edge server to process the pending tasks of each terminal device is less than or equal to the total computing resources of the edge server, then it is determined that the edge server is capable of processing the pending tasks of the multiple terminal devices; if the sum of the minimum computing resources required to process the pending tasks of each terminal device is greater than the total computing resources of the edge server, then it is determined that the pending tasks of the multiple terminal devices will be processed according to the priority of the terminal devices.

[0013] In one possible implementation, the task to be processed is assigned to terminal devices and edge servers for processing, including: determining the allocation coefficient based on the task information of the task to be processed and the allocation coefficient decision algorithm; and assigning the task to be processed to terminal devices and edge servers for processing based on the allocation coefficient.

[0014] Secondly, this application provides a task offloading device, which includes: a communication unit and a processing unit; the communication unit is used to acquire task information of the task to be processed by the terminal device and the location information of the terminal device; the processing unit is used to determine the target network device based on the location information and mobility of the terminal device, wherein the target network device is configured with an edge server; the processing unit is also used to allocate the task to be processed to the terminal device and the edge server for processing.

[0015] In one possible implementation, the task information includes the task deadline delay; wherein, the processing time of the terminal device is determined based on the terminal device's computing speed and computing frequency; and the processing time of the edge server is determined based on the edge server's computing speed, the computing resources allocated by the edge server to the task to be processed, and the data transmission rate between the terminal device and the target network device.

[0016] In one possible implementation, the data transmission rate between the terminal device and the target network device is determined based on the channel bandwidth of the edge server, the transmission power of the terminal device, the channel gain between the terminal device and the target network device, and the distance between the terminal device and the target network device.

[0017] In one possible implementation, the data transmission rate between the terminal device and the target network device satisfies the following formula:

[0018] Where W represents the channel bandwidth of the edge server, P i，trans h represents the transmission power of the terminal device. i Dis represents the channel gain between the terminal device and the target network device. ij β represents the distance between the terminal device and the target network device, N0 represents the Gaussian white noise power, and I represents the interference power.

[0019] In one possible implementation, there are multiple terminal devices. For each terminal device, the processing unit is further configured to determine the task processing time corresponding to each terminal device. The task processing time is the maximum time for the edge server to process the pending tasks of the terminal device. The processing unit is further configured to determine the minimum computing resources required for the edge server to process the pending tasks of the terminal device based on the task processing time corresponding to the terminal device.

[0020] In one possible implementation, if the sum of the minimum computing resources required by the edge server to process the pending tasks of each terminal device is less than or equal to the total computing resources of the edge server, the processing unit is further configured to determine that the edge server is capable of processing the pending tasks of the multiple terminal devices; if the sum of the minimum computing resources required to process the pending tasks of each terminal device is greater than the total computing resources of the edge server, the processing unit is further configured to determine that the pending tasks of the multiple terminal devices are processed according to the priority of the terminal devices.

[0021] In one possible implementation, the processing unit is further configured to determine the allocation coefficient based on the task information of the task to be processed and the allocation coefficient decision algorithm; the processing unit is further configured to allocate the task to be processed to the terminal device and the edge server for processing based on the allocation coefficient.

[0022] Thirdly, this application provides a task offloading apparatus, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the task offloading method as described in the first aspect and any possible implementation thereof.

[0023] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the task offloading method as described in the first aspect and any possible implementation thereof.

[0024] Fifthly, this application provides a computer program product containing instructions that, when run on a task unloading device, cause the task unloading device to perform the task unloading method as described in the first aspect and any possible implementation thereof.

[0025] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the task offloading method as described in the first aspect and any possible implementation thereof.

[0026] Specifically, the chip provided in this application also includes a memory for storing computer programs or instructions. Attached Figure Description

[0027] Figure 1 is a schematic diagram of the composition of a task unloading device provided in an embodiment of this application;

[0028] Figure 2 is a flowchart of a task unloading method provided in an embodiment of this application;

[0029] Figure 3 is a flowchart of another task unloading method provided in an embodiment of this application;

[0030] Figure 4 is a flowchart of another task unloading method provided in an embodiment of this application;

[0031] Figure 5 is a schematic diagram of a task unloading device provided in an embodiment of this application;

[0032] Figure 6 is a schematic diagram of a communication network structure provided in an embodiment of this application;

[0033] Figure 7 is a schematic diagram of a reward value curve provided in an embodiment of this application;

[0034] Figure 8 is a schematic diagram of a task execution success rate provided in an embodiment of this application;

[0035] Figure 9 is a schematic diagram of another reward value curve provided in an embodiment of this application;

[0036] Figure 10 is a schematic diagram of another task execution success rate provided in an embodiment of this application;

[0037] Figure 11 is a schematic diagram of the structure of a task unloading device provided in an embodiment of this application. Detailed Implementation

[0038] The task unloading method, apparatus, and storage medium provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0039] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0040] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0041] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0042] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0043] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0044] With the continuous advancement of information technology, the amount of data that terminal devices need to process has increased dramatically. In industrial application scenarios, the number of computationally intensive and time-sensitive tasks is also growing significantly. However, since most terminal devices have limited computing power, relying solely on the local computing resources of the terminal device to execute all tasks will result in significant execution latency, failing to meet latency requirements.

[0045] Currently, deploying computing resources at the network edge through mobile edge computing (MEC) technology effectively alleviates the shortage of computing resources in terminal devices. MEC technology has become a common means of mitigating the problem of insufficient computing resources for terminal devices in current industrial scenarios.

[0046] However, existing task offloading solutions do not consider the mobility characteristics of terminal devices. In real-world industrial applications, the physical location of terminal devices is not fixed. Some terminal devices exhibit mobility, such as intelligent inspection robots, material handling carts, and unmanned delivery vehicles; their physical locations change over time. This mobility can impact the access and handover process between the terminal device and the base station, introducing additional latency or causing connection failures, thus leading to frequent task offloading failures for the terminal device.

[0047] In view of this, embodiments of this application provide a task offloading method, which includes: obtaining task information of a task to be processed on a terminal device and location information of the terminal device; determining a target network device based on the location information and mobility of the terminal device; and assigning the task to be processed to an edge server configured on the terminal device and the target network device for processing. Since the target network device is determined based on the location information and mobility of the terminal device, the connection success rate between the terminal device and the target network device is high, and the task offloading success rate is also high. In other words, the task offloading method provided in this application takes into account the mobility characteristics of the terminal device, thereby improving the success rate of task offloading by the terminal device.

[0048] For example, FIG1 is a schematic diagram of the composition of a task unloading device 10 provided in an embodiment of this application. As shown in FIG1, the task unloading device 10 may include a processor 101 and a bus 102.

[0049] Furthermore, the task unloading device 10 may also include a communication interface 103 and a memory 104. The processor 101, the memory 104, and the communication interface 103 can be connected via a bus 102.

[0050] The processor 101 can be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 101 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.

[0051] Bus 102 is used to transmit information between the components included in the task unloading device 10.

[0052] Communication interface 103 is used to communicate with other devices or other communication networks. These other communication networks can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Communication interface 103 can be a module, circuit, communication interface, or any device capable of enabling communication.

[0053] Memory 104 is used to store instructions. These instructions can be computer programs.

[0054] The memory 104 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., without limitation.

[0055] It should be noted that the memory 104 can exist independently of the processor 101 or can be integrated with the processor 101. The memory 104 can be used to store instructions, program code, or some data, etc. The memory 104 can be located inside or outside the task unloading device 10, without limitation. The processor 101 is used to execute the instructions stored in the memory 104 to implement the task unloading method provided in the following embodiments of this application.

[0056] In one example, processor 101 may include one or more CPUs, such as CPU0 and CPU1 (not shown in the figure).

[0057] As an optional implementation, the task offloading device 10 includes multiple processors.

[0058] As an optional implementation, the task offloading device 10 also includes output devices and input devices. For example, the input device is a device such as a keyboard, mouse, microphone or joystick, and the output device is a device such as a display screen or speaker.

[0059] It should be noted that the task offloading device 10 can be a desktop computer, a laptop computer, a web server, a mobile phone, a tablet computer, a wireless terminal, an embedded device, a chip system, or a device with a similar structure to that in Figure 1. Furthermore, the composition shown in Figure 1 does not constitute a limitation on the various devices in Figure 1. In addition to the components shown in Figure 1, each device in Figure 1 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0060] In this embodiment of the application, the chip system may be composed of chips or may include chips and other discrete devices.

[0061] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0062] The task unloading method provided in the embodiments of this application is described below with reference to the accompanying drawings. The actions, terminology, etc., involved in the various embodiments of this application can be referred to mutually without limitation. The message names or parameter names in the messages exchanged between various devices in the embodiments of this application are merely examples; other names may be used in specific implementations without limitation. The actions involved in the various embodiments of this application are merely examples; other names may be used in specific implementations. For example, "included in" in the embodiments of this application can be replaced with "carried on" or "carried in," etc.

[0063] As shown in Figure 2, this application embodiment proposes a task unloading method, which includes:

[0064] S201. Obtain the task information of the pending task of the terminal device and the location information of the terminal device.

[0065] The terminal equipment includes fixed terminal equipment and mobile terminal equipment. Fixed terminal equipment is not mobile, while mobile terminal equipment is mobile.

[0066] In one possible implementation, the task information of the tasks to be processed by the terminal device and the location information of the terminal device are obtained in each time slot, with a preset time slot as the unit time period.

[0067] For example, the task information of the pending tasks of the aforementioned terminal device may include the task quantity and the task deadline. For instance, the task information of the pending tasks can be represented as D = {d, ddl}, where d represents the task quantity in bits, and ddl represents the task deadline. In the t-th time slot, the task information of the pending tasks of the terminal device can be represented as d... t and DDLt .

[0068] It should be noted that in each time slot, the terminal device will generate a task to be processed with a preset probability. The generation of the task to be processed follows a Poisson distribution, and a random task deadline of less than 100 milliseconds is assigned to the task to be processed.

[0069] S202. Determine the target network device based on the location information and mobility of the terminal device.

[0070] The target network device is equipped with an edge server.

[0071] In one possible implementation, when the terminal device is a fixed terminal device (i.e., not mobile), the location information of the terminal device is used to determine which network device's coverage area the terminal device belongs to, and that network device is identified as the target network device. When the terminal device is a mobile terminal device (i.e., mobile), the location information of the terminal device is used to determine the network device closest to the terminal device, and that closest network device is identified as the target network device.

[0072] For example, taking an active terminal device as an example, assume that the active terminal device only moves horizontally when it moves. The initial position information of the active terminal device i can be represented as (x i y i In the t-th time slot, the location of active terminal device i is updated to... in, Let represent the x-coordinate of active terminal device i at the end of the previous time slot, v represent the velocity of active terminal device i in the t-th time slot, and Δt represent the duration of the time slot. The location information of network devices can be represented as (X... Bj Y Bj ), j = 0, 1, 2, ..., where j represents the network device number. Calculate the distance between the active terminal and each network device. And determine the network device j = argmin that is closest to the active terminal device i. j dis ij If network device j is the target network device corresponding to the active terminal device i in the current time slot, and the active terminal device i is within the coverage area of ​​the target network device j.

[0073] Optionally, the aforementioned network equipment can be a base station, which is configured with an edge server.

[0074] S203. Distribute the tasks to be processed to terminal devices and edge servers for processing.

[0075] In one possible implementation, the task information of the task to be processed is represented as D = {d, ddl}. The task is allocated to terminal devices for local processing at a ratio of α, and to edge servers for processing at a ratio of 1-α. Here, α can be called the allocation coefficient. Specifically, the processing flow of the task to be processed includes the following three parts.

[0076] 1-1. The terminal device processes some of the pending tasks locally.

[0077] Specifically, in the t-th time slot, the amount of work processed locally by terminal device i can be expressed as: It can be obtained according to the following formula 1.

[0078] Where α represents the proportion of tasks to be processed that are allocated to terminal devices for processing. This indicates the number of tasks to be processed.

[0079] The processing time of terminal device i on a local basis can be expressed as: It can be determined based on the computing speed and frequency of the terminal device i. Formula 2 can be satisfied.

[0080] in, This represents the amount of work processed locally by terminal device i. i This represents the number of CPU cycles (i.e., processing speed) required for terminal device i to calculate 1 bit of task. This represents the computing frequency of terminal device i, expressed in CPU cycles per second.

[0081] The energy consumption of terminal device i during local processing can be expressed as: It can be obtained according to the following formula 3.

[0082] Where κ represents the effective energy coefficient. This represents the computation frequency of terminal device i. This represents the local energy consumption calculation factor. This represents the amount of work processed locally by terminal device i. i This represents the number of CPU cycles required for terminal device i to calculate 1 bit of task.

[0083] 1-2. The terminal device uploads some of the tasks to be processed to the edge server.

[0084] Specifically, in the t-th time slot, the amount of data uploaded by terminal device i to the edge server for processing can be represented as follows: It can be obtained according to the following formula 4.

[0085] Where 1-α represents the proportion of tasks to be processed that are allocated to edge servers for processing. This indicates the number of tasks to be processed.

[0086] It should be noted that the process by which terminal device i uploads a portion of its pending tasks to the edge server can be as follows: Terminal device i sends an unload request to the corresponding target network device. This unload request includes the amount of tasks to be unloaded and the task deadline. After receiving the unload request, the target network device sends an unload request to the configured edge server.

[0087] The transmission time for terminal device i to upload the task assigned to the edge server for processing to the edge server can be expressed as: It can be obtained based on the amount of tasks assigned to the edge server for processing (i.e., the second amount of tasks) and the data transmission rate between the terminal device and the corresponding target network device. Formula 5 can be satisfied.

[0088] in, This indicates the amount of work processed by the edge server. ij This represents the data transmission rate between terminal device i and the corresponding target network device.

[0089] Furthermore, the data transmission rate r between the aforementioned terminal device i and the corresponding target network device... ij The value of r can be obtained based on the channel bandwidth of the edge server, the transmission power of terminal device i, the channel gain between terminal device i and the target network device, and the distance between terminal device i and the target network device. ij Formula 6 can be satisfied.

[0090] Where W represents the channel bandwidth of the edge server. h represents the transmission power of terminal device i in the t-th time slot. i This represents the channel gain between terminal device i and the target network device. ij This represents the distance between terminal device i and the target network device. β represents the path loss coefficient. N0 represents the Gaussian white noise power. I represents the interference power.

[0091] Optionally, the interference power I can be obtained from the following formula 7.

[0092] Here, x can take the value 0 or 1, used to determine whether the terminal device is in an area with interference. P represents the transmission power of the terminal device. H represents the channel state from the terminal device to the target network device.

[0093] The energy consumption of terminal device i when uploading part of the tasks to be processed to the edge server can be expressed as: It can be obtained according to the following formula 8.

[0094] in, This represents the transmission power of terminal device i in the t-th time slot. This indicates the transmission time for terminal device i to upload the task assigned to the edge server for processing to the edge server.

[0095] 1-3. The edge server processes some of the received pending tasks.

[0096] Specifically, the edge server performs calculations on the received partial tasks to be processed and returns the calculation results to the terminal device.

[0097] It should be noted that the processing time of the edge server includes the transmission time of the terminal device uploading part of the pending tasks to the edge server and the processing time of the edge server for the received part of the pending tasks.

[0098] The task offloading method provided in this application obtains task information of the task to be processed on the terminal device and the location information of the terminal device. Based on the location information and mobility of the terminal device, a target network device is determined, and the task to be processed is assigned to an edge server configured on both the terminal device and the target network device for processing. Since the target network device is determined based on the location information and mobility of the terminal device, the connection success rate between the terminal device and the target network device is high, resulting in a high task offloading success rate. In other words, the task offloading method provided in this application takes into account the mobility characteristics of the terminal device, thereby improving the success rate of task offloading on the terminal device.

[0099] Optionally, if there are multiple terminal devices, after S203, the edge server can allocate computing resources for each pending task uploaded by the terminal device. Therefore, as shown in FIG3, the task offloading method described in this embodiment may further include the following steps.

[0100] S301. For each terminal device, determine the corresponding task processing time.

[0101] Among them, the task processing time is the maximum time for the edge server to process the pending tasks of the terminal device.

[0102] Specifically, the maximum processing time for the task pending on terminal device i by the edge server can be expressed as: It can be obtained according to the following formula 9.

[0103] in, This indicates the task deadline for the pending task of terminal device i. This indicates the transmission time for terminal device i to upload the task assigned to the edge server for processing to the edge server.

[0104] S302. Based on the task processing time corresponding to the terminal device, determine the minimum computing resources required for the edge server to process the tasks to be processed by the terminal device.

[0105] Specifically, the minimum computing resources required for the edge server to process the task to be processed by terminal device i can be expressed as: It can be obtained according to the following formula 10.

[0106] in, This represents the amount of work that terminal device i uploads to the edge server for processing. C represents the number of CPU cycles required for the edge server to compute 1 bit of work.

[0107] Furthermore, within each time slot, the edge server allocates computing resources to each terminal device's task based on the minimum computing resources required to process each terminal device's task.

[0108] If the sum of the minimum computing resources required by the edge server to process the tasks of each terminal device is less than or equal to the total computing resources of the edge server, this can be expressed by the formula: Among them, f n，j Let f represent the total computing resources of the edge server configured for the target network device j. This determines whether the edge server can handle the pending tasks assigned to multiple terminal devices (i.e., task unloading is successful). The computing resources allocated by the edge server to the pending tasks of terminal device i can be represented as f. i off f i off It can be obtained according to the following formula 11.

[0109] in, This represents the minimum computing resources required for the edge server to process the task pending on terminal device i. This represents the sum of the minimum computing resources required for the edge server to process the tasks pending on each terminal device. n，j This represents the total computing resources of the edge servers.

[0110] If the sum of the minimum computing resources required by the edge server to process the tasks of each terminal device is less than or equal to the total computing resources of the edge server, this can be expressed by the formula: If the task unloading fails, the edge server will determine the priority of the terminal devices and process the tasks assigned to multiple terminal devices.

[0111] Optional, in In this scenario, the edge server can determine the priority for allocating computing resources to each terminal device's tasks based on the task deadline of each terminal device. The task with the earliest deadline has the highest priority. All terminal device offloading tasks are defined as task queue M, arranged from highest to lowest priority. The edge server then allocates computing resources to each terminal device's tasks sequentially according to the order in task queue M, and processes the tasks based on the allocated computing resources.

[0112] Furthermore, in In some cases, the total computing resources of the current time-slot edge server are insufficient to allocate computing resources to all tasks of all terminal devices. Even in this situation, the edge server can still allocate computing resources to each terminal device's tasks sequentially based on its task deadline. Tasks of terminal devices that have not received computing resources are recorded, and computing resources are allocated to these tasks when the edge server's computing resources are released.

[0113] In one embodiment, as shown in FIG4, the above-mentioned S203 can be specifically determined by the following S401 to S402.

[0114] S401. Determine the allocation coefficients based on the task information of the task to be processed and the allocation coefficient decision algorithm.

[0115] The allocation coefficient refers to the proportion of tasks to be processed locally on the terminal device, and can be represented as α.

[0116] In one possible implementation, the state information of the current time slot is obtained, and the state information is input into the allocation coefficient decision algorithm to obtain the allocation coefficient α of terminal device i.

[0117] Optionally, the above allocation coefficient decision algorithm can be based on the twin delayed deep deterministic policy gradient (TD3) algorithm. The TD3 algorithm includes a total of 6 network structures: 1 Actor network, 2 Critic networks, 1 target Actor network, and 2 target Critic networks.

[0118] The Actor network comprises a multi-layered neural network used to extract features from the input state and output actions. The Critic network evaluates the value of the actions output by the Actor network. In the TD3 algorithm, there are two Critic networks (Critic1 and Critic2), which have the same network structure but different parameters. Critic1 and Critic2 calculate their evaluation values ​​separately, and the smaller of the two is taken as the final evaluation value to suppress overestimation. The target Actor network and target Critic network are "target versions" of the Actor and Critic networks, used to calculate target values. During training, the target network updates its parameters periodically or according to a preset strategy to maintain consistency with the original network, but the update speed is slow to achieve a stable learning process.

[0119] It should be noted that a centralized training and distributed decision-making approach is used to obtain the allocation coefficients for each terminal device within the coverage area of ​​each network device. Specifically, each network device is treated as an independent intelligent agent. These agents operate independently without interfering with each other and can make independent decisions based on the actual situation of the terminal devices within their coverage area. The centralized training phase is responsible for optimizing the agent's decision-making model. The distributed decision-making phase allows each agent to determine the appropriate allocation coefficients for the tasks to be processed by the terminal devices based on the actual state.

[0120] Furthermore, since the TD3 algorithm optimizes towards maximizing reward values, using negative energy consumption as reward values ​​can encourage the agent to optimize towards minimizing energy consumption. Total energy consumption includes the energy consumption of processing tasks locally on the terminal device, the energy consumption of transmitting tasks to the edge server, and the energy consumption of processing tasks on the edge server.

[0121] Optionally, since the energy consumption of computing the task on the edge server is negligible compared to the energy consumption of computing the task locally on the terminal device and the energy consumption of transmitting the task to the edge server, the total energy consumption only considers the sum of the energy consumption of computing the task locally on the terminal device and the energy consumption of transmitting the task to the edge server. The total energy consumption can be expressed as... It can be obtained according to the following formula 12.

[0122] in, This represents the energy consumption of terminal device i during local processing. This represents the energy consumption of terminal device i when uploading some of the tasks to be processed to the edge server.

[0123] S402. Based on the allocation coefficient, the tasks to be processed are allocated to terminal devices and edge servers for processing.

[0124] In one possible implementation, tasks to be processed are allocated to terminal devices i in proportion α according to the allocation coefficient α, and tasks to be processed are allocated to edge servers in proportion (1-α).

[0125] For each pending task on a terminal device, a reward value is given if the task is completed within the task deadline. If the pending task is not completed within the task deadline, a reward value r will be given. i =-20.

[0126] For example, the process of storing a dataset using an allocation coefficient decision algorithm may include the following steps.

[0127] Step 1: Obtain the state information of each agent.

[0128] Specifically, each base station is treated as an intelligent agent, and the state information of the agent can be represented as S. t S t ={s 0，t s 1，t , ......, s i，t}. Among them, s i，t This indicates the state of the i-th terminal device within the coverage area of ​​the base station in time slot t. Among them, f n，j，t This indicates the total computing resources of the edge servers configured for this base station in the current time slot. This represents the number of tasks to be processed by the i-th terminal device in the current time slot. h represents the task deadline for the i-th terminal device in the current time slot. i，t Dis represents the channel gain between the i-th terminal device and the base station. ij This represents the distance between the i-th terminal device and the base station.

[0129] Step 2: Transfer the status information S t The Actor network of the input allocation coefficient decision algorithm is used to obtain action information A. t .

[0130] Among them, action information A t This represents the set of allocation coefficients for each terminal device within the coverage area of ​​a base station.

[0131] Step 3: The terminal device determines the action information A. t (i.e., allocation coefficient) is used to offload tasks, and the base station is given action A based on the actual execution energy consumption.t The reward value R obtained afterwards t .

[0132] Among them, the reward value R t R is the sum of the reward values ​​obtained after each terminal device performs an action. t =∑r i .

[0133] Step 4: Critic network for A t Scoring is performed, and a Q-value is given.

[0134] Understandably, the Critic network evaluates the quality of the allocation coefficients to help assess the performance of each terminal device within the coverage area of ​​each base station.

[0135] Step 5: Update the state information of the agent to obtain the state information S of the agent in the next time slot. t+1 A set of quadruple data (S) was obtained. t A t R t S t+1 The quadruple data is then stored in the experience cache pool corresponding to the agent.

[0136] Understandably, storing data in an experience cache pool makes it easier to use the data in the experience cache pool to train the allocation coefficient decision algorithm later.

[0137] Step 6: Repeat steps 1 to 5 above to obtain multiple sets of quadruplets.

[0138] For example, the training process of the allocation coefficient decision algorithm may include the following steps.

[0139] Step 7: Agent j randomly retrieves a set of four-tuple data (S) from the corresponding experience cache pool. t A t R t S t+1 ).

[0140] Step 8: Transfer the status information S t+1 Inputting the Actor network, the action information A for the next time slot is predicted. t+1 .

[0141] Among them, A t+1 =π(θ, S) t S t+1 )+ζ,ζ~clip(N(0,σ),-e,e),ζ represents noise, used to increase exploration,θ represents the parameters of the Actor network, andπ represents the policy function of the Actor network.

[0142] Step 9: The two Critic networks will process action A. t The Q value is obtained by scoring:

[0143] Q1=Q(S t A t ;ω1) and Q2=Q(S t A t ;ω2).

[0144] Where ω1 represents the parameters of the Critic1 network, and ω2 represents the parameters of the Critic2 network. Q(S t A t ;ω) represents the action value function of the Critic network.

[0145] Step 10: The two target Critic networks predict action A. t+1 Score and get value:

[0146] in, Indicates the target Critic1 network parameters. This represents the target Critic2 network parameters. This represents the action value function of the target Critic network.

[0147] Step 11: Calculate the target value y t :

[0148] Where γ represents the discount factor.

[0149] Step 12: Update the Critic network.

[0150] Specifically, the Critic network is updated by minimizing the mean squared error loss function:

[0151] Update the target Critic network parameters using gradient descent:

[0152] Where τ represents the learning rate, which is the weighted average of the old target network parameters and the new corresponding network parameters, and then assigned to the target network.

[0153] Step 13: Update the Actor network.

[0154] Specifically, after the Critic network updates by d steps, the Actor network updates its parameters using gradient ascent.

[0155] Step 14: Repeat steps 6 to 13 above to obtain the optimized allocation coefficient decision algorithm.

[0156] Specifically, when the fluctuation range of the reward value gradually decreases and remains within a small range for a certain period of time, it indicates that the allocation coefficient decision algorithm has converged or is close to convergence, and a better allocation coefficient decision algorithm is obtained.

[0157] For example, Figure 5 is a schematic diagram of the modules of a task unloading device provided in an embodiment of this application. As shown in Figure 5, the task unloading device includes an information collection module and an allocation coefficient decision module.

[0158] The information collection module includes a terminal device information collection module, a network device information collection module, and an edge server information collection module. The terminal device information collection module collects the initial location information, task information of pending tasks, transmission power, and local computing resources for each fixed and active terminal device. This module includes a location information update module and a task generation module. The location information update module updates the location information of active terminal devices in each time slot and determines the target network device corresponding to each active terminal device in each time slot. The task generation module generates pending tasks for both fixed and active terminal devices in each time slot. The network device information collection module collects the location information, bandwidth resource information, and channel information for each network device. The edge server information collection module collects the computing resource information for each edge server.

[0159] The allocation coefficient decision module includes an agent decision allocation coefficient module, a local calculation module for tasks to be processed, an edge calculation module for tasks to be processed, a reward value calculation module, a data storage module, and an algorithm training module.

[0160] For example, Figure 6 is a schematic diagram of a communication network structure provided in an embodiment of this application. As shown in Figure 6, the communication network includes multiple base stations, each equipped with one edge server. The coverage area of ​​each base station includes at least one fixed terminal device and at least one active terminal device. The active terminal device will move horizontally within the coverage area of ​​all base stations, and its location information will change. The location information of the fixed terminal device will not change.

[0161] It should be noted that, in the case of the communication network structure shown in Figure 6, the task offloading method provided in this application embodiment can intelligently determine a suitable task allocation coefficient for the tasks to be processed by the terminal device, under the premise of meeting latency requirements and energy-saving constraints, so that the tasks of the terminal device can be executed collaboratively on the local side and the edge side, while the edge server can allocate reasonable computing resources for the tasks uploaded by the terminal device.

[0162] Furthermore, based on the above task unloading method, a specific simulation experiment was conducted.

[0163] For example, Figure 7 is a schematic diagram of a reward curve provided in an embodiment of this application. As shown in Figure 7, the number of active terminal devices is set to M=1, and the number of fixed terminal devices is set to N=5, N=10, and N=15 respectively. The horizontal axis of Figure 7 represents the episode, and the vertical axis represents the reward.

[0164] As shown in Figure 7, all reward value curves eventually converge. Furthermore, while keeping the number of active terminal devices constant, the convergence speed of the reward value curves slows down as the number of fixed terminal devices increases; however, all curves still eventually converge.

[0165] Furthermore, Figure 8 is a schematic diagram of the task execution success rate provided in an embodiment of this application, which is the same as the task execution success rate corresponding to Figure 7. As shown in Figure 8, when the number of active terminal devices M=1, the average success rate is 0.9996 when the number of fixed terminal devices is N=5, N=10, and N=15. Therefore, it can be seen that the task unloading method provided in this embodiment of the application works very well in this application scenario.

[0166] For example, Figure 9 is a schematic diagram of another reward value curve provided in an embodiment of this application. As shown in Figure 9, the number of fixed terminal devices is set to N=5, and the number of active terminal devices is set to M=1, M=2, and M=3 respectively. The horizontal axis of Figure 9 is episode, and the vertical axis is reward.

[0167] According to Figure 9, all reward value curves eventually converge. Furthermore, while maintaining a constant number of fixed terminal devices, as the number of active terminal devices increases, the convergence difficulty of the reward value curves increases, and the convergence speed also slows down. However, all curves can still converge after a certain training process.

[0168] Furthermore, Figure 10 is a schematic diagram of another task execution success rate provided by an embodiment of this application, which is a schematic diagram of the task execution success rate corresponding to Figure 9. As shown in Figure 9, when the number of fixed terminal devices N=5, the average success rates when the number of active terminal devices M=1, M=2, and M=3 are 0.9997, 0.9994, and 0.9994, respectively. It can be seen that, under the premise of a constant number of fixed terminal devices, as the number of active terminal devices increases, the ratio of task execution success rate decreases slightly but still reaches 0.9994, indicating that the task unloading method provided by this embodiment of the application works very well in this application scenario.

[0169] It is understood that the above-described task unloading method can be implemented by a task unloading device. To achieve the above functions, the task unloading device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments disclosed in this application.

[0170] The embodiments disclosed in this application can divide the task unloading device generated by the above method example into functional modules. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments disclosed in this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0171] Figure 11 is a schematic diagram of a task unloading device provided in an embodiment of this application. As shown in Figure 11, the task unloading device 110 can be used to execute the task unloading methods shown in Figures 2-4. The task unloading device 110 includes a communication unit 1101 and a processing unit 1102.

[0172] The communication unit 1101 is used to acquire task information of the task to be processed by the terminal device and the location information of the terminal device; the processing unit 1102 is used to determine the target network device based on the location information and mobility of the terminal device, wherein the target network device is configured with an edge server; the processing unit 1102 is also used to assign the task to be processed to the terminal device and the edge server for processing.

[0173] In one possible implementation, the task information includes the task deadline delay; wherein, the processing time of the terminal device is determined based on the terminal device's computing speed and computing frequency; and the processing time of the edge server is determined based on the edge server's computing speed, the computing resources allocated by the edge server to the task to be processed, and the data transmission rate between the terminal device and the target network device.

[0174] In one possible implementation, the data transmission rate between the terminal device and the target network device is determined based on the channel bandwidth of the edge server, the transmission power of the terminal device, the channel gain between the terminal device and the target network device, and the distance between the terminal device and the target network device.

[0175] In one possible implementation, the data transmission rate between the terminal device and the target network device satisfies the following formula:

[0176] Where W represents the channel bandwidth of the edge server, P i，trans h represents the transmission power of the terminal device. i Dis represents the channel gain between the terminal device and the target network device. ij β represents the distance between the terminal device and the target network device, N0 represents the Gaussian white noise power, and I represents the interference power.

[0177] In one possible implementation, there are multiple terminal devices. For each terminal device, the processing unit 1102 is further configured to determine the task processing time corresponding to each terminal device. The task processing time is the maximum time for the edge server to process the pending tasks of the terminal device. The processing unit 1102 is further configured to determine the minimum computing resources required for the edge server to process the pending tasks of the terminal device based on the task processing time corresponding to the terminal device.

[0178] In one possible implementation, if the sum of the minimum computing resources required by the edge server to process the pending tasks of each terminal device is less than or equal to the total computing resources of the edge server, then the processing unit 1102 is further configured to determine that the edge server is capable of processing the pending tasks of the multiple terminal devices; if the sum of the minimum computing resources required to process the pending tasks of each terminal device is greater than the total computing resources of the edge server, then the processing unit 1102 is further configured to determine that the pending tasks of the multiple terminal devices are processed according to the priority of the terminal devices.

[0179] In one possible implementation, the processing unit 1102 is further configured to determine the allocation coefficient based on the task information of the task to be processed and the allocation coefficient decision algorithm; the processing unit 1102 is further configured to allocate the task to be processed to the terminal device and the edge server for processing based on the allocation coefficient.

[0180] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0181] This disclosure also provides a computer-readable storage medium storing instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the task offloading method provided in the embodiments of this disclosure described above.

[0182] This disclosure also provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the task unloading method provided in the above-described embodiments of this disclosure.

[0183] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0184] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A task unloading method, the method comprising: Obtain the task information of the pending task from the terminal device and the location information of the terminal device; Based on the location information and mobility of the terminal device, a target network device is determined, wherein the target network device is configured with an edge server; The task to be processed is assigned to the terminal device and the edge server for processing.

2. The method according to claim 1, wherein, The task information includes the task deadline; The processing time of the terminal device is determined based on the computing speed and computing frequency of the terminal device. The processing time of the edge server is determined based on the edge server's computing speed, the computing resources allocated by the edge server to the task to be processed, and the data transmission rate between the terminal device and the target network device.

3. The method according to claim 2, wherein, The data transmission rate between the terminal device and the target network device is determined based on the channel bandwidth of the edge server, the transmission power of the terminal device, the channel gain between the terminal device and the target network device, and the distance between the terminal device and the target network device.

4. The method according to claim 3, wherein, The data transmission rate between the terminal device and the target network device satisfies the following formula: Where W represents the channel bandwidth of the edge server, P i，trans h represents the transmission power of the terminal device. i Dis represents the channel gain between the terminal device and the target network device. ij The distance between the terminal device and the target network device is represented by β, the path loss coefficient is represented by N0, the Gaussian white noise power is represented by I, and the interference power is represented by I.

5. The method according to claim 1, wherein, The number of terminal devices is multiple, and the method further includes: For each terminal device, a corresponding task processing time is determined, which is the maximum time for the edge server to process the pending tasks of the terminal device. Based on the task processing time corresponding to the terminal device, the minimum computing resources required for the edge server to process the pending tasks of the terminal device are determined.

6. The method according to claim 5, wherein, The method further includes: If the sum of the minimum computing resources required by the edge server to process the pending tasks of each terminal device is less than or equal to the total computing resources of the edge server, then it is determined that the edge server is capable of processing the pending tasks of the multiple terminal devices. If the sum of the minimum computing resources required to process the pending tasks of each terminal device is greater than the total computing resources of the edge server, then the pending tasks of the multiple terminal devices will be processed according to the priority of the terminal devices.

7. The method according to any one of claims 1-6, wherein, The step of assigning the task to be processed to the terminal device and the edge server for processing includes: The allocation coefficient is determined based on the task information of the task to be processed and the allocation coefficient decision algorithm; Based on the allocation coefficient, the task to be processed is allocated to the terminal device and the edge server for processing.

8. A task unloading device, the device comprising: Communication unit and processing unit; The communication unit is used to acquire task information of the task to be processed by the terminal device and the location information of the terminal device; The processing unit is used to determine a target network device based on the location information and mobility of the terminal device, wherein the target network device is configured with an edge server; The processing unit is further configured to assign the task to be processed to the terminal device and the edge server for processing.

9. A task unloading device, wherein, include: Memory and processor; The memory and the processor are coupled; The memory is used to store instructions that can be executed by the processor; When the processor executes the instructions, it performs the task unloading method as described in any one of claims 1-7.

10. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the task unloading method as described in any one of claims 1-7.

11. A computer program product, wherein, The computer program product includes computer program instructions that, when executed by a processor, implement the task unloading method as described in any one of claims 1-7.

Images