Cache auxiliary task cooperative unloading and resource allocation method based on meta reinforcement learning

A resource allocation and reinforcement learning technology, applied in the field of edge computing technology and reinforcement learning, can solve the problems of unguaranteed user experience and high load, and achieve the effects of fast computing offloading decisions, reducing energy consumption and delay, and improving computing performance

Active Publication Date: 2021-09-24
BEIJING UNIV OF POSTS & TELECOMM
View PDF13 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, because it considers the user offloading strategy separately, there is still a lot of overhead in the repeated calculation in the mobile collaborativ

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cache auxiliary task cooperative unloading and resource allocation method based on meta reinforcement learning
  • Cache auxiliary task cooperative unloading and resource allocation method based on meta reinforcement learning
  • Cache auxiliary task cooperative unloading and resource allocation method based on meta reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] Embodiment 1 model establishment

[0047] The invention establishes a cache-assisted task cooperation unloading and resource allocation model in a mobile cooperative application scenario. The steps of model building are as follows:

[0048] 1 Edge environment: Group edge servers into cooperative clusters according to the geographical distribution of base stations. In a cooperative cluster, the edge nodes use the set express. Each edge server has cache and computing resources, Q m and C m Respectively represent the cache space capacity and maximum computing power of the edge node m. In order to better reuse the original data and calculation results in the calculation, a cache is added in each edge server. Based on the cache efficiency, the computing tasks executed on the edge server are cached and the cache status is obtained. If subsequent computing tasks need to use / get the same data, there is no need to repeat the transmission / computation.

[0049] 2 Computin...

Embodiment 2

[0088] Embodiment 2 algorithm design

[0089] In order to solve the above-mentioned Markov decision process, the present invention designs a seq2seq network fitting strategy function and value function based on a recurrent neural network. The network includes two parts of an encoder and a decoder, both of which are realized by a multilayer recurrent neural network. The network adopts a shared parameter network architecture, that is, the policy function and the value function share all network structures and parameters except the output layer (the policy function uses a softmax output layer; the value function uses a fully connected output layer). Embedding tasks into sequence T G =(task 1 , task 2 ,..., task |V| ) and offload scheduling decision history sequence A G =(a 1 , a 2 ,...,a i-1 ) into the network. The encoder sequentially inputs the task embedding sequence T G , outputting the features of the final hidden layer DAG. The decoder initializes its hidden layer...

Embodiment 3

[0117] Embodiment 3 algorithm flow

[0118] The present invention proposes a cache-assisted computing offloading method based on meta-reinforcement learning, which is divided into three stages: obtaining the cache state of the request task, obtaining the learning model (algorithm 2) and solving the unloading decision (algorithm 1). The overall idea is as figure 1 shown.

[0119] (1) Get task cache status

[0120] Step 1: Enter the cache execution threshold δ s and the cache removal threshold δ d . The field value determines whether to cache the task.

[0121] Step 2: Calculate the cache benefit CU(i). The smaller the storage space occupied, the greater the network income, and the higher the number of requests, the greater the value of the content cache utility.

[0122] Step 3: Determine the task cache state K. When the cache gain is greater than the execution threshold δ s When , cache it on the node of mobile edge computing; when the cache gain is less than delete δ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cache auxiliary task cooperative unloading and resource allocation method based on meta reinforcement learning. Comprising the following steps: establishing a cache-assisted task collaborative unloading and resource allocation model in a mobile collaborative application scene; acquiring a cache state of the request task; obtaining a learning model; and solving an unloading decision so as to reduce the energy consumption and time delay of the mobile user in the calculation unloading process under the mobile cooperation application program scene. According to the invention, the user preference, the tradeoff of energy consumption and time delay and the influence of the cache state of the task on the unloading decision are comprehensively considered, and a cache auxiliary strategy is provided. The task cache hit rate is improved while the network overhead is balanced, and the cache state of the task is determined according to the task cache hit rate; and finally, an online calculation unloading strategy is provided based on meta reinforcement learning, and the problem that the sampling efficiency of a traditional deep reinforcement learning algorithm on new tasks is low is solved. Experimental results prove that compared with other algorithms, the time delay and energy consumption of the mobile user can be effectively reduced, and the user experience quality is improved.

Description

technical field [0001] The present invention relates to the field of edge computing technology and reinforcement learning technology, and in particular to a method for collaborative unloading and resource allocation of caching auxiliary tasks based on meta-reinforcement learning. Background technique [0002] With the development and wide application of the Internet of Things and edge computing technology, various mobile devices have brought a surge of complex and computationally intensive mobile applications, and the emergence of more new applications poses a challenge to user quality of experience (QoE). Mobile devices with higher requirements and limited resources are faced with many constraints in power, capacity and computing power that are difficult to handle. Internet of Things users tend to offload and execute a large number of content blocks of data streams, but due to certain limitations of computing offloading, especially in dynamic edge environments with limited ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/445G06F9/50
CPCG06F9/44594G06F9/5072Y02D10/00
Inventor 芮兰兰高志鹏陈世优杨杨李文璟
Owner BEIJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap