Icn-based mobile edge computing system video stream autonomous edge caching method

Abstract

The application discloses a kind of based on ICN's mobile edge computing system video stream autonomous edge cache method, constructs the mobile edge computing network based on ICN, and edge layer and user layer are connected using ICN architecture, and communication is carried out in the form of interest packet and data packet, and edge layer and cloud layer rely on the physical connection of core network and carry out communication;Propose the distribution mode of user content request, and the situation modeling of content cache hit, calculate local cache hit rate and collaborative cache hit rate;The content with high local popularity in base station range is cached in base station, and the preference of user is learned in real time using base station, and personalized content is cached in user end;Base station is used as intelligent agent, and soft actor-critic algorithm is adopted, and Markov decision process modeling is carried out based on long-term interest of user and video content popularity, and the result of cache placement is output.The application can realize the improvement of edge cache hit rate and cache diversity, reduce the delay of user request video content.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of mobile edge computing, and particularly relates to a video stream autonomous edge caching method based on an ICN mobile edge computing system. BACKGROUND

[0002] With the development of Internet applications and the popularity of intelligent terminals, people's demand for multimedia services is increasing, and the traffic of video applications in mobile wireless networks is growing rapidly. In the face of increasingly stringent requirements for QoE and QoS, the traditional cloud computing mode is limited by the bandwidth capacity of the backhaul link, and a large number of video requests will frequently cause network congestion and excessive response delay, resulting in a sharp decline in network service quality and user experience. In order to overcome the limitations of the existing mode, mobile edge computing (MEC) has emerged. MEC deploys servers near the network edge to provide computing and storage services for the edge. Users can directly request video content from the nearby edge server without having to obtain it from the remote cloud, thereby reducing the waiting time and improving user QoE.

[0003] The existing MEC architecture is usually built based on the TCP / IP architecture. IP networks request and distribute content based on host addresses. In the face of massive user video content requests, the host address-based communication method will cause a large number of repeated content requests, resulting in network congestion and waste of network resources. In addition, there are many drawbacks in terms of mobility, security and scalability. In the user's large-scale data request, the user is not interested in the host address, but in the content information itself. Therefore, the combination of information-centric networking (ICN) and MEC is considered to reduce network duplicate traffic and support user mobility.

[0004] The caching mechanism is one of the core mechanisms of ICN. In ICN, when facing massive content distribution problems, a reasonable cache placement scheme can improve the cache hit rate of the system and reduce the content request delay. At the same time, a good cache placement scheme also needs to consider the effective use of cache resources, improve cache diversity, and reduce cache redundancy. The cache placement scheme needs to consider two problems, one is to choose what kind of content to cache, and the other is where to cache. At present, the cache strategy of ICN is mostly based on the popularity of the cache mechanism. However, due to the instability of user requests, the popularity of content changes in real time, and the cache strategy cannot adapt to the real-time nature of content, resulting in a low cache hit rate. Therefore, designing a cache strategy that can adapt to the changes in system state in real time to make the best choice will greatly improve the cache performance of the network. SUMMARY

[0005] The application discloses an ICN-based mobile edge computing system video stream autonomous edge caching method, which can effectively solve at least one technical problem involved in the background art.

[0006] To achieve the above-mentioned purpose, the technical scheme of the application is:

[0007] An ICN-based mobile edge computing system video stream autonomous edge caching method comprises the following steps:

[0008] Step S1: constructing an ICN-based mobile edge computing network architecture model, the edge layer and the user layer being connected using the ICN architecture to communicate in the form of interest packets and data packets, and the edge layer and the cloud layer communicating by relying on the physical connection of the core network;

[0009] Step S2: proposing a distribution mode of user content requests, modeling the content cache hit condition in the distribution mode, and calculating the local cache hit rate and the cooperative cache hit rate at the current moment;

[0010] Step S3: calculating the global content popularity and the dynamic popularity threshold of the current base station area, obtaining the globally popular content in the current base station, and caching the content in the base station;

[0011] Step S4: taking the base station as an agent, learning the user preference in real time, modeling the long-term and short-term interests of the user, then combining the cache state information of the user and the base station at the current moment as state input to establish a cache decision;

[0012] Step S5: modeling the cache decision by using a Markov decision process, proposing a multi-agent deep reinforcement learning algorithm based on the Soft Actor-Critic algorithm to train the model, and obtaining an optimal cache placement scheme.

[0013] As a preferred improvement of the application, the mobile edge computing network architecture model comprises a cloud server, which stores all the content that can be requested by the user, and the content library contains f video files each with a size of δ, and can communicate with the base stations of the edge layer through the physical connection of the core network; the edge layer has N base stations, each base station has a plurality of users within its coverage range, and the base station is responsible for managing the cache information of the users in the coverage area to facilitate the cooperative caching; the users in the user layer only belong to one base station at the same moment, and will periodically update the cache information to the base station; the user layer and the edge layer are connected through the ICN architecture, the user sends a request content in the form of an interest packet, and the cache node responds in the form of a data packet.

[0014] As a preferred improvement of the application, the content stored in the cloud server is represented by a set F={1,2,…,f}, and contains the ratings R={r u，f, where r u，f represents the score of user u to content f, and the value of 0 represents that user u has not scored the file f, and the value of 0.5-5 represents the degree of user's preference to the file, and the larger the value represents the more preferred the file; the base station B = {1, 2, …, N}, and the storage space size is C B ; the user set in the coverage range of a base station is U = {1, 2, …, u}, and the storage space size is C u ; the user equipment and the base station are connected based on the architecture of ICN, and all maintain three lists:

[0015] a content storage table for storing content copies cached by nodes;

[0016] a pending request table for recording the outgoing but not yet corresponding information packets and their corresponding incoming interfaces;

[0017] a forwarding information table for recording the outgoing information packets and their corresponding forwarding interfaces.

[0018] As a preferred improvement of the application, the distribution mode of the proposed user content request includes six cases:

[0019] Case one, when a user requests a certain content f, the user caches a copy of the content f in the content storage table of itself;

[0020] Case two, when the user does not cache a copy of the content f, the user broadcasts an interest packet to other user equipment in its communication range, and if there is a user equipment caching a copy of the content f in its broadcast range, the caching user returns the data packet directly to the requesting user;

[0021] Case three, when there is no device caching the content f in the broadcast range of the user, the requesting user sends an interest packet to the home base station, and if the home base station caches the content, the home base station returns the data packet to the requesting user;

[0022] Case four, when the home base station does not find the cached information of the content f, it searches other user equipment in its coverage range, and if it finds a device caching the content, it communicates with the device and receives the data packet in the caching device, and then transmits it to the requesting user;

[0023] Case five, when the home base station does not find other devices caching the content f, it broadcasts an interest packet to the adjacent base station, and the adjacent base station performs the same operation as cases three and four;

[0024] Case six, when the above cases do not find the interest packet, the base station sends a request to the cloud.

[0025] As a preferred improvement of the present application, the content cache hit in the distribution mode is modeled, in particular:

[0026] In each time slot t, the content request Req t = {req t (u, f), u ∈ U, f ∈ F}, where req t (u, f) represents the number of times that user u requests content f at time t; the cache information X i = {x i (f, t), f ∈ F} of user i at time t, where x i (f, t) = true represents that user i caches content f; the cache information Y i = {y i (f, t), f ∈ F} of base station h at time t, where y i (f, t) = true represents that base station h caches content f; a set of binary variables S = {S1, S2, S3, S4, S5, S6} is defined to record the process of user request being satisfied:

[0027] S1 = x i (f, t)

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] where h represents the home base station of user i, and NE i represents the neighbor user set of user i.

[0034] As a preferred improvement of the present application, after the local cache hit rate and the cooperative cache hit rate at the current time are calculated, the cache benefit is defined and the maximum cache benefit objective function is defined.

[0035] As a preferred improvement of the present application, in step S3, the expressions of the global content popularity and the dynamic popularity threshold are respectively:

[0036]

[0037]

[0038] wherein, represents the global popularity of content f at time t, represents the number of requests of content f at time t, represents the weight parameter, T t represents the dynamic popularity threshold; when T> T , content f is considered as a globally popular content. t

[0039] As a preferred improvement of the present application, in step S4, the long-term interest of the user is extracted from the high-score feedback of the user on some content, denoted as S long = {i1, i2, …, in} L The long short-term memory network is used to learn the high-score evaluation record of the user for a long period of time to capture the long-term interest of the user, and then integrated into the user long-term interest matrix:

[0040] f t = σ(W f · [h t-1 , x t ] + b f )

[0041] i t = σ(W i · [h t-1 , x t ] + b i )

[0042]

[0043]

[0044] o t = σ(W o · [h t-1 , x t ] + b o )

[0045] h t = o t * tanh(C t )

[0046] where f t is the forget gate, σ is the sigmoid function, W f , W i , W C , and W o are the corresponding weights, h t-1 is the hidden state information of the previous time, x t is the current input information, b f , b i , b C , and b oare the corresponding biases, i t is the input gate, is the temporary cell state, C t is the current time long-term cell state, C t-1 is the previous time long-term cell state, o t is the output gate, h t is the current time hidden state information.

[0047] The content requested by the user frequently in a short period of time can be considered as the short-term interest of the user, and the short-term interest can be extracted from the content ranked in the top N in the current time node popularity, i.e., S short = {i1, i2, …, i N}, the user's short-term interest is captured by using the gated recurrent unit, and then integrated into the user's short-term interest matrix:

[0048]

[0049] z t = σ(x t W xz +h t-1 W hz +b z )

[0050] r t = σ(x t W xr +h t-1 W hr +b r )

[0051]

[0052]

[0053] wherein Pop(u, f) is the user popularity, z t is the update gate, r t is the reset gate, W xz , W hz are the weights of the update gate to the current input vector and the previous hidden state, respectively, W xr , W hr are the weights of the reset gate to the current input vector and the previous hidden state, respectively, W xh , W hh are the weights of the candidate hidden state to the current input vector and the previous hidden state, respectively, b z , b r , b h are the biases of the update gate, the reset gate and the candidate hidden state, respectively, is the candidate hidden state.

[0054] As a preferred improvement of the present application, the cache decision is modeled as a Markov decision process, specifically, a four-tuple M = <S, A, P, R> is defined, where,

[0055] State space: At each decision time t, the base station h extracts the environment state from the recorded information, and the user long-term interest matrix Short-term interest matrix User cache state information X t and base station cache state information Y t As the input state, it is represented as

[0056] Action space: At time t, the base station h needs to decide the cache placement problem of content f, denoted as {a u,f ,u∈U,f∈F};

[0057] State transition: After the system updates the cache content at time t, the user's request information and feedback information are recorded, and the state is transferred to S t+1 , i.e., update the user's long-term and short-term interest at time t+1 and cache state information;

[0058] Reward function: After performing the cache decision action, each base station will count the reward value obtained at the current time, and the reward obtained by the base station h after performing the cache action at time t is defined as the cache revenue at time t.

[0059] As a preferred improvement of the present application, the base station is regarded as an agent, each base station includes an actor network, a critic network, a target actor network and a target critic network, only the actor network and the critic network are trained: according to the CTDE framework and the SAC algorithm, the parameters of the actor network are updated by minimizing the target, the minimum target is obtained by using the reparameterization sampling method, and then the random gradient descent method is used to update the actor network; the parameters of the critic network are updated by minimizing the Bellman error of the base station.

[0060] The beneficial effects of the present application are as follows:

[0061] 1. The method proposed in the present application can make optimal cache decisions in real time according to the network state, optimize network resource utilization, reduce cache redundancy, and improve user experience quality under the condition of ensuring cache hit rate;

[0062] 2、The application effectively improves the edge cache hit rate and cache diversity, and reduces the delay of user request video content by modeling six cache hit cases to calculate the cache hit rate, and then learning the user's preferences in real time using the base station to cache personalized content at the user end.

[0063] 3、By using ICN and MEC technology, user mobility is supported, network duplication traffic is reduced, and the efficiency of service automatic deployment can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0064] Fig. 1 A scene structure schematic diagram provided for the embodiment of the application is provided.

[0065] Fig. 2 A flowchart of the distribution mode provided for the embodiment of the application is provided.

[0066] Fig. 3 A cache decision process schematic diagram provided for the embodiment of the application is provided. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the application will be clearly and completely described in conjunction with the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the application.

[0068] It should be noted that all directionality indications (such as up, down, left, right, front, back, etc.) in the embodiments of the application are only used to explain the relative position relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directionality indications also change accordingly.

[0069] In addition, the description such as "first", "second" and the like in the application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0070] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixing" and the like should be understood in a broad sense, for example, "fixing" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship of two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0071] In addition, the technical solutions among various embodiments of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the protection scope required by the present application.

[0072] Please refer to Figs. 1 to 3 As shown in the accompanying drawings, the present application provides an ICN-based mobile edge computing system video stream autonomous edge caching method, comprising the following steps:

[0073] Step S1: Construct an ICN-based mobile edge computing network architecture model, the edge layer and the user layer are connected using the ICN architecture, and communicate in the form of interest packets and data packets, and the edge layer and the cloud layer communicate through the physical connection of the core network.

[0074] Specifically, the ICN-based mobile edge computing network architecture model includes a cloud server that stores all content that users can request, the content library contains f video files each with a size of δ, and can communicate with the base stations of the edge layer through the physical connection of the core network; the edge layer has N base stations, each base station has a number of users within its coverage range, and the base station is responsible for managing the cache information of the users within the coverage area to facilitate the cooperation of caching; the users in the user layer belong to only one base station at the same time, and will periodically update the cache information to the base station, and the user layer and the edge layer are connected through the ICN architecture, and the user sends a request content in the form of an interest packet, and the cache node responds in the form of a data packet.

[0075] The content F stored in the cloud server = {1, 2,..., f} contains the ratings of the users, and the rating matrix is represented as R = {r u，f ,u∈U,f∈F}, where r u，f represents the rating of user u on content f, and the value range is 0 or 0.5-5, when the value is 0, it represents that user u has not rated file f, and when the value is 0.5-5, it represents the degree of preference of the user to the file, and the larger the value, the more preferred the file; the base station B = {1, 2,..., N} has a storage space size of C B, a set of users in the coverage of a base station is U = {1, 2, …, u}, and the size of the storage space is C u .

[0076] The user equipment and the base station are connected based on the ICN architecture, each user equipment and the base station maintain three lists: a content storage table (CS), a pending interest table (PIT) and a forwarding information base (FIB); wherein the CS is used to store the content copies cached by the node, the PIT is used to record the information packets that have been sent out but have not been returned and the corresponding incoming interfaces, and the FIB is used to record the information packets that have been sent out and the corresponding forwarding interfaces.

[0077] The packet structure of the interest packet is shown in Table 1, and the packet structure of the data packet is shown in Table 2:

[0078] Table 1 Packet structure of interest packet

[0079]

[0080] Table 2 Packet structure of data packet

[0081]

[0082] Step S2: Propose a distribution mode of user content request, model according to content cache hit, calculate the local cache hit rate and the cooperative cache hit rate at the current time.

[0083] Specifically, the distribution mode of the user content request includes six cases, which are:

[0084] Case one, when a user requests a content f, if a copy of the content f is cached in the content CS of the user itself, the user itself is a cache hit;

[0085] Case two, when the user itself does not cache a copy of the content f, the user equipment broadcasts an interest packet to other user equipment in its communication range, if there is a user equipment that caches a copy of the content f in the broadcast range of the user, the user equipment returns the data packet directly to the requesting user, and this case is a D2D hit;

[0086] Case three, when there is no user equipment that caches the content f in the D2D range of the user, the requesting user sends an interest packet to the home base station, if the home base station caches the content, the home base station returns the data packet to the requesting user;

[0087] Case 4: If the home base station does not find cached information for content f, it searches the cached information table of users within its coverage area. If it finds a device that caches the content, it communicates with it. The cached device sends the corresponding data packet back to the base station, which then transmits it to the requesting user.

[0088] Case 5: If the home base station of the requesting user does not find any device that caches content f, it broadcasts an interest packet to the neighboring base station. The neighboring base station performs the same operation as in cases 3 and 4. If a cached device is found, the data packet is returned through the original base station.

[0089] Scenario 6: If none of the above scenarios are found or the interest package expires, a request is sent to the cloud via the base station.

[0090] Then, we model the above six content cache hit scenarios, which can be specifically represented as follows: The system runs for an infinitely long time, which can be divided into multiple time slots, denoted as t = 0, 1, 2... Within each time slot t, each user's content request Req... t ={req t (u,f),u∈U,f∈F}, where req t (u,f) represents the number of times user u requests content f at time t; the cached information X of user i at time t. t ={x t (i,f),i∈U,f∈F}, where x t (i,f) = true means that user i has cached content f; the cached information Y of base station h at time t. t ={y t (h,f),h∈B,f∈F}, where y t (h,f) = true indicates that base station h has cached content f. A set of binary variables S = {S1, S2, S3, S4, S5, S6} is defined to record the process of a user request being fulfilled.

[0091] S1=x t (i,f)

[0092]

[0093]

[0094]

[0095]

[0096]

[0097] Among them, NE i Let h represent the set of neighboring users of user i, and let h represent the home base station of user i.

[0098] Next, the local cache hit rate and the cooperative cache hit rate at time t are calculated according to six modeling conditions,

[0099]

[0100]

[0101] Further, the delay cache benefit is defined:

[0102]

[0103]

[0104] wherein j represents a responding node within the range of the home base station, and k represents a responding node within the range of the cooperative base station, represents the hop count when the node i requests the content f and the responding node is k at time t.

[0105] The cache benefit of the base station h at time t is calculated according to the cache hit rate and the delay cache benefit, and the maximum system future cache benefit is defined:

[0106]

[0107]

[0108]

[0109]

[0110] wherein a and β are parameters for controlling the proportion of local benefit and cooperative benefit, a+β=1 and a>β, so that the benefit of cache hit in the local is more important than the benefit of cache hit through the cooperative base station.

[0111] Step S3: The global content popularity and the dynamic popularity threshold of the current base station area are calculated to obtain the globally popular content in the current base station and cache the content in the base station.

[0112] Specifically, the global content popularity and the dynamic popularity threshold of the current base station area are calculated according to an exponential weighted moving average model:

[0113]

[0114]

[0115] wherein, represents the global popularity of the content f at time t, represents the number of requests of the content f at time t, represents a weight parameter, Tt Indicates the dynamic popularity threshold; when Greater than T t At that time, content f is considered as globally popular content, and then the globally popular content is cached in the base station, updating the base station cache information.

[0116] Step S4: Treat the base station as an intelligent agent to learn user preferences in real time, model the user's long-term and short-term interests, and then combine the current user and base station cache state information as state input to establish cache decision.

[0117] Specifically, this involves extracting users' long-term interests from their high-scoring feedback on certain content, and representing the content for which users have given high-scoring feedback as their long-term interests, denoted as S. long ={i1,i2,…,i L}. A Long Short-Term Memory (LSTM) network is used to learn high-scoring user evaluations over a longer period. First, information h from the hidden state of the previous time step is processed. t-1 and the currently input information x t Simultaneously input into the forget gate f t The sigmoid function σ controls which data should be forgotten. The output value is between 0 and 1; the closer to 0, the more data should be forgotten, and the closer to 1, the more data should be retained.

[0118] f t =σ(W f ·[h t-1 ,x t ]+b f (10)

[0119] Then, according to h t-1 and x t Calculate the input gate i t and temporary unit states

[0120] i t =σ(W i ·[h t-1 ,x t ]+b i (11)

[0121]

[0122] Next, calculate the long-term cell state C at the current moment. t ,

[0123]

[0124] Finally, calculate the output gate o. t and the current hidden state information h t ,

[0125] o t = σ(W o ·[h t-1 ,x t ]+b o ) (14)

[0126] h t = o t *tanh(C t ) (15)

[0127] The input vector x t is S long = {i1, i2, …, i L}, and the base station integrates the output vectors of all users at time t into a user long-term interest matrix

[0128] where W f , W i , W C , W o are the weights of the forget gate, the input gate, the temporary unit state, and the output gate, respectively, and b f , b i , b C , b o are the biases of the forget gate, the input gate, the temporary unit state, and the output gate, respectively.

[0129] The content with a high request frequency in a short period of time of a user can be considered as the short-term interest of the user, and the short-term interest can be extracted from the content ranked in the top N items in the current time node popularity, i.e., S short = {i1, i2, …, i N}. The gated recurrent unit GRU is used to capture the short-term interest of the user.

[0130] The user node popularity calculation formula is:

[0131]

[0132] The GRU uses an update gate z t to update the state and uses a reset gate r t to control whether the information of the previous sequence is important:

[0133] z t = σ(x t W xz +h t-1 W hz +b z ) (17)

[0134] r t = σ(x tW xr +h t-1 W hr +b r ) (18)

[0135]

[0136]

[0137] GRU current time input vector S short = {i1, i2, …, i N}, output vector is the final hidden state h t , the base station will integrate all users t time short-term interest vector into a short-term interest matrix

[0138] In the formula, W xz , W hz The weights of the update gate and the last hidden state to the current input vector, W xr , W hr The weights of the reset gate and the last hidden state to the current input vector, W xh , W hh The weights of the candidate hidden state and the last hidden state to the current input vector, b z , b r , b h The bias of the update gate, the reset gate and the candidate hidden state, The candidate hidden state.

[0139] Step S5: First, the cache decision in step S4 is modeled by Markov decision process, define a four tuple M = <S, A, P, R> composed of state space S, action space A, state transition P and reward function R, wherein,

[0140] State space: at each decision time t, the base station h extracts the environment state from the recorded information, and the user long-term interest matrix Short-term interest matrix User cache state information X t and base station cache state information Y t As input state, expressed as

[0141] Action space: at t time, the base station h needs to decide the cache placement problem of content f, expressed as Wherein, a u,f = 1 indicates caching content f at user u, a u,f = 0 does not cache;

[0142] State transition: After the system updates the cached content at time t, it records the user's request information and feedback information, and the state transitions to S. t+1 That is, to update the user's short-term and long-term interests and cached state information at time t+1;

[0143] Reward function: After executing the caching decision action, each base station will calculate the reward value obtained at the current time. The reward that base station h receives after performing the caching action at time t is defined as the caching revenue at time t.

[0144] Then, the base station is used as an agent to observe the environmental state. The Soft Actor-Critic algorithm is extended using the CTDE framework to obtain a multi-agent deep reinforcement learning algorithm based on maximum entropy, which learns the optimal cache placement scheme in real time. Specifically:

[0145] The actor network of the base station is denoted as π = {π1, ..., π}. B The parameters of each actor network are θ = {θ1, ..., θ}. B}, The parameter is θ h actor network π h Similarly, using The parameter is w h The critical network Q h According to the CTDE framework and SAC, base station h minimizes the target Update the actor network:

[0146]

[0147] In the formula, This represents the experience revisit pool, used to store the base station's state x, action a, reward r, and next state x', using quadruples.<x,a,r,x'> As A record in the memory, where x = {s1, ..., s h ,…,s B}, a={a1,…,a h ,…,a B}, r={r1,…,r h ,…,r B}, x'={s1',…,s h ',…,s B '}. Input is from The states x and actions observed by all base stations obtained from sampling in The output is used to perform buffering actions for the base station. The Q value.

[0148] Actor network with decentralized execution Only need to use the observation s of the base station itself h The probability distribution of the action can be obtained, and the base station obtains the specific execution action by sampling the obtained probability distribution The sampling method using the reparameterization method can obtain the differentiable form of the target:

[0149]

[0150] In order to make it still differentiable after sampling, let The action probability distribution is sampled and the sampling operation is performed. The expectation in can be approximated by sampling from and using the stochastic gradient descent method to update the actor network of the base station h.

[0151] The parameters w of the critic network of the base station h can be updated by minimizing the Bellman error J Q (w h ) of the base station:

[0152]

[0153] In the formula, is the parameter of the target critic network of the base station h, is the parameter of the target actor network of the base station h, and in , a fixed batch of samples is obtained by random sampling to approximate the calculation of J Q (w h ), wherein the base station h inputs the observation s h ′ of the next state into the target actor network and then samples according to the obtained action probability distribution to obtain the next action a h ′ of the base station h, that is a′={a′1,…,a′ h ,…a′ B}.

[0154] Each base station has four deep neural networks, namely the actor network, the critic network, and the target actor network and the target critic network. During the training process, only the actor network and the critic network are trained, and the target actor network and the target critic network are used to stabilize the learning effect of the actor network and the critic network.

[0155] The actor network and the target actor network of the base station h utilize the current input environment state s of the base station h and the next observed state s h to generate the current action a h and the target action a' h The input of the critic network is the current all base station observed state x and action a, and the output is the Q value Q of the action of the base station h h The input of the target critic network is the next state base station observation x' and action a', and the output is the Q value TQ of the target action of the base station h h .

[0156] Meanwhile, after each actor network and critic network parameter update, the target actor network and the target critic network need to be soft updated, for ensuring stable operation of the algorithm:

[0157]

[0158]

[0159] In the formula, tau is a hyperparameter for controlling soft update of the target network.

[0160] The beneficial effects of the present application are as follows:

[0161] 1. The method proposed in the present application can make optimal caching decisions in real time according to network states, and can simultaneously optimize network resource utilization, reduce caching redundancy, in the case of guaranteeing caching hit rate and improving user experience quality;

[0162] 2. The present application calculates the caching hit rate by modeling six caching hit conditions, and then learns the user's preferences in real time using the base station, caches personalized content at the user end, effectively improves the edge caching hit rate and caching diversity, and reduces the delay of user request video content;

[0163] 3. By combining ICN and MEC technologies, user mobility is supported, network duplicate traffic is reduced, and the efficiency of service automated deployment can be improved.

[0164] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. An ICN-based mobile edge computing system video stream autonomous edge caching method, characterized in that, The method comprises the following steps: Step S1: constructing an ICN-based mobile edge computing network architecture model, the edge layer and the user layer being connected using an ICN architecture to communicate in the form of interest packets and data packets, and the edge layer and the cloud layer communicating through the physical connection of the core network; Step S2: proposing a distribution mode of user content requests, modeling the content cache hit in the distribution mode, and calculating the local cache hit rate and the cooperative cache hit rate at the current time; Step S3: calculating the global content popularity and the dynamic popularity threshold of the current base station area, obtaining the globally popular content in the current base station, and caching the content in the base station; Step S4: taking the base station as an agent, learning the user preferences in real time, modeling the long-term and short-term interests of the user, and then combining the cache state information of the user and the base station at the current time as the state input to establish a cache decision; Step S5: modeling the cache decision by a Markov decision process, proposing a multi-agent deep reinforcement learning algorithm based on the Soft Actor-Critic algorithm to train the model, and obtaining an optimal cache placement scheme.

2. The method of claim 1, wherein, The mobile edge computing network architecture model comprises a cloud server storing all content that can be requested by users, a content library containing f video files each having a size of δ, and being capable of communicating with the base stations of the edge layer through the physical connection of the core network; The edge layer has N base stations, each base station covering a plurality of users, and the base station being responsible for managing the cache information of the users in the coverage area to facilitate cooperative caching; The users in the user layer belong to only one base station at the same time, and periodically update the cache information to the base station, and the user layer and the edge layer are connected through the ICN architecture, the user sends a request content in the form of an interest packet, and the cache node responds in the form of a data packet.

3. The method of claim 2, wherein, The content stored in the cloud server consists of a collection This indicates that it includes user ratings. ,in, Indicates user For content The rating, with a value of 0 representing the user's rating. No file A rating of 0.5-5 represents the user's level of preference for the file, with higher values ​​indicating a stronger preference. (Base station) Storage space size is The set of users within the coverage area of ​​a base station is Storage space size is The user equipment and base station are connected based on an ICN architecture, and both maintain three lists: A content storage table is used to store the content copies cached by the nodes. A pending request table is used to record the information packets that have been sent out but have not been received, and the corresponding incoming interfaces. A forwarding information table is used to record the information packets that have been sent out and the corresponding forwarding interfaces.

4. The method of claim 3, wherein, The distribution mode of user content requests includes six cases: Case 1, when the user requests a certain content , a copy of the content is cached in the content storage table of the user's own ; Case two, when the user does not cache the copy of the content itself, it broadcasts an interest packet to other user devices within its communication range, if there is a user device that caches the copy of the content within its broadcast range, the caching user returns the data packet directly to the requesting user; Scenario 3: When no cached content exists within the user's broadcast range. The device requests the user to send the interest packet to the home base station. If the home base station has cached the content, the home base station will return the data packet to the requesting user. Case four, when the home base station does not find the content cache information, it looks for other user devices in its coverage range, if it finds a device that caches the content, it communicates with the device and receives the data packets in the cache device, and then transmits them to the requesting user. Case five, when the home base station does not find other devices that have cached the content , it broadcasts an interest packet to neighboring base stations, which perform the same operations as in cases three and four. Case six: when none of the above cases finds an interest packet, the base station sends a request to the cloud.

5. The method of claim 4, wherein, The modeling of the content cache hit in the distribution mode is as follows: Within each time slot t, each user's content request ,in On behalf of users exist Content requested at any time Number of times; users exist Cache information at any time ,in On behalf of users Content cached Base station h in Cache information at any time ,in This indicates that base station h has cached the content. Define a set of binary variables Record the process of fulfilling a user's request: where h denotes the home base station of user i, denotes the set of neighbor users of user i.

6. The method of claim 1, wherein, After calculating the local cache hit rate and the cooperative cache hit rate at the current time, the cache revenue is defined, and the maximum cache revenue objective function is defined.

7. The method of claim 1, wherein, In step S3, the expressions of the global content popularity and the dynamic popularity threshold are as follows: wherein, represents the global popularity of content at time t, represents the number of requests for content at time t, represents a weight parameter, represents a dynamic popularity threshold; when is greater than content is considered a globally popular content.

8. The method of claim 1, wherein, In step S4, the long-term interest of the user is extracted from the high-score feedback of the user on some content, denoted as The long-term interest of the user is learned by using a long short-term memory network to learn the high-score evaluation records of the user for a long period of time to capture the long-term interest of the user, and then integrated into a user long-term interest matrix: wherein, is a forget gate, is a sigmoid function, , and are respective weights, is a previous time hidden state information, is a current input information, , , and are respective biases, is an input gate, is a temporary cell state, is a current time long-term cell state, is a previous time long-term cell state, is an output gate, is a current time hidden state information; The content requested by the user frequently in a short time can be considered as the short-term interest of the user, and the short-term interest can be extracted from the content ranked in the top N items in the current time node popularity, that is The short-term interest of the user is captured by using a gating recurrent unit, and then integrated into a user short-term interest matrix: wherein, is a user popularity, is an update gate, is a reset gate, , are weights of the update gate for the current input vector and the previous hidden state, respectively, , , , are weights of the candidate hidden state for the current input vector and the previous hidden state, respectively, , , are biases of the update gate, the reset gate and the candidate hidden state, respectively, is the candidate hidden state.

9. The method of claim 1, wherein, The cache decision is modeled as a Markov decision process, specifically, a four tuple consisting of a state space S, an action space A, a state transition P, and a reward function R wherein, State space: At each decision time t, the base station h extracts the environment state from the recorded information, the user long-term interest matrix , the short-term interest matrix , the user buffer state information and the base station buffer state information as input state, denoted as ; Action space: At time t, the base station h needs to decide the caching placement problem for content f, denoted as ; State transition: after the system updates the cache content at time t, the request information and feedback information of the user are recorded, and the state is transferred to , i.e., the long-term and short-term interests of the user at time t and the cache state information are updated . Reward function: after executing the cache decision action, each base station will calculate the reward value obtained at the current time, and the reward obtained by the base station h after executing the cache action at time t is defined as the cache revenue at time t.

10. The method of claim 1, wherein, The base station is taken as an agent, each base station includes an actor network, a critic network, a target actor network and a target critic network, and only the actor network and the critic network are trained: according to the CTDE framework and the SAC algorithm, the parameters of the actor network are updated by minimizing the target, the differentiable form of the minimization target is obtained by using the reparameterization sampling method, and then the random gradient descent method is used to realize the update of the actor network; the parameters of the critic network are updated by minimizing the Bellman error of the base station.

Images