An intelligent caching method for practical heterogeneous cellular networks

Abstract

The present invention relates to the field of data processing technology, and in particular to an intelligent caching method for practical heterogeneous cellular networks. The method comprises: S1: during initialization, a small base station k (SBS k) caches the content in a file library at least once; S2: SBS k obtains the initial value required for estimating the content by observing the reward of the cached content; and S3: each SBS performs steps in each time slot, etc. The intelligent caching method proposed by the present invention, in the content caching research of SBS, models the content caching problem of unknown content popularity as a random combination MAB problem, and mainly solves three sub-problems: uncertainty, the impact of the actual size of the content on the performance of the caching algorithm, and the time slot length problem.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent caching method for a practical heterogeneous cellular network. Background Art

[0002] With the development of communication technology, a large amount of mobile data traffic is generated every day, especially in some emerging applications and video services. Therefore, in order to meet the rapid growth of mobile data traffic, heterogeneous cellular networks (HetNets) are considered to be an effective method. In HetNets, short-range communication services are provided to users by deploying small base stations (SBSs) and macro base stations (MBSs), thereby improving network capacity and spectrum utilization. However, the high-speed link cost required for HetNet to connect to the core network is extremely high, and the link quality seriously affects the user's quality of experience. In addition, an interesting phenomenon is that a small part of the content accounts for the vast majority of mobile data traffic. Therefore, caching the most popular content on SBS has become an effective way to improve user quality and reduce backhaul traffic. In the content caching problem of SBS, a common indicator for designing an optimal caching strategy is the content popularity profile, which refers to the probability of users requesting each file, that is, user preference. However, in practice, user preferences cannot be known in advance. Considering this realistic condition, researchers use online learning methods to learn user preferences step by step. In the online learning method, the learning agent is the MBS. The entire time is divided into multiple fixed-length time slots. In each time slot, the MBS formulates a caching strategy for each SBS and then observes the instantaneous demand for cached files, that is, the number of requests for each cached file. The instantaneous demand is regarded as the reward of the caching strategy. If the user requests a cached file, then a unit of reward will be obtained.

[0003] In recent years, reinforcement learning techniques have been used to solve the content caching problem with unknown content popularity. Reinforcement learning is usually based on Markov decision processes (MDPs) or multi-armed bandits (MABs). MABs are typical single-state (or stateless) models, while MDPs usually model the problem as a multi-state decision process, i.e., the states of the model affect the decision (e.g., different states have different action sets) and evolve after each decision. However, the performance of these MDP-based RL algorithms is not theoretically guaranteed, i.e., it is impossible to mathematically prove that they converge to the optimal solution. On the other hand, MAB can simply solve the content caching problem without modeling MDP, and the performance can be guaranteed in theory. More importantly, in practice, MAB algorithms are usually easier to implement. For example, the upper confidence bound (UCB) algorithm is one of the most famous MAB algorithms, which only spends nlog(n) (n is the total number of arms) in each time slot. Therefore, many works in the content caching problem of SBS are based on MAB. In the research on content caching problems without prior knowledge, these problems are modeled as random combination MAB problems or random context MAB problems. In the random context MAB problem, the learning agent learns an optimal caching strategy based on the user's context information. The user's context information includes their age, gender, location, etc. However, in real life, users are reluctant to disclose context information in order to protect their privacy, and it is difficult for network operators to collect this context information. Therefore, the present invention does not consider modeling the content caching problem as a random context MAB problem.

[0004] In solving the content caching problem of SBS, the prior art firstly only considers user preferences and ignores the actual size of the file. However, in actual systems, the actual size of the content is different, such as multimedia files, which have many types, such as movies, short videos, and music, and different types have different sizes. In addition, due to the limited cache capacity of SBS, the size of the content will affect the performance of the caching scheme; secondly, they divide the entire service time of each SBS into multiple fixed-length time slots. However, in HetNet, each SBS covers a different area, and users in different areas have different preferences, that is, they may like different file types. For example, young residents like to watch short videos, while elderly residents prefer TV programs. Due to the different sizes of different file types, the time required for users to use them is also different. If it is assumed that the time slot length allocated to each SBS is the same, the following situations will occur: One of the situations is that the allocated time slot is too short for the users served by a certain SBS, and most users still use the content in the previous time slot, resulting in a significant reduction in the available users in the current time slot (i.e., users who can request files in the request phase). Therefore, there is not enough knowledge to learn content popularity, resulting in a decrease in learning efficiency; another situation is that the allocated time slot is too long for the users served by a certain SBS. Most users have consumed the content requested in the current time slot and need to wait for the arrival of the next time period, thereby extending the total learning time of the learning agent. Therefore, it is necessary to provide a solution to divide the time slot length according to the user preferences of a certain SBS service. Summary of the invention

[0005] In view of this, the present invention discloses an intelligent caching method for practical heterogeneous cellular networks to solve the problem of ignoring file size and time slot length in the existing content caching problem of unknown content popularity, and models the content caching problem of unknown content popularity as a random combination MAB problem.

[0006] The technical solution provided by the present invention is specifically an intelligent caching method for a practical heterogeneous cellular network, comprising the following steps:

[0007] S1: SBS k caches the contents of the file library at least once during initialization;

[0008] S2: SBS k obtains the initial value required to estimate the content by observing the reward of the cached content and

[0009] S3: Each SBS performs the following steps in each time slot:

[0010] (i) MBS uses the previous time slot and As input for the current time slot;

[0011] (ii) MBS calculates the UCB evaluation of SBS k and formulates a caching strategy for SBS k;

[0012] (iii) SBS k refreshes the cache according to the cache strategy determined by MBS and broadcasts the cached content list to the served users; all users consume their favorite content, and SBS k observes and reports the rewards of the cached content to MBS;

[0013] (iv) MBS Update and And call scheme A to obtain the length of the next time slot, wherein the scheme A dynamically allocates an appropriate time slot length to the SBS according to the preferences of users located in the same SBS, and its purpose is to calculate the length of the next time slot.

[0014] Furthermore, we use reinforcement learning based on MAB to learn the file popularity, and adopt the UCB method to construct a confidence radius around the estimate to handle uncertainty;

[0015] Use θ to denote the file content popularity profile, and define the upper confidence bound of SBSk for file f in time slot t as:

[0016]

[0017] in, is the number of times file f is selected, is the average experience reward of cache file f when running UCB, and The value needs to be continuously obtained and updated. MBS updates in S3 and The rules are as follows:

[0018]

[0019]

[0020] Further, in S3, MBS calculates the UCB evaluation of SBS k based on formula (3);

[0021] Set the file size to S f Added to the UCB indicator calculation formula; UCB indicator calculation formula Multiply it by the inverse of the file size to form a new indicator calculation formula, recorded as This formula represents the popularity value per unit file size; The representation is as follows:

[0022]

[0023] Furthermore, the specific steps of scheme A are:

[0024] (i) Set a stack and a queue for each SBS as the input of Scheme A, where the stack stores the probability value of each file type being requested by the user in the current time slot, and the queue stores the number of times each file type is requested by the user in the current time slot;

[0025] (ii) At the end of each time slot, MBS calculates the probability value of each file type being requested by the user based on the user's request queue, arranges them in descending order and pushes them into the stack in sequence;

[0026] (iii) Determine whether there is an element with a value greater than 50 in the stack. If so, set the length of the next time slot to the time required for the file type to be consumed; otherwise, set the length of the next time slot to the maximum value of the time required for all file types to be consumed.

[0027] This paper proposes an intelligent caching method for practical heterogeneous cellular networks. The difference from other works is that this paper considers the impact of file size on cache performance and the length of each time slot. In the content caching research of SBS, the content caching problem of unknown content popularity is modeled as a random combination MAB problem. This solution mainly solves three sub-problems: uncertainty, the impact of the actual size of the content on the performance of the cache algorithm, and the time slot length problem.

[0028] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 It is a schematic diagram of an application scenario of the intelligent caching method for a practical heterogeneous cellular network provided by the present invention;

[0032] Figure 2 It is an experimental scene diagram of the optimal solution described in the disclosed embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the results of a comparative simulation experiment disclosed in an embodiment of the present invention;

[0034] Figure 4 It is a schematic diagram of average reward and accumulated regret of each scheme under different SBSs cache capacities disclosed in the embodiment of the present invention;

[0035] Figure 5 It is a schematic diagram of average reward and accumulated regret of each scheme under different popularity parameters γ disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of systems consistent with some aspects of the present invention as detailed in the appended claims.

[0037] In order to solve the problems of ignoring file size and time slot length in the content caching problem of unknown content popularity, this embodiment models the content caching problem of unknown content popularity as a random combination MAB problem, and provides an intelligent caching method for actual heterogeneous cellular networks. When designing an effective caching method, this embodiment takes three practical problems into consideration. The first is uncertainty (no prior knowledge about file popularity). Second, due to the limited cache capacity of the SBS, the actual size of the content will affect the performance of the caching scheme. Therefore, the actual size of the content should be considered when designing a caching method. Third, the time slot length of each SBS should be considered because users in different areas have different hobbies. Figure 1 This is an application scenario of the intelligent caching solution for actual heterogeneous cellular networks provided by the present invention. In this scenario, the present invention provides an intelligent caching method for actual heterogeneous cellular networks, which can effectively solve the problem of considering file size and time slot length in the content caching problem of unknown content popularity. The specific content is:

[0038] First of all, for the first problem of not knowing the popularity of the file in advance, a simple idea is that it can be estimated. Therefore, there will be a dilemma between utilization (pulling the empirically best arm to obtain the highest instantaneous reward) and exploration (estimating the reward of the other arm to reduce the impact of uncertainty). The present invention uses a reinforcement learning scheme based on MAB to learn file popularity. In the MAB scheme, the UCB scheme can better balance utilization and exploration. At the same time, the UCB scheme is not only easy to implement, but also has good theoretical guarantees. Therefore, the UCB scheme is adopted as the basis of the intelligent caching scheme proposed in the present invention. The core idea of ​​the UCB scheme is to construct a confidence radius around the estimate to deal with uncertainty. Use θ to mark the file content popularity profile file, then define the upper confidence bound of SBSk for file f in time slot t as:

[0039]

[0040] in, is the number of times file f is selected, is the average experience reward of cache file f. When running UCB, and The values ​​of need to be continuously obtained and updated. The update rules for these two values ​​are as follows:

[0041]

[0042]

[0043] For the second problem, the present invention converts the file size (S f ) is added to the UCB indicator calculation formula, so that MBS can consider not only the content popularity but also the actual size of the file when formulating cache strategies for SBSs. Specifically, the UCB indicator calculation formula Multiply it by the inverse of the file size to form a new indicator calculation formula, recorded as This formula represents the popularity value per unit file size. The advantage of the above modification is that the MBS considers not only the content popularity but also the actual size of the file when formulating the caching strategy for the SBSs. The representation is as follows:

[0044]

[0045] For the third problem, the present invention considers that users in different areas have different hobbies, and their hobbies may change over time. Therefore, the present invention designs a scheme to dynamically allocate appropriate time slot lengths for each SBS according to the hobbies of users in the same area, which is recorded as Scheme A. The scheme will calculate the length of the next time slot, and its specific steps are: (i) First, a stack and a queue are set for each SBS as the input of Scheme A, where the stack stores the probability value of each file type being requested by the user in the current time slot, and the queue stores the number of times each file type is requested by the user in the current time slot. (ii) Secondly, at the end of each time slot, the MBS calculates the probability value of each file type being requested by the user based on the user's request queue, arranges them in descending order and pushes them into the stack in sequence. (iii) Next, it is determined whether there is an element with a value greater than 50 in the stack. If so, the length of the next time slot is set to the time required for the file type to be consumed, otherwise the length of the next time slot is set to the maximum value of the time required for all file types to be consumed.

[0046] After solving the above three problems, the present invention proposes an intelligent caching method for practical heterogeneous cellular networks, called RETINA. This method can effectively solve the problem of considering file size and time slot length in the content caching problem of unknown content popularity. The specific steps of this scheme are as follows: For SBS k, the content in the file library is cached at least once during initialization, and then SBS k observes the reward of the cached content to obtain the initial value required to estimate the content and Next, each SBS performs the following steps in each time slot (here taking SBS k as an example): (i) MBS uses the data obtained in the previous time slot and as the input of the current time slot. (ii) MBS calculates the UCB evaluation of SBS k based on formula (3) and formulates a cache strategy for SBS k. (iii) SBS k refreshes the cache according to the cache strategy determined by MBS and broadcasts the cached content list to the served users. All users consume their favorite content, and then SBS k observes and reports the reward of cached content to MBS. (iv) MBS updates according to formula (2) and And call scheme A to obtain the length of the next time slot.

[0047] This implementation scheme simulates the proposed intelligent caching method for actual heterogeneous cellular networks on pycharm and compares it with ε-greedy and other UCB-based algorithms (UCB-CS, MCUCB). Compared with these algorithms, the present invention can increase the total reward of the edge cache system by 5%-15%. In addition, in order to better verify the performance of these solutions under the condition of unknown content popularity, the present invention uses a performance upper bound solution, called the optimal solution. The experimental scenario is as follows Figure 2 As shown in the figure, the scenario has one macro base station and three small base stations. The users served by the three small base stations are 30, 50, and 70 respectively. Other parameters are set as follows: the total number of files in the file library is 1000, the sum of the sizes of all files is 50000MB, and the size of each file is randomly set from 10MB to 100MB. The cache capacity of each SBS is 3000MB, which is 6% of the total file size. There are 5 types of files, and the length consumed by each type of file is a different time slot. In addition, the two indicators of average reward and cumulative regret are used to evaluate these schemes.

[0048] In the attached Figure 3 The results of the comparative simulation experiment are shown in , which shows the average reward and cumulative regret of the five algorithms. The number of users is different in these algorithms. It can be seen from the figure that as the number of users increases, the average reward of each scheme also increases. This is because the rewards of these schemes are set according to immediate needs, and the trend of popular content has not changed. In addition, the optimal scheme is better than other schemes because it knows the expectations of all content rewards in advance, and it will choose the best content every time. The proposed scheme RETINA is better than other online learning schemes because it considers the impact of file size on cache performance and adds content size to the indicator calculation formula to obtain better results.

[0049] Attached Figure 4 The average reward and cumulative regret of each scheme under different SBSs cache capacity are shown. The cache capacity of SBSs is set to 5%, 10%, and 15% of the total file size, respectively. As can be seen from the figure, as the SBS cache capacity increases, the average reward will also increase. This is because if the cache capacity of SBS increases, more users will be able to access their favorite content through SBS, thereby increasing the reward of the cache system. There is no doubt that the optimal scheme has the best performance. In addition, in addition to the optimal scheme, the RETINA scheme is better than other cache schemes because it takes into account the content size due to the limited cache capacity of SBS, so it will make the total reward of the system as large as possible. In short, the three capacity sizes set are also more realistic. Therefore, the RETINA scheme proposed in the present invention is effective and practical.

[0050] Attached Figure 5 The average reward and cumulative regret of each scheme under different popularity parameters γ are shown. The popularity parameters are set to 0, 0.5, 0.75, and 1 respectively. It can be seen from the figure that as the popularity parameter value increases, the performance of these caching schemes becomes better and better. This is because file requests are increasingly concentrated on a small number of files, and the estimation of content demand usually plays a positive role. In addition, it can be seen that the optimal scheme has the best performance because it can know all the cache information in advance, thereby obtaining the best cache strategy. For other schemes, when γ = 0, it means that the content requested by the user is relatively scattered. Due to the limited cache capacity of SBSs, the performance of these schemes is not very good. However, the RTETINA scheme proposed in the present invention is better than other schemes because RETINA takes into account the impact of content size on cache performance. When the content requested by the user is relatively scattered, the size of the content plays a certain role. As the γ value increases, the performance of the RETINA scheme of this scheme becomes closer and closer to the UCB-CS scheme, especially when γ = 0. This is because the cache capacity of the SBS may not be less than the size of the popular content as the requested content is increasingly concentrated on very few contents. In summary, the applicable γ value range of RETINA proposed in the present invention is [0, 1], which is also of practical significance.

[0051] The above experimental results show that the present invention provides an intelligent caching method for practical heterogeneous cellular networks. Compared with other solutions, the solution provided by the present invention can increase the total reward of the edge cache system by 5%-15%, and the solution provided can also obtain a sublinear regret.

[0052] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These changes and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent caching method for a practical heterogeneous cellular network, characterized in that: The steps include: S1: During initialization, SBS k caches the contents of the file library at least once; S2: SBS k obtains the initial value required to estimate the content by observing the reward of the cached content and S3: Each SBS performs the following steps in each time slot: (i) MBS uses the previous time slot and As input for the current time slot; (ii) MBS calculates the UCB evaluation of SBS k and formulates a caching strategy for SBS k; (iii) SBS k refreshes the cache according to the cache strategy determined by MBS and broadcasts the cached content list to the served users; all users consume their favorite content, and SBS k observes and reports the rewards of the cached content to MBS; (iv) MBS Update and and calling scheme A to obtain the length of the next time slot, wherein the scheme A dynamically allocates an appropriate time slot length to the SBS according to the preferences of users located in the same SBS, and its purpose is to calculate the length of the next time slot; Reinforcement learning based on MAB is used to learn file popularity, and a confidence radius is constructed around the estimate using the UCB method to handle uncertainty; Use θ to denote the file content popularity profile, and define the upper confidence bound of SBSk for file f in time slot t as: in, is the number of times file f is selected, is the average experience reward of cache file f when running UCB, and The value needs to be continuously obtained and updated. MBS updates in S3 and The rules are as follows: In S3, MBS calculates the UCB evaluation of SBS k based on formula (3); Set the file size to S f Added to the UCB indicator calculation formula; UCB indicator calculation formula Multiply it by the inverse of the file size to form a new indicator calculation formula, recorded as This formula represents the popularity value per unit file size; The representation is as follows: The specific steps of Scheme A are: (i) Set a stack and a queue for each SBS as the input of Scheme A, where the stack stores the probability value of each file type being requested by the user in the current time slot, and the queue stores the number of times each file type is requested by the user in the current time slot; (ii) At the end of each time slot, MBS calculates the probability value of each file type being requested by the user based on the user's request queue, arranges them in descending order and pushes them into the stack in sequence; (iii) Determine whether there is an element with a value greater than 50 in the stack. If so, set the length of the next time slot to the time required for the file type to be consumed; otherwise, set the length of the next time slot to the maximum value of the time required for all file types to be consumed.

Images