A method for constructing an optimal network profit allocation mechanism based on edge caching

Abstract

This invention discloses a method for constructing an edge-caching-based network profit optimization distribution mechanism, comprising the following specific steps: constructing a joint ISP-CP profit model based on a system model of the ISP-CP joint content distribution mechanism; and solving the ISP-CP joint profit model using reinforcement learning. This method provides an edge-caching-based network profit optimization distribution mechanism. By constructing separate cost-revenue models for ISPs and CPs, a reasonable ISP-CP total profit model formula is constructed, and reinforcement learning is used to optimize the joint content distribution mechanism. This method ensures that content finds the shortest possible transmission path, thereby reducing transmission costs and achieving optimal profits while ensuring efficient network content distribution.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and more particularly to a method for constructing a network profit optimization distribution mechanism based on edge caching. Background Art

[0002] With the increasing popularity of the internet, the continued development of Wi-Fi, the growth of major operators, and the widespread adoption of network services, the number of global internet users continues to grow, driving exponential growth in mobile cellular network traffic. While this generates significant traffic revenue for content providers (CPs) and Internet service providers (ISPs), it also places pressure on existing networks in areas such as base station deployment costs, operating costs, and user experience. Analysis of how collaborative content distribution between CPs and ISPs can reduce network costs while achieving latency targets reveals significant advantages in ISP-CP collaboration, enabling more efficient content distribution through incentive mechanisms.

[0003] Through network architecture analysis, content-centric networking (CCN), a next-generation network architecture that represents a disruptive shift from traditional IP networks, has garnered widespread attention from scholars both domestically and internationally. This has spurred a wealth of research on network architecture. Existing research has examined the design choices and capabilities of content-oriented CCN network architectures, demonstrating their feasibility and security advantages across both private platforms for home content sharing and platforms like Haitian Information Network.

[0004] Due to redundant transmission issues in the network, obtaining content from a local edge cache does not require link and node transmission. To improve edge caching technology, researchers have proposed various online caching and collaborative forwarding solutions in software strategies to increase cache utilization and reduce network latency. Regarding caching hardware, researchers have proposed hardware-based caching technologies to reduce overall costs. While these studies have optimized base station caching, they have not optimized caching from a user perspective, considering that user requests are influenced by content popularity.

[0005] Therefore, how to provide an ISP-CP joint profit model that takes edge caching and content popularity into account, and use reinforcement learning to further optimize the joint content distribution mechanism so that content can find the shortest path for transmission, thereby reducing transmission costs and obtaining optimal profits, while improving network service capabilities and achieving mutual benefit and win-win results for ISPs and CPs, is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0006] In view of this, the present invention provides a method for constructing an edge-caching-based network profit optimization distribution mechanism for ISPs and CPs. While achieving a win-win situation for both parties, the present invention effectively improves content distribution efficiency. Based on actual conditions, the present invention establishes a system model of the joint content network distribution mechanism of ISPs and CPs. The cost-profit model of each ISP and CP in the content distribution process is constructed using mathematical formulas and physical meanings, and the cost-profit model is solved to reduce transmission costs and obtain optimal profits.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for constructing an edge cache-based network profit optimization distribution mechanism includes the following specific steps:

[0009] Based on the ISP and CP joint content distribution mechanism system model, build an ISP and CP joint profit model;

[0010] The ISP and CP joint profit model is solved based on reinforcement learning.

[0011] Furthermore, the specific steps of the ISP and CP joint content distribution mechanism system model include:

[0012] Build a content popularity model based on Zipf distribution;

[0013] Construct a network model; the network model includes a network service and architecture model, and a network topology model.

[0014] Furthermore, the content popularity model is constructed by assuming that the number of content types is K and the video content is numbered from 1 to k. In a given fixed time, the total number of network requests is R, and the distribution of content popularity with content number k is:

[0015]

[0016] Among them, Zipf's skewness coefficient α represents the popularity of content.

[0017] Furthermore, the ISP and CP joint profit model is:

[0018]

[0019] Wherein, I0 is the profit of the ISP and CP joint profit model; Revenue for ISP revenue model; Revenue from the CP revenue model; is the cost of the ISP cost model, is the cost of the CP cost model.

[0020] Furthermore, the ISP revenue model includes incentive revenue to encourage ISPs to deploy caches near base stations, fees charged by ISPs for providing content transmission to CPs, and fees charged by ISPs for providing content transmission to CPs:

[0021]

[0022] in, is the number of requests from user i for content k, s k is the size of content k, P ISP is the average profit generated by the ISP per unit bandwidth, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, and I is the incentive fee paid by the CP to encourage ISPs to deploy caches near base stations.

[0023] Furthermore, the revenue of the CP revenue model comes from membership fees paid by end users:

[0024] Among them, M is the number of mobile users, P cp Membership fees paid by end users.

[0025] Furthermore, the cost of the ISP cost model comes from the cost of purchasing network bandwidth, network traffic processing cost, cache cost, and retrieval cost:

[0026]

[0027] in, is the number of requests from user i for content k, s k is the size of content k, C0 represents the average cost per unit of bandwidth purchased by the ISP, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, C is the number of ca To represent the cost of deploying a unit cache, C re To represent the retrieval cost of each user request arriving at the cache, C n To represent the average processing cost per unit of network traffic of the path node between the base station and CP, C1 is used to represent the average processing cost per unit of network traffic of the link between the base station and CP, and H j,cp Indicates the number of hops on the path between the base station and the CP.

[0028] Furthermore, the cost expenditure of the CP cost model comes from network traffic processing cost, network bandwidth cost, incentive cost, and retrieval cost:

[0029]

[0030] Among them, M is the number of mobile users, K is the number of content types, B is the number of base stations, is the number of requests from user i for content k, s k is the size of the content k, is a Boolean variable, P ISP is the average profit generated by ISP per unit bandwidth, C re To represent the retrieval cost of each user request arriving at the cache, C cp The average processing cost per unit of network traffic paid for by CP.

[0031] Furthermore, based on the constructed ISP and CP joint profit model, a reinforcement learning algorithm is designed to find the optimal path for content transmission, and the Q-learning algorithm is combined with the profit model to solve the problem.

[0032] Furthermore, the specific implementation of the Q-Learning algorithm is:

[0033] Set the topology background, use the adjacency matrix to represent the adjacency relationship of network nodes, initialize the Q table, and then directly replace and update the values in the Q table according to the Q table update strategy;

[0034] A Q-learning algorithm is used. When a content request fails to reach its target node, it randomly selects the next node to which it will be directed. The reward matrix is used to obtain the current reward for this action. A new Q-value is calculated based on this reward, and the Q-value for the corresponding position in the Q-table is updated. After repeated training, the resulting Q-table serves as a strategic basis for content request routing. This algorithm can determine the optimal action for a content request located at any node to reach its target node.

[0035] Through the above technical solution, it can be seen that compared with the existing technology, the present invention discloses a method for constructing a network profit optimization distribution mechanism based on edge caching. By constructing the respective cost-revenue models of ISP and CP, a reasonable ISP-CP total profit model formula is constructed, and reinforcement learning is used to optimize the joint content distribution mechanism, so that the content can find the shortest path for transmission, thereby reducing the transmission cost, and then obtaining the optimal profit, while ensuring the distribution efficiency of network content.

[0036] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0037] (1) Based on the actual situation, a system model of the joint content network distribution mechanism of ISP and CP was established, and the cost-profit model of ISP and CP in the content distribution process was expressed using mathematical formulas and physical meanings.

[0038] (2) A non-deployed cache strategy was set as the baseline control model. Two additional cache update strategies were set: Online and Offline. Offline served as the ideal optimal solution control model, while Online served as a more realistic real-time solution. This multifaceted strategy allows for systematic observation of the performance optimization results of the ISP and CP joint model.

[0039] (3) Using Python, we combined the reinforcement learning QL algorithm with the joint system model and plotted the performance curves for RL-Online and RL-Offline. By comparing them with the corresponding curves without reinforcement learning, we visualized the performance characteristics of the QL algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1 , a schematic diagram of the network model structure of the present invention;

[0042] Figure 2 , the total network profit of the present invention under different network topologies

[0043] Figure 3 ,The total network profit of the present invention under different total number of content requests;

[0044] Figure 4 ,The total network profit of the present invention under different cache sizes;

[0045] Figure 5 ,The total network profit of the present invention under different content popularity;

[0046] Figure 6 ,The total network profit of the present invention under different numbers of content types;

[0047] Figure 7 , a flow chart of the present invention. DETAILED DESCRIPTION

[0048] The following is a clear and complete description of the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0049] The embodiment of the present invention discloses a method for constructing a network profit optimization distribution mechanism based on edge caching. Figure 7 is a flow chart of a construction method according to an embodiment of the present invention, such as Figure 7 As shown, the process includes the following steps:

[0050] Step S101: Establishing a system model of the ISP and CP joint content distribution mechanism;

[0051] Step S102: Constructing a joint profit model between ISP and CP;

[0052] Step S103: Solving the ISP and CP joint profit model based on reinforcement learning.

[0053] Through the above steps, based on the ISP and CP joint profit model obtained by the above optimization, by constructing the respective cost-revenue models of ISP and CP, and using reinforcement learning to optimize the joint content distribution mechanism, the content can find the shortest path for transmission, thereby reducing transmission costs, and then obtaining the optimal profit, while ensuring the distribution efficiency of network content.

[0054] Example 1

[0055] In this embodiment, in the above step S101, constructing the system model includes:

[0056] Step S1011: constructing a content popularity model based on Zipf distribution;

[0057] Step S1012: Construct a network model; the network model includes a network service and architecture model and a network topology model.

[0058] Furthermore, step S1011 constructs a content popularity model.

[0059] It's important to understand that there are two paths for mobile users on the network to obtain requested content. First, the ISP provides access services, helping users access the network and receiving content requests. The ISP then determines whether the content required by the user is already cached in the base station. If the content required by the user has already been pre-stored in the base station cache at the edge of the network, the content is distributed directly from the base station cache to the end user. This path, with the help of edge caching, minimizes network costs and redundant content transmission, improves content distribution efficiency and user experience, and increases overall network profits.

[0060] When the content required by the user does not exist in the base station cache, the ISP provides navigation services to help the user find the information they need on the Internet, and routes the content from a distance through network links and nodes, downloading and transmitting the content required to the user across domains or networks.

[0061] We design a popularity model for online video content based on Zipf's Law. Assume the number of content types is K, and the video content is numbered from 1 to k. Within a given fixed timeframe, the total number of network requests is R, and the distribution of the popularity of content numbered k is:

[0062]

[0063] Among them, Zipf's skewness coefficient α represents the popularity of the content. The larger the value of α, the more concentrated the content requests are and the greater the number of requests for popular data.

[0064] Step S1012: Construct a network model.

[0065] In this embodiment, the service model includes three participants: the ISP, the CP, and the MC. In the traditional Internet, the ISP is solely responsible for transmitting all content requests from the CP to the MC, which results in higher power consumption for both the ISP and the CP. However, in an edge-caching-based network, the ISP can deploy an in-network cache within the access network, providing both network support and content delivery services. Clearly, the adoption of edge caching transforms the existing network service model. By satisfying the majority of content requests at the network edge, it can reduce power consumption, further improving service quality and the end-user experience.

[0066] like Figure 1 As shown in Figure 1, the edge cache is placed in the ISP's base station (BS), and the ISP and CP provide collaborative content delivery services to improve the efficiency of data distribution. In the network model, X ik is a Boolean variable indicating whether the cache deployed on BS i caches content k. If BS i caches content k, X ik If the value is 1, then it is 0; otherwise, it is 0. Content is sorted by popularity, and popular content can be cached in the BS to meet the needs of most end users. While the introduction of edge caching will provide additional caching capacity for ISPs, it can significantly reduce the traffic burden on the entire network. Therefore, the overall power efficiency of both ISPs and CPs can be significantly improved.

[0067] Step S102: The ISP and CP joint profit model is:

[0068]

[0069] Wherein, I0 is the profit of the ISP and CP joint profit model; Revenue for ISP revenue model; Revenue from the CP revenue model; is the cost of the ISP cost model, is the cost of the CP cost model.

[0070] Step S1021: The ISP revenue model includes incentive income for encouraging ISPs to deploy caches near base stations, charges for ISPs providing content transmission to CPs, and charges for ISPs providing content transmission to CPs.

[0071]

[0072] in, is the number of requests from user i for content k, s k is the size of content k, P ISP is the average profit generated by the ISP per unit bandwidth, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, and I is the incentive fee paid by the CP to encourage ISPs to deploy caches near base stations.

[0073] Furthermore, to more conveniently express user needs that are not met by edge caches using formulas, two Boolean variables are defined:

[0074]

[0075]

[0076] Step 1022: The revenue of the CP revenue model comes from membership fees paid by end users:

[0077] Among them, M is the number of mobile users, P cp Membership fees paid by end users.

[0078] Step 1023: The cost of the ISP cost model is derived from the cost of purchasing network bandwidth, network traffic processing cost, cache cost, and retrieval cost:

[0079]

[0080] in, is the number of requests from user i for content k, s k is the size of content k, C0 represents the average cost per unit of bandwidth purchased by the ISP, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, C is the number of ca To represent the cost of deploying a unit cache, C re To represent the retrieval cost of each user request arriving at the cache, C n To represent the average processing cost per unit of network traffic of the path node between the base station and CP, C1 is used to represent the average processing cost per unit of network traffic of the link between the base station and CP, and H j,cp Indicates the number of hops on the path between the base station and the CP.

[0081] Step 1024: The cost expenditure of the CP cost model comes from network traffic processing cost, network bandwidth cost, incentive cost, and retrieval cost:

[0082]

[0083] Among them, M is the number of mobile users, K is the number of content types, B is the number of base stations, is the number of requests from user i for content k, s k is the size of the content k, is a Boolean variable, P ISP is the average profit generated by ISP per unit bandwidth, C re To represent the retrieval cost of each user request arriving at the cache, C cp The average processing cost per unit of network traffic paid for by CP.

[0084] Step 103: Based on the established ISP and CP joint profit model, a reinforcement learning algorithm is designed to find the optimal content transmission path, and the Q-learning algorithm is combined with the profit model to solve the problem.

[0085] In terms of integration with the profit formula, the QL algorithm's main function is to transmit data locally and output the shortest hop count, which primarily affects the ISP's transmission cost in the profit model. The ISP's transmission cost formula is:

[0086]

[0087] By inserting counting variables for content transmission nearby and content transmission from the source server and storing the corresponding hop counts in the QL algorithm, the profit formula can be calculated effectively.

[0088] By inserting counting variables for content transmission near the source server and content transmission from the source server into the QL algorithm and storing the corresponding hop counts, the profit formula can be effectively calculated. While the requested content is still not reaching the target node, a step variable is defined. Each step increments by one, and the step value after the loop ends is the number of hops the requested content has taken from the request node to the target node. When the requested content of the requesting node can be obtained from the nearest node, the variable takefrom near node number is defined and incremented by one. After traversing all the request data for the requesting node, the final value of takefromnear node number is the number of nearby transmissions of the requesting node's content. Similarly, the number of request content transmitted from the source server can be calculated. Through this traversal count, the total profit of the profit model can be numerically solved.

[0089] Step 1031, background setting;

[0090] The 64-node US continental backbone network US64 is selected as the topological background, and the adjacency matrix is used to represent the adjacency relationship of network nodes. At the same time, the Q table is initialized as a 64 by 64 zero matrix. Subsequently, the values in the Q table are directly replaced and updated according to the Q table update strategy.

[0091] Step 1032, Q-learning algorithm setting;

[0092] If a content request does not reach its target node, the next node is randomly selected. The reward matrix is used to obtain the current reward for this action. A new Q-value is calculated based on this reward, and the Q-value for the corresponding position in the Q-table is updated. After repeated training, the resulting Q-table serves as a strategic basis for content request routing, identifying the optimal action for a content request located at any of the 64 nodes to reach the target node.

[0093] The performance of the method involved in the present invention is analyzed and compared in combination with the simulation experiment results.

[0094] In the simulation, five simulation strategies were explored, and the relationship between the total network profit and the network topology, the total number of content requests, the size of the network cache capacity, the popularity of network content, and the number of network content types was obtained. Figure 2-6 shown. Figure 2 is the total network profit under different network topologies, Figure 3 is the total network profit under different content request totals, Figure 4 is the total network profit under different cache sizes, Figure 5 is the total network profit under different content popularity, Figure 6 The total network profit for different content types is shown in Figure 2. As can be seen, after deploying edge caching, the proposed joint model can effectively increase the total network profit, significantly outperforming the profit without edge caching. The reinforcement learning algorithm performs similarly to the optimal solution under ideal conditions, demonstrating a profit advantage.

[0095] Five simulation strategies are used, namely online cache strategy, offline cache strategy, and ideal no-cache strategy. Five specific strategies based on these three strategies combined with power consumption models in two platform environments are designed, as shown in Table 1, to discuss the system performance.

[0096] Table 1

[0097]

[0098] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0099] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing an edge cache-based network profit optimization distribution mechanism, characterized in that: The specific steps include: Based on the ISP and CP joint content distribution mechanism system model, a joint profit model of ISP and CP is constructed; the joint profit model of ISP and CP is as follows: Wherein, I0 is the profit of the ISP and CP joint profit model; Revenue for ISP revenue model; Revenue from the CP revenue model; is the cost of the ISP cost model, is the cost of the CP cost model; Solve the ISP and CP joint profit model based on reinforcement learning; The ISP revenue model includes incentive revenue to encourage ISPs to deploy caches near base stations, fees charged by ISPs for providing content delivery to CPs, and fees charged by ISPs for providing content delivery to CPs. in, is the number of requests from user i for content k, s k is the size of content k, P ISP is the average profit generated by the ISP per unit bandwidth, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, and I is the incentive fee paid by the CP to encourage ISPs to deploy caches near base stations; The revenue of the CP revenue model comes from the membership fees paid by end users: Among them, M is the number of mobile users, P cp Membership fees paid by end users; The cost of the ISP cost model comes from the cost of purchasing network bandwidth, network traffic processing cost, cache cost, and retrieval cost: in, is the number of requests from user i for content k, s k is the size of content k, C0 represents the average cost per unit of bandwidth purchased by the ISP, is a Boolean variable, M is the number of mobile users, K is the number of content types, B is the number of base stations, C is the number of ca To represent the cost of deploying a unit cache, C re To represent the retrieval cost of each user request arriving at the cache, C n To represent the average processing cost per unit of network traffic of the path node between the base station and CP, C1 is used to represent the average processing cost per unit of network traffic of the link between the base station and CP, and H j,cp Indicates the number of hops in the path between the base station and the CP; The cost expenditure of the CP cost model comes from network traffic processing cost, network bandwidth cost, incentive cost, and retrieval cost: Among them, M is the number of mobile users, K is the number of content types, B is the number of base stations, is the number of requests from user i for content k, s k is the size of the content k, is a Boolean variable, P ISP is the average profit generated by ISP per unit bandwidth, C re To represent the retrieval cost of each user request arriving at the cache, C cp The average processing cost per unit of network traffic paid for by CP.

2. The method for constructing a network profit optimization distribution mechanism based on edge caching according to claim 1, characterized in that: The specific steps of the ISP and CP joint content distribution mechanism system model include: building a content popularity model based on Zipf distribution; Construct a network model; the network model includes a network service and architecture model, and a network topology model.

3. The method for constructing an edge cache-based network profit optimization distribution mechanism according to claim 2, characterized in that: The content popularity model is constructed by assuming that the number of content types is K and the video content is numbered from 1 to k. In a given fixed time, the total number of network requests is R, and the distribution of content popularity with content number k is: Among them, Zipf's skewness coefficient α represents the popularity of content.

4. The method for constructing an edge cache-based network profit optimization distribution mechanism according to claim 1, characterized in that: Based on the constructed ISP and CP joint profit model, a reinforcement learning algorithm is designed to find the optimal path for content transmission, and the Q-learning algorithm is combined with the profit model to solve the problem.

5. The method for constructing an edge cache-based network profit optimization distribution mechanism according to claim 4, characterized in that: Specific implementation of the Q-Learning algorithm: Set the topology background, use the adjacency matrix to represent the adjacency relationship of network nodes, initialize the Q table, and then directly replace and update the values in the Q table according to the Q table update strategy; Set up a Q-learning algorithm. When a content request does not reach the target node, randomly select the next destination node and obtain the current reward value of this action based on the reward matrix. Calculate the new Q value based on this reward value and update the Q value of the corresponding position in the Q table. After repeated training, the final Q table is used as a strategic basis for content request routing, which can find the best action selection for a content request to reach the target node when it is located at a node position in any node.

Images