Base station cache replacement method, device, equipment, medium and product

Abstract

The invention discloses a base station cache replacement method and device, equipment, a medium and a product, and relates to the technical field of edge computing, and the method comprises the steps: collecting the basic information of a base station in a cellular network and a cacheable high-popularity service; modeling a cache replacement problem into a Markov decision process, and defining a state space, an action space and a reward function; determining an optimal cache replacement strategy by adopting a Q-learning algorithm based on the state space, the action space and the reward function; on the basis of the basic information and the high-popularity service, according to the optimal cache replacement strategy, the proportion of different services in the cache of the base station is adjusted or cache service data in the base station is replaced, the cache utilization rate is increased, and the edge computing advantage is fully played.

Description

Technical Field

[0001] The present application relates to the field of edge computing technology, and in particular to a base station cache replacement method, device, equipment, medium and product. Background Art

[0002] With the rapid development of mobile communication networks, as users' demand for high-quality data services continues to grow, cellular networks are facing huge traffic load pressure; in order to alleviate this problem and improve network response speed and user experience, edge computing technology has gradually received widespread attention in the industry.

[0003] Edge computing deploys computing and storage resources at the edge of the network, namely at the base station system (BSs), so that data can be processed nearby, thereby reducing data transmission delays and traffic load on the core network.

[0004] In cellular networks, the cache space at base stations is limited. How to effectively manage and utilize these cache resources has become an urgent problem to be solved. Traditional cache replacement strategies, such as Least Recently Used (LRU) and Least Frequently Used (LFU), often decide which data should be replaced based on simple rules. They are unable to make intelligent decisions based on the actual state of the network and user needs, resulting in low cache utilization and difficulty in fully leveraging the advantages of edge computing. Summary of the invention

[0005] The purpose of this application is to provide a base station cache replacement method, device, equipment, medium and product to solve the problem of low cache utilization and difficulty in fully utilizing the advantages of edge computing.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a base station cache replacement method, comprising:

[0008] Collecting basic information of base stations in a cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold;

[0009] The cache replacement problem is modeled as a Markov decision process, and a state space, an action space, and a reward function are defined; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is to adjust the number of cache coding packets for each service in the base station; and the reward function is the change in transmission cost caused by the cache replacement operation calculated based on historical data requests and transmission conditions between base stations;

[0010] Based on the state space, the action space and the reward function, a Q-learning algorithm is used to determine an optimal cache replacement strategy;

[0011] Based on the basic information and the high-popularity service, and according to the optimal cache replacement strategy, the ratio of different services in the cache of the base station is adjusted or the cached service data in the base station is replaced.

[0012] In a second aspect, the present application provides a base station cache replacement device, including:

[0013] A collection module, used to collect basic information of base stations in the cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold;

[0014] A modeling module, for modeling the cache replacement problem as a Markov decision process, and defining a state space, an action space, and a reward function; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is for adjusting the number of cache coding packets for each service in the base station; and the reward function is a calculated change in transmission cost caused by a cache replacement operation based on historical data requests and transmission conditions between base stations;

[0015] An optimal cache replacement strategy determination module, used to determine an optimal cache replacement strategy using a Q-learning algorithm based on the state space, the action space and the reward function;

[0016] An adjustment and replacement module is used to adjust the proportion of different services in the cache of the base station or replace the cache service data in the base station based on the basic information and the high popularity service and according to the optimal cache replacement strategy.

[0017] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the base station cache replacement methods described above.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the base station cache replacement methods described above.

[0019] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the base station cache replacement methods described above.

[0020] According to the specific embodiments provided by this application, this application discloses the following technical effects: this application collects basic information of base stations in cellular networks and cacheable high-popularity services; models the cache replacement problem as a Markov decision process, and defines the state space, action space and reward function; and uses the Q-learning algorithm to determine the optimal cache replacement strategy, according to which the optimal cache replacement strategy is used to adjust the proportion of different services in the cache of the base station or replace the cache service data in the base station. This application models the actual cache replacement problem into a Markov decision process (Markov Decision Process, MDP) model, uses a reinforcement learning algorithm to find the optimal solution, finds the optimal cache replacement strategy based on the actual state of the base station, and makes full use of the high-speed cache space of the edge base station to store services with high popularity, reducing propagation delays and improving user satisfaction. This application makes intelligent decisions based on the actual state of the base station and user needs, improves the utilization of the cache, and can give full play to the advantages of edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is a flow chart of a base station cache replacement method in one embodiment of the present application;

[0023] Figure 2 for Figure 1 Detailed process flow diagram. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] like Figure 1 As shown, the base station cache replacement method provided in the embodiment of the present application is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or can be executed jointly by a terminal and a server. In the embodiment of the present application, the method includes the following steps 101 to 104.

[0027] Step 101: Collect basic information of base stations in a cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold.

[0028] Step 102: Model the cache replacement problem as a Markov decision process, and define a state space, an action space, and a reward function; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is for adjusting the number of cached coding packets for each service in the base station; and the reward function is a transmission cost change caused by a cache replacement operation calculated based on historical data requests and transmission conditions between base stations.

[0029] Step 103: Based on the state space, the action space and the reward function, a Q-learning algorithm is used to determine an optimal cache replacement strategy.

[0030] Step 104: Based on the basic information and the high-popularity service, according to the optimal cache replacement strategy, adjust the proportion of different services in the cache of the base station or replace the cached service data in the base station.

[0031] In an exemplary embodiment, step 104 specifically includes: when a user downloads service s, the current base station requests the coding packets of the service s from the cache of the remaining base stations; the remaining base stations respond to the request and send the coding packets of the service s stored in the cache of the remaining base stations to the current base station; if the storage space of the current base station is higher than the space threshold, according to the optimal cache replacement strategy, the proportion of different services in the cache of the current base station is adjusted to increase the coding packets of the service s; if the storage space of the current base station is lower than the space threshold, according to the optimal cache replacement strategy, the coding packets of the service s replace the services cached in the current base station.

[0032] Further, a cellular network with n base stations (BS) is defined, denoted as set N; there are L services with high popularity, which can be cached at the BS in the cellular network; when a user tries to download service s, the serving BS requests the packet of service s from the cache of other BSs; let c ij represents the transmission cost of the packet from BSi to BSj via the backhaul link; similar to peer to peer (P2P) networks, nearby BSs will respond to requests and send packets stored in their own caches; then the serving BS will adjust the proportion of different services in its cache, that is, the packets of service s increase; due to the limited storage space of the base station, the data of other services must be replaced to avoid overflow of the base station cache space.

[0033] In an exemplary embodiment, for each service s, each service s is divided into M s The base station encodes the services stored in the base station according to the linear random network coding of the original packets to determine the coded packets for each service s. For successful decoding, service s requires K(s) coded packets on average.

[0034] Furthermore, since random linear network coding increases the redundancy of data by generating coded packets, even if some coded packets are lost or damaged during network transmission, as long as the number of coded packets received reaches or exceeds the K(s) required for decoding, service s can still successfully decode and restore the original data. This redundancy significantly improves the robustness of data transmission and reduces service interruptions caused by packet loss or errors.

[0035] In an exemplary embodiment, each base station buffers transmission information h ij (s) and H ij (s); h ij (s) is the coded packet of service s sent from BSi to BSj; BSi is the i-th base station; BSj is the j-th base station; H ij (s) is the upper limit of the coded packets that BSj hopes to obtain from BSi; when BSi sends a data request to BSj and BSj sends a coded packet of service s to BSi, the transmission information h ij (s) and H ij (s) is automatically obtained by BSi and BSj from the corresponding request and transmission signaling.

[0036] Furthermore, in order to consider the cache information of other BSs, each BS saves the transmission information h ij (s) and H ij(s), which represent the packets of service s sent from BSi to BSj and the upper limit of the packets that BSj hopes to obtain from BSi; when BSi sends a data request to BSj and BSj sends a packet of service s to BSi, the parameter h ij (s) and H ij (s) can be automatically obtained by BSi and BSj from the relevant request and transmission signaling; these parameters are updated when the next packet transmission process occurs between BSi and BSj; the parameter h ij (s) and H ij (s) No additional control signaling is required for information exchange, but can be obtained only from regular signaling during data request and transmission between BSs, reducing the control signaling overhead in the network and improving network efficiency.

[0037] In an exemplary embodiment, the cache replacement problem is modeled as a Markov decision process, specifically including: defining an MDP model by a tuple {M, A, R(m, a)}; wherein MDP is a Markov decision process; M is a set of states consisting of all base stations, and the state is a service stored by the base station; A is a set of joint actions; R(m, a) is a reward function fed back to the base station when action a is executed in state m; and action a is a cache replacement action.

[0038] Furthermore, a cache replacement MDP framework is designed; define m i is the state of BS i in the set M; m i =(m i (s1), m i (s2), ..., m i(s L )) consists of the number of cached packets for each service; all states of the base stations constitute the state of the entire system; A is the set of joint actions, where a i represents the action of BS i; specifically, a i =(a i (s1), a i (s2), ..., a i (s L )) is the adjustment of the number of cached packets for each service in BS i; BSs select their actions from set A; R(m,a) is the reward function that determines the reward fed back to the BS when action a is performed in state m; the goal is to minimize the transmission cost, and the reward is the change in transmission cost caused by cache replacement.

[0039] In an exemplary embodiment, the modeling process of the MDP model of BSi at time t is as follows: let BSi sense the current cache and obtain the current state in, based on Let BSi choose the current action based on The current state Transition to a new state And get the rewards generated during the conversion process and will Feedback to all base stations; Based on the reward Use Q-learning to estimate the optimal expected discounted reward for each BSi and determine the optimal expected discounted reward The corresponding optimal strategy π * (m); where π * (m) = argmax a∈A Q * (m, a), γ is the discount factor that determines the impact of future rewards on current decisions, is the optimal Q value of BSi, m' i For state m i The state after taking action a, Q * (m,a) is the optimal state-action value function, that is, the optimal value of taking action a in state m.

[0040] Furthermore, we define the strategy π * (m) = {π * (m1),π * (m2),...,π * (m N )} as the optimal action profile for each BS, which is selected for each state to maximize the expected discounted reward V of the cellular system over infinite time * .

[0041] Furthermore, Q learning is used to find the corresponding V * The optimal strategy π * (m) without exact prior knowledge about the system; since BSs make their action decisions in a distributed and independent manner, it is necessary to obtain the optimal γ is the discount factor that determines the impact of future rewards on current decisions, 0≤γ<1.

[0042] To estimate A Q-value is defined for each state-action pair, where the optimal Q-value of BS i is defined as: Among them, m i is the current state of BS i, m' i is in state m i is the state to which it is transferred after taking action a, and is state m i to m' i The transition probability of It has the following relationship with the Q value: If the optimal Q-value for each state-action is known, the optimal policy can be given by π * (m) = argmax a∈A Q * (m, a) is determined; the Q-learning algorithm recursively finds Q using a simple update rule for each BS * (m,a); Where α is the learning rate, m i +a i is to perform action a at time t i From state m i state reached, and

[0043] In an exemplary embodiment, each BS determines its cache replacement action based on its local information in a distributed manner. ij (s) and H ij (s) Calculate R i .

[0044] The change in transmission cost is used as an action to evaluate the corresponding reward; the reward R i It consists of the following four parts: in, is the positive reward that BSi gets from the increased coding grouping of service s; is the positive reward that the remaining base stations get from increasing the coded packets of service s; assuming s represents the service whose data the BS decides to reduce, is the negative reward of BSi due to the reduction of the coding group of service s'; is the negative reward of BSi due to BSi reducing the storage of service s'; is the negative reward of the remaining base stations due to the reduction of the coded packets serving s'; is the negative reward obtained by the remaining base stations due to the reduction of the storage of service s' by base station BSi.

[0045] In an exemplary embodiment, the number of coded packets k determined by any base station is defined; the base station that sends the coded packets of service s is set as S; the elements in base station S are divided into three categories according to the transmission cost: 1) j∈S full represents the set of base stations requesting all coded packets cached in BSj, S full is the base station of all coding groups; 2) j∈S partrepresents the set of base stations that request the partially encoded packets cached in BSj, S part is the base station of the partial coding group; 3) j∈S none express.

[0046]

[0047] Among them, when j∈S none , When j∈S part and c ji is the transmission cost of the coded packet from BSi to BSj via the backhaul link; The request to BSi to update the storage content causes the change in the number of service coding packets transmitted by the remaining base stations to BSi in actual transmission; N is the total number of base stations.

[0048] When j∈S part and

[0049] When j∈S full And h ji (s)≠H ji (s),

[0050] When j∈S full And h ji (s)=H ji (s),

[0051]

[0052] Among them, when j∈S none , Reducing the storage of service s' for BSi results in a change in the number of coded packets for service s requested by the remaining base stations from BSi in actual transmission.

[0053] When j∈S part And H ji (s')-h ji (s') <k,

[0054] When j∈S part And H ji (s')-h ji (s')>k, When j∈S full ,

[0055] Among them, when m i (s)≥h ij (s), m i (s) is the number of coded packets storing service s in BSi; The storage of service s is increased for requesting BSi, resulting in a change in the number of coded packets of service s sent by BSi to the remaining base stations in actual transmission.

[0056] When h ij (s)-k≤m i (s)≤h ij (s),

[0057] When h ij (s)-k≥m i (s),

[0058] Calculate the negative rewards of service S of BSs other than the serving BS, In order to request BSi to reduce the storage of service s', the number of coded packets of service s' sent by BSi to the remaining base stations is changed in actual transmission.

[0059] Among them, when m i (s')-k>h ij (s'), m i (s') is the number of coded packets storing service s' in BSi; ij (s') is the number of coded packets sent by BSi to BSj for service s'.

[0060] When 0≤m i (s')-k≤h ij (s'),

[0061] When m i (s') <k,

[0062] Based on the same inventive concept, the embodiment of the present application also provides a station cache replacement device for implementing the station cache replacement method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more station cache replacement device embodiments provided below can refer to the limitations of the station cache replacement method above, and will not be repeated here.

[0063] In an exemplary embodiment, a base station cache replacement device includes:

[0064] The collection module is used to collect basic information of base stations in the cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold.

[0065] A modeling module is used to model the cache replacement problem as a Markov decision process and define a state space, an action space and a reward function; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is for adjusting the number of cache coding packets for each service in the base station; and the reward function is the change in transmission cost caused by the cache replacement operation calculated based on historical data requests and transmission conditions between base stations.

[0066] The optimal cache replacement strategy determination module is used to determine the optimal cache replacement strategy based on the state space, the action space and the reward function using a Q-learning algorithm.

[0067] An adjustment and replacement module is used to adjust the proportion of different services in the cache of the base station or replace the cache service data in the base station based on the basic information and the high popularity service and according to the optimal cache replacement strategy.

[0068] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store station cache replacement data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a station cache replacement method is implemented.

[0069] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0070] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0071] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0072] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0073] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0074] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A base station cache replacement method, characterized in that: The base station cache replacement method comprises: Collecting basic information of base stations in a cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold; The cache replacement problem is modeled as a Markov decision process, and a state space, an action space, and a reward function are defined; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is to adjust the number of cache coding packets for each service in the base station; and the reward function is the change in transmission cost caused by the cache replacement operation calculated based on historical data requests and transmission conditions between base stations; Based on the state space, the action space and the reward function, a Q-learning algorithm is used to determine an optimal cache replacement strategy; Based on the basic information and the high-popularity service, and according to the optimal cache replacement strategy, the ratio of different services in the cache of the base station is adjusted or the cached service data in the base station is replaced.

2. The base station cache replacement method according to claim 1, characterized in that: Based on the basic information and the high-popularity service, adjusting the proportion of different services in the cache of the base station or replacing the cached service data in the base station according to the optimal cache replacement strategy specifically includes: When a user downloads service s, the current base station requests the coding packet of the service s from the cache of the remaining base stations; the remaining base stations respond to the request and send the coding packet of the service s stored in the cache of the remaining base stations to the current base station; for each service s, each service s is divided into multiple original packets, and the service stored in the base station is encoded according to the linear random network coding of the original packets to determine the coding packet of each service s; If the storage space of the current base station is higher than the space threshold, adjusting the proportion of different services in the cache of the current base station according to the optimal cache replacement strategy, and increasing the coding grouping of the service s; If the storage space of the current base station is lower than the space threshold, the coded grouping of the service s replaces the service cached in the current base station according to the optimal cache replacement strategy.

3. The base station cache replacement method according to claim 2, characterized in that: Each base station has a cache of transmission information h ij (s) and H ij (s); h ij (s) is the coded packet of service s sent from BSi to BSj; BSi is the i-th base station; BSj is the jth base station; H ij (s) is the upper limit of the coded packets that BSj hopes to obtain from BSi; When BSi sends a data request to BSj and BSj sends a coded packet of service s to BSi, the transmission information h ij (s) and H ij (s) is automatically obtained by BSi and BSj from the corresponding request and transmission signaling.

4. The base station cache replacement method according to claim 3, characterized in that: The cache replacement problem is modeled as a Markov decision process, which includes: The MDP model is defined by the tuple {M, A, R(m, a)}; where MDP is a Markov decision process; M is the set of states consisting of all base stations, and the state is the service stored by the base station; A is the set of joint actions; R(m, a) is the reward function fed back to the base station when action a is executed in state m; action a is a cache replacement action; The modeling process of the MDP model of BSi at time t is: Let BSi sense the current cache and obtain the current status in, based on Let BSi choose the current action based on The current state Transition to a new state And get the rewards generated during the conversion process and will Feedback to all base stations; Based on the reward Use Q-learning to estimate the optimal expected discounted reward for each BSi and determine the optimal expected discounted reward The corresponding optimal strategy π * (m); where π * (m) = argmax a∈A Q * (m, a), γ is the discount factor that determines the impact of future rewards on current decisions, is the optimal Q value of BSi, m' i For state m i The state after taking action a, Q * (m,a) is the optimal state-action value function.

5. The base station cache replacement method according to claim 4, characterized in that: The reward function is in, is the positive reward that BSi gets from the increased coding grouping of service s; is the positive reward that the remaining base stations get from the added coded packets of service s; is the negative reward of BSi due to the reduction of the coding group of service s'; is the negative reward of BSi due to BSi reducing the storage of service s'; is the negative reward of the remaining base stations due to the reduction of the coded packets serving s'; is the negative reward obtained by the remaining base stations due to the reduction of the storage of service s' by base station BSi.

6. The base station cache replacement method according to claim 5, characterized in that: Define the number k of coding groups that any base station decides to add; Let the base station that sends the coded packet of service s be S; the elements in base station S are divided into three categories according to the transmission cost: The first type: j∈S full represents the set of base stations requesting all coded packets cached in BSj, S full Base stations for all coding groups; Category 2: j∈S part represents the set of base stations that request the partially encoded packets cached in BSj, S part A base station for partial coding grouping; The third category: j∈S none represents the set of base stations that do not request any coded packets cached in BSj, S none Base stations for any coding group; Among them, when j∈S none , When j∈S part And h ji (s) <k, c ji is the transmission cost of the coded packet from BSi to BSj via the backhaul link; The change in the number of service coding packets transmitted by the remaining base stations to BSi in the actual transmission caused by requesting BSi to update the storage content; N is the total number of base stations; When j∈S part And h ji (s)≥k, When j∈S full And h ji (s)≠H ji (s), When j∈S full And h ji (s)=H ji (s), Among them, when j∈S none , Reducing the storage of service s' for BSi results in a change in the number of coded packets that the remaining base stations request for service s from BSi in actual transmission; When j∈S part And H ji (s')-h ji (s') <k, When j∈S part And H ji (s')-h ji (s')>k, When j∈S full , Among them, when m i (s)≥h ij (s), m i (s) is the number of coded packets storing service s in BSi; In order to request BSi to increase the storage of service s, it causes a change in the number of coded packets of service s sent by BSi to the remaining base stations in actual transmission; When h ij (s)-k≤m i (s)≤h ij (s), When h ij (s)-k≥m i (s), To request BSi to reduce the storage of service s', resulting in a change in the number of coded packets of service s' sent by BSi to the remaining base stations in actual transmission; Among them, when m i (s')-k>h ij (s'), m i (s') is the number of coded packets storing service s' in BSi; ij (s') is the number of coded packets sent by BSi to BSj for service s'; When 0≤m i (s')-k≤h ij (s'), When m i (s') <k, 7. A base station cache replacement device, characterized in that: The base station cache replacement device comprises: A collection module, used to collect basic information of base stations in the cellular network and cacheable high-popularity services; the basic information includes location and storage capacity; the high-popularity services are services whose popularity is higher than a set threshold; A modeling module, for modeling the cache replacement problem as a Markov decision process, and defining a state space, an action space, and a reward function; wherein the state space includes the number of service coding packets currently cached by each base station; the action space is for adjusting the number of cache coding packets for each service in the base station; and the reward function is a calculated change in transmission cost caused by a cache replacement operation based on historical data requests and transmission conditions between base stations; An optimal cache replacement strategy determination module, used to determine an optimal cache replacement strategy using a Q-learning algorithm based on the state space, the action space and the reward function; An adjustment and replacement module is used to adjust the proportion of different services in the cache of the base station or replace the cache service data in the base station based on the basic information and the high popularity service and according to the optimal cache replacement strategy.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the base station cache replacement method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the base station cache replacement method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the base station cache replacement method according to any one of claims 1 to 6 is implemented.

Images