A model caching method for a dynamic edge environment

Abstract

This invention provides a model caching method for dynamic edge environments. The method includes: step S1, obtaining parameter block information cached by each base station in the previous time step and the popularity of each parameter block; step S2, based on the parameter block information cached by each base station in the previous time step and the popularity of each parameter block, generating a caching policy for the current time step using a pre-trained policy network, so that each base station can obtain and update the parameter blocks it needs to cache from the cloud according to the caching policy for the current time step; wherein, the parameter block information cached by each base station in the previous time step and the popularity of each parameter block are used as the observation space, the parameter block information that each base station needs to cache at the current time step is used as the action space, and the switching cost and latency cost constructed under the influence of user mobile behavior and social behavior are used as the optimization objectives, and a preset reinforcement learning algorithm is used to obtain the policy network.

Description

Technical Field

[0001] This invention relates to the field of mobile communications, specifically to edge intelligent caching technology in the field of mobile communications, and more specifically, to a model caching method for dynamic edge environments. Background Technology

[0002] In recent years, the integration of Mobile Edge Computing (MEC) and Artificial Intelligence (AI) has further given rise to the emerging paradigm of "Mobile Edge Intelligence," which provides intelligent services at the edge of mobile communication networks to achieve intelligent service capabilities with low latency and high privacy protection.

[0003] Mobile edge intelligence mainly consists of four parts: edge training, edge inference, edge offloading, and edge caching. Edge training focuses on model updates and lightweight optimization; edge inference emphasizes completing model computation at the edge to improve service real-time performance; edge offloading improves resource utilization through reasonable cloud-edge-device resource allocation; and edge caching stores model parameters and intermediate results on nodes closer to the user, enabling model-to-user service. Among these, model caching, as a key component of edge caching, supports the rapid response of both edge inference and edge offloading, directly impacting the response latency and resource utilization of AI services.

[0004] Model caching can be seen as an extension of Content Delivery Networks (CDNs) in the realm of intelligence. Unlike traditional edge caching paradigms that cache static content, model caching caches models. Nevertheless, model caching still inherits the core mechanism of content caching, namely, improving data delivery efficiency by storing the data needed by users near the edge.

[0005] Currently, traditional content caching methods typically use content popularity and access frequency as the main decision-making criteria, employing caching strategies such as Least Recently Used (LRU), Least Frequently Used (LRU), or First-In-First-Out (FIFO) to improve cache hit rates. These methods are highly effective for static content scenarios such as videos, but they have limitations in model caching. Because they do not fully consider the inherent properties of network models, such as their internal structure and parameter sharing, the final model caching decisions are often redundant, resulting in a waste of edge-side storage resources.

[0006] While some studies consider network model properties, their designs are often based on idealized assumptions, neglecting the dynamic complexity of edge environments, resulting in poor adaptability to real-world scenarios. For example, some methods utilize the parameter-sharing property of network models to cache multi-base station collaborative network models on a parameter block basis, achieving network model reuse. However, they fail to fully consider the dynamic characteristics of edge environments, such as the impact of user preferences, user mobility, and user social interaction on system performance, making such methods unsuitable for real-world scenarios. Some methods combine the sparse structure of network models, proposing an optimization strategy for Mixture of Experts (MoE) models. This strategy caches more popular expert modules based on expert activation probabilities to improve resource utilization. However, this method only considers static users; once users move within the scene, its performance may fluctuate significantly.

[0007] Furthermore, existing model caching methods typically use a single objective as the system optimization goal, ignoring the inherent trade-offs between multiple objectives, thus limiting the overall performance of model caching methods in real-world environments. For example, existing research often focuses on user experience quality, emphasizing the optimization of metrics such as user experience quality and latency, while neglecting the impact of system operating costs such as network model switching costs. This can lead to excessively high operating costs that are difficult to maintain, rendering the solution impractical. Simultaneously, this single-objective optimization method can cause uneven resource allocation, affecting the overall system efficiency.

[0008] In summary, current methods still have shortcomings in terms of model attribute compatibility, real-world scenario adaptation, and multi-objective collaborative optimization. Therefore, it is necessary to explore a new approach to cache optimization that comprehensively considers network model attributes, dynamic edge environment characteristics, and multi-dimensional system objectives, in order to achieve the optimal balance between service performance and system cost in practical applications of model caching.

[0009] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0010] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a model caching method for dynamic edge environments.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] According to a first aspect of the present invention, a model caching method for dynamic edge environments is provided, used to cache parameter blocks according to a generated caching strategy. The dynamic edge environment includes a cloud, multiple base stations, and multiple users, wherein: the cloud is used to store a set of parameter blocks, the set of parameter blocks including multiple backbone network parameter blocks and multiple task header parameter blocks, and a task header parameter block and a backbone network parameter block constitute a network model; the base stations are used to obtain and cache some parameter blocks from the cloud, and to respond to network model requests from different users at different times; the method generates a caching strategy for each time moment to guide the base station to cache parameter blocks in the following manner: Step S1, obtain the parameter block information cached by each base station in the previous time moment, and the popularity of each parameter block, wherein the parameter block The popularity reflects the probability that a parameter block is requested by a user; Step S2: Based on the parameter block information cached by each base station in the previous time step and the popularity of each parameter block, a pre-trained policy network is used to generate a caching policy for the current time step. The caching policy indicates the parameter block information that each base station needs to cache at the current time step, so that each base station can obtain and update the parameter blocks that it needs to cache from the cloud according to the caching policy at the current time step; wherein, the parameter block information cached by each base station in the previous time step and the popularity of each parameter block are used as the observation space, the parameter block information that each base station needs to cache at the current time step is used as the action space, and the handover cost and latency cost constructed under the influence of user mobile behavior and social behavior are used as the optimization objectives, and a preset reinforcement learning algorithm is used to obtain the policy network.

[0013] In some embodiments of the present invention, a pre-trained policy network is obtained as follows: A parameter block set, a model set, and a simulation time period of a dynamic edge environment are obtained. The model set includes network models for various task types, with each task type corresponding to multiple network models. The simulation time period includes multiple time slots. The set of neighboring base stations corresponding to each base station in the dynamic edge environment, the initial location information of each user, and the movement speed and direction of each user in each time slot are determined. The set of neighboring base stations includes one or more base stations, and the distance between all base stations in the set and their corresponding base stations is less than or equal to a preset threshold. Based on the initial location information of each user and the movement speed and direction of each user in each time slot, the location information of each user in each time slot is calculated according to a preset mobility policy. Based on the model set and the location information of each user in each time slot, the probability of each user requesting each network model in each time slot under the influence of user movement behavior and social behavior is calculated according to a preset probability calculation rule. The probability of each user requesting each network model in each time slot is then calculated based on the user's request in each time slot. Each network model is randomly selected by probability as the network model request for the user in the current time slot. In each time slot, each user selects the nearest base station as the local base station and sends a network model request to the local base station. A caching strategy for each time slot is generated based on each user's network model request in each time slot. Based on each user's network model request in each time slot, the set of neighboring base stations corresponding to each base station, and the caching strategy for each time slot, the transmission latency for each user to obtain the corresponding network model in each time slot and the latency cost within the simulated time period are calculated according to a preset latency calculation rule. Based on the caching strategy for each time slot, the handover cost within the simulated time period is calculated according to a preset cost calculation rule. The parameter block information cached by each base station at the previous time step and the popularity of each parameter block are used as the observation space, and the parameter block information that each base station needs to cache at the current time step is used as the action space. The handover cost and latency cost within the simulated time period constructed under the influence of user mobile behavior and social behavior are used as the optimization objective. A preset reinforcement learning algorithm is used to obtain the policy network.

[0014] In some embodiments of the present invention, the preset mobility strategy calculates the location information of each user in each time slot in the following manner:

[0015]

[0016] in, Indicates user In the time slot The x-coordinate position, Indicates user In the time slot The ordinate position, Indicates user In the time slot The x-coordinate position, Indicates the user in the time slot The ordinate position, Indicates user In the time slot movement speed, Indicates user In the time slot The direction of movement, Indicates the time slot length.

[0017] In some embodiments of the present invention, the preset probability calculation rule is as follows: the probability of each user requesting each network model in each time slot under the influence of user mobile behavior and social behavior is calculated as follows:

[0018]

[0019] in,

[0020]

[0021]

[0022] in, Indicates user In the time slot Request network model The probability, Indicates in time slot Active Request Network Model The probability, Indicates user In the time slot Based on social behavior passive request network model The probability, This represents the coefficient of social influence. Representation of network model In the time slot The popularity ranking of network models is based on the frequency of requests for each network model within a historical time window, sorted from highest to lowest. Indicates the distribution skewness parameter. This indicates the number of network models in the model set. Indicates user In the time slot A social set that contains users All users whose distance is less than or equal to a preset social distancing threshold, Indicates user Belongs to social collection , This indicates an indicator function. The indicator function returns a value of 1 if the condition within the parentheses is true, and a value of 0 if the condition within the parentheses is false. , Representation of network model The type of task it belongs to, Indicates user In the time slot The network model requested at that time. Representation of network model The type of task it belongs to, Representation of network model With network model When belonging to the same task type, the value is 1; network model With network model The value is 0 when they do not belong to the same task type. Representation and Network Model A collection of all network models belonging to the same task type. Representation and Network Model The number of all network models belonging to the same task type Represents social sets The number of users included Indicates user In the time slot There is social behavior. Indicates user In the time slot There was no social behavior.

[0023] In some embodiments of the present invention, the preset latency calculation rule is as follows: the transmission latency of each user acquiring the corresponding network model in each time slot and the latency cost within the simulated time period are calculated in the following manner:

[0024]

[0025]

[0026] in,

[0027]

[0028] in,

[0029]

[0030] in, Indicates user In the time slot Request the transmission latency of the network model. Indicates user In the time slot Request Parameters Block Transmission delay, Representing the composition network model Required parameter block, This represents the latency cost over the simulated time period. Indicates the duration of the simulated time period. Indicates the number of users. Indicates user Obtain parameter blocks from the corresponding local base station. Transmission delay, Represents parameter block In the time slot Base station cache, Indicates user Obtain parameter blocks from the set of neighboring base stations corresponding to its own local base station. Transmission delay, Represents parameter block In the time slot Not by base station cache, Represents parameter block In the time slot Base station cache, Indicates user Retrieve parameter blocks from the cloud Transmission delay, Represents a set of base stations. Represents parameter block Size, Indicates base station With users In the time slot Link speed, Indicates base station To base station Link speed, Indicates cloud to base station The link rate.

[0031] In some embodiments of the present invention, the preset cost calculation rule is to calculate the switching cost within the simulated time period in a certain manner:

[0032]

[0033] in, This represents the switching cost within the simulated time period. This represents the cache switching cost for a single parameter block. Indicates in time slot Number of times the parameter block is switched.

[0034] In some embodiments of the present invention, the optimization objective is:

[0035]

[0036] in, Indicates the optimization objective. Indicates base station Cache capacity limit, Represents a set of simulated time periods. This represents a set of parameter blocks.

[0037] Preferably, the preset reinforcement learning algorithm is the MAPPO algorithm.

[0038] Compared with the prior art, the advantages of the present invention are: (1) The parameter block-level caching method is adopted, so that the base station does not need to cache the complete network model, but only needs to cache the necessary parameter blocks. This caching design reduces the storage of duplicate parameter blocks, reduces the cache occupation of the model on the edge side, and reduces the amount of data transmission during the network model download process, thereby reducing user latency and improving the efficiency of AI services on the edge side; (2) By simulating user behavior through mobility modeling and user request modeling based on social relationships, the dynamic changes of user location and user preferences are reflected to obtain the correlation information of different user requests, so that the subsequent caching strategy can more accurately predict user needs, thereby better supporting multi-base station collaborative caching; (3) By combining switching costs and latency costs, while taking into account user experience quality and operating costs, a more reasonable and stable caching decision is achieved. Attached Figure Description

[0039] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0040] Figure 1 A schematic diagram illustrating the design composition of the model caching method according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the model caching method according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram illustrating an example of parameter block composition according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating the construction process of a policy network according to an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram illustrating an example of the impact of user social behavior according to an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram comparing latency costs and switching costs according to an embodiment of the present invention;

[0046] Figure 7 This is a schematic diagram comparing the average user cost in different time slots according to an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram comparing the average user cost under different cache capacities according to an embodiment of the present invention;

[0048] Figure 9 This is a schematic diagram comparing the average cost per user under different numbers of users according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0050] As mentioned in the background section, existing model caching methods still have shortcomings in terms of model attribute compatibility, adaptation to real-world scenarios, and multi-objective collaborative optimization, which restricts the actual performance and application effect of model caching in mobile edge intelligence scenarios.

[0051] To address the aforementioned issues, the inventors analyzed existing model caching methods and discovered that traditional content caching methods primarily target independent files such as videos and web pages. These methods can only cache content as a whole, which limits the caching efficiency of edge nodes under limited resources.

[0052] Furthermore, the inventors discovered that most existing caching methods are designed for static or specific scenarios. These studies typically assume that users are static and their preferences are relatively stable. However, in real-world mobile edge environments, users move frequently, and their preferences are influenced by both popularity and social relationships, resulting in highly dynamic user behavior. Existing parameter block caching strategies are often relatively fixed and struggle to adapt to environmental changes in a timely manner.

[0053] Furthermore, the inventors discovered that existing model caching methods typically optimize a single metric. However, experiments show that optimizing only a single metric, such as latency or hit rate, can lead to a significant deterioration in the performance of other key metrics. For example, when user latency is the sole objective, base stations frequently adjust the cache state of parameter blocks to reduce latency, resulting in a significant increase in handover costs and ultimately a decrease in overall system performance. This demonstrates a natural conflict between metrics reflecting user experience quality and system cost. In scenarios where user behavior changes dynamically, this conflict is amplified, making it difficult for optimization methods based on a single metric to maintain system stability.

[0054] Therefore, existing model caching methods suffer from three main problems: First, they fail to fully consider the inherent properties of the network model, such as its internal structure and parameter sharing, leading to redundancy in caching decisions and significant waste of edge storage space. Second, they are designed only for simplified scenarios, neglecting key influencing factors in edge caching, such as dynamic conditions like user social relationships and mobility, resulting in insufficient adaptability in real-world environments. Third, they focus only on optimizing a single metric, such as cache hit rate, making it difficult to achieve an effective balance between service quality and system cost.

[0055] Based on the above analysis, the inventors propose a model caching method applicable to mobile edge intelligence. This method leverages the shareable nature of network model parameters for block-level parameter caching in scenarios where user behavior is constantly changing. Simultaneously, with the optimization goals of reducing latency and switching costs, a multi-agent reinforcement learning approach is employed to generate caching decisions, thereby achieving a more stable and efficient model caching effect in complex dynamic environments. In short, this method includes... Figure 1 The three-part mechanism design is as follows: First, a parameter block-level caching design based on parameter sharing divides the network model into backbone network parameter blocks and task header parameter blocks to cache the network model. It is not necessary to cache the complete network model, but only the necessary parameter blocks. Second, a mobile user behavior modeling method based on social relationships obtains the user's social relationships based on the real-time location of the mobile user and combines the network model popularity to model the user's request behavior, so as to accurately predict the user's network model request. Third, a multi-objective joint optimization method based on multi-agent reinforcement learning is proposed. With the optimization objectives of reducing latency cost and handover cost, a multi-agent near-end policy optimization algorithm is introduced to generate caching decisions for each base station, thereby improving the overall performance.

[0056] In summary, such as Figure 2As shown, this invention proposes a model caching method for dynamic edge environments, used to cache parameter blocks according to a generated caching strategy. The dynamic edge environment includes a cloud, multiple base stations, and multiple users. The cloud stores a set of parameter blocks, including multiple backbone network parameter blocks and multiple task header parameter blocks. A task header parameter block and a backbone network parameter block together form a network model. Base stations retrieve and cache some parameter blocks from the cloud, and respond to network model requests from different users at different times. The method generates a caching strategy for each time step to guide the base station in caching parameter blocks as follows: Step S1: Obtain the parameter block information cached by each base station in the previous time step, and the popularity of each parameter block. Popularity reflects the probability that a parameter block is requested by a user; Step S2: Based on the parameter block information cached by each base station in the previous time step and the popularity of each parameter block, a pre-trained policy network is used to generate a caching policy for the current time step. The caching policy indicates the parameter block information that each base station needs to cache at the current time step, so that each base station can obtain and update the parameter blocks that it needs to cache from the cloud according to the caching policy at the current time step; wherein, the parameter block information cached by each base station in the previous time step step and the popularity of each parameter block are used as the observation space, the parameter block information that each base station needs to cache at the current time step is used as the action space, and the handover cost and latency cost constructed under the influence of user mobile behavior and social behavior are used as the optimization objectives, and a preset reinforcement learning algorithm is used to obtain the policy network.

[0057] Before explaining each step, let's first introduce the dynamic edge environment.

[0058] A dynamic edge environment includes a cloud, multiple base stations, and multiple users. Let the set of base stations in the dynamic edge environment be denoted as [missing information]. , The total number of base stations is represented by the user set. , This represents the total number of users.

[0059] In this invention, the cloud is used to store multiple network models. Considering the parameter sharing characteristic of network models, each network model is divided into several smaller parameter blocks and stored in the cloud. The set of network models in the cloud is denoted as […]. , The total number of network models is represented by , and the set of parameter blocks is denoted as . , Indicates the total number of parameter blocks, model The correspondence between it and its contained parameter blocks is denoted as a set. ,Right now Representation Model The required parameter block (set).

[0060] Based on the hierarchical structure of deep learning models, parameter block sets can be divided into two categories: backbone network parameter blocks and task head parameter blocks. The combination relationship between the two types of parameter blocks is as follows: Figure 3 As shown (a network model consists of a task header parameter block and a backbone network parameter block). The backbone network parameter block set is as follows: , , This represents the total number of backbone network parameter blocks. This part is used to construct the general architecture and core capabilities of the network model, mainly undertaking feature extraction and representation learning tasks, and possesses high versatility and cross-task reusability; the task header parameter block set is represented as... , This section is used for domain adaptation for different tasks. Each task header corresponds to a specific downstream task, and the set of downstream task types is denoted as... , This indicates the total number of supported task types, which include typical vertical applications such as medicine, law, and finance.

[0061] To establish the correspondence between network models and task types, this invention defines a mapping function. ,in, Representation of network model The corresponding downstream task type is Since each network model contains a task header parameter block, and the task header determines the task type to which the model belongs, therefore, a task can only be considered if and only if there exists a task header parameter block. Corresponding task type Sometimes, Therefore, the backbone network parameter block set and the task header parameter block set do not overlap, satisfying the condition that... and .

[0062] The base station is used to retrieve and cache partial parameter blocks from the cloud, as well as to respond to network model requests from different users at different times.

[0063] To better understand the present invention, each step will be described in detail below with reference to specific embodiments.

[0064] I. Step S1

[0065] In step S1, the parameter block information cached by each base station at the previous time step and the popularity of each parameter block are obtained. The popularity of a parameter block reflects the probability that the parameter block is requested by a user. The popularity of each parameter block is determined based on the number of times that parameter block was requested and the number of times other parameter blocks were requested within a historical time window. For example, at time t, parameter block... The popularity is calculated from the request records within a historical time window from time t-6 to time t-1 (the time window can be set to 5 time periods, and the popularity of each parameter block is determined by the request frequency of each parameter block within the past 5 time periods). Specifically, based on the number of times each parameter block was requested within the historical time window, the popularity rankings of each parameter block are obtained by sorting them in descending order, and the request probability (popularity) of each parameter block is obtained through a Zipf distribution based on the popularity ranking of each parameter block.

[0066] II. Step S2

[0067] In step S2, based on the parameter block information cached by each base station at the previous time step and the popularity of each parameter block, a pre-trained policy network is used to generate a caching policy for the current time step. The caching policy indicates the parameter block information that each base station needs to cache at the current time step, so that each base station can obtain and update the parameter blocks that it needs to cache from the cloud according to the caching policy at the current time step. The observation space is the parameter block information cached by each base station at the previous time step and the popularity of each parameter block, the action space is the parameter block information that each base station needs to cache at the current time step, and the handover cost and latency cost constructed under the influence of user mobile behavior and social behavior are used as optimization objectives. A preset reinforcement learning algorithm is used to obtain the policy network.

[0068] According to an embodiment of the present invention, a pre-trained policy network is obtained as follows: A parameter block set, a model set, and a simulation time period of a dynamic edge environment are obtained. The model set includes network models for various task types, with multiple network models corresponding to each task type. The simulation time period includes multiple time slots. The set of neighboring base stations corresponding to each base station in the dynamic edge environment, the initial location information of each user, and the movement speed and direction of each user in each time slot are determined. The set of neighboring base stations includes one or more base stations, and the distance between all base stations in the set and their corresponding base stations is less than or equal to a preset threshold. Based on the initial location information of each user and the movement speed and direction of each user in each time slot, the location information of each user in each time slot is calculated according to a preset mobility policy. Based on the model set and the location information of each user in each time slot, the probability of each user requesting each network model in each time slot under the influence of user movement behavior and social behavior is calculated according to a preset probability calculation rule. The probability of each user requesting each network model in each time slot is then calculated based on the user's request in each time slot. Each network model is randomly selected by probability as the network model request for the user in the current time slot. In each time slot, each user selects the nearest base station as the local base station and sends a network model request to the local base station. A caching strategy for each time slot is generated based on each user's network model request in each time slot. Based on each user's network model request in each time slot, the set of neighboring base stations corresponding to each base station, and the caching strategy for each time slot, the transmission latency for each user to obtain the corresponding network model in each time slot and the latency cost within the simulated time period are calculated according to a preset latency calculation rule. Based on the caching strategy for each time slot, the handover cost within the simulated time period is calculated according to a preset cost calculation rule. The parameter block information cached by each base station at the previous time step and the popularity of each parameter block are used as the observation space, and the parameter block information that each base station needs to cache at the current time step is used as the action space. The handover cost and latency cost within the simulated time period constructed under the influence of user mobile behavior and social behavior are used as the optimization objective. A preset reinforcement learning algorithm is used to obtain the policy network.

[0069] According to one embodiment of the present invention, the preset mobility policy calculates the location information of each user in each time slot as follows:

[0070]

[0071] in, Indicates user In the time slot The x-coordinate position, Indicates user In the time slot The ordinate position, Indicates user In the time slot The x-coordinate position, Indicates the user in the time slot The ordinate position, Indicates user In the time slot movement speed, Indicates user In the time slot The direction of movement, Indicates the time slot length.

[0072] According to one embodiment of the present invention, the preset probability calculation rule is as follows: the probability of each user requesting each network model in each time slot under the influence of user mobile behavior and social behavior is calculated as follows:

[0073]

[0074] in,

[0075]

[0076]

[0077] in, Indicates user In the time slot Request network model The probability, Indicates in time slot Active Request Network Model The probability, Indicates user In the time slot Based on social behavior passive request network model The probability, This represents the coefficient of social influence. Representation of network model In the time slot Popularity ranking Representation of network model In the time slot The popularity ranking of network models is based on the frequency of requests for each network model within a historical time window, sorted from highest to lowest. Indicates the distribution skewness parameter. This indicates the number of network models in the model set. Indicates user In the time slot A social set that contains users All users whose distance is less than or equal to a preset social distancing threshold, Indicates user Belongs to social collection , This indicates an indicator function. The indicator function returns a value of 1 if the condition within the parentheses is true, and a value of 0 if the condition within the parentheses is false. , Representation of network model The type of task it belongs to, Indicates user In the time slot The network model requested at that time. Representation of network model The type of task it belongs to, Representation of network model With network model When belonging to the same task type, the value is 1; network model With network model The value is 0 when they do not belong to the same task type. Representation and Network Model A collection of all network models belonging to the same task type. Representation and Network Model The number of all network models belonging to the same task type Represents social sets The number of users included Indicates user In the time slot There is social behavior. Indicates user In the time slot There was no social behavior.

[0078] According to one embodiment of the present invention, the preset latency calculation rule is to calculate the transmission latency of each user in each time slot for obtaining the corresponding network model and the latency cost within the simulated time period in the following manner:

[0079]

[0080]

[0081] in,

[0082]

[0083] in,

[0084]

[0085] in, Indicates user In the time slot Request the transmission latency of the network model. Indicates user In the time slot Request Parameters Block Transmission delay, Representing the composition network model Required parameter block, This represents the latency cost over the simulated time period. Indicates the duration of the simulated time period. Indicates the number of users. Indicates user Obtain parameter blocks from the corresponding local base station. Transmission delay, Represents parameter block In the time slot Base station cache, Indicates user Obtain parameter blocks from the set of neighboring base stations corresponding to its own local base station. Transmission delay, Represents parameter block In the time slot Not by base station cache, Represents parameter block In the time slot Base station cache, Indicates user Retrieve parameter blocks from the cloud Transmission delay, Represents a set of base stations. Represents parameter block Size, Indicates base station With users In the time slot Link speed, Indicates base station To base station Link speed, Indicates cloud to base station The link rate.

[0086] According to one embodiment of the present invention, the preset cost calculation rule is to calculate the switching cost within the simulated time period in a certain manner:

[0087]

[0088] in, This represents the switching cost within the simulated time period. This represents the cache switching cost for a single parameter block. Indicates in time slot Number of times the parameter block is switched.

[0089] According to an embodiment of the present invention, the optimization objective is:

[0090]

[0091] in, Indicates the optimization objective. Indicates base station Cache capacity limit, Represents a set of simulated time periods. This represents a set of parameter blocks.

[0092] According to one embodiment of the present invention, the preset reinforcement learning algorithm is the MAPPO algorithm.

[0093] Based on the foregoing embodiments, the construction process of the policy network can be roughly divided into the following steps: Figure 4 The three stages are as follows: the first stage is updating user location and user request information; the second stage is calculating switching cost and time delay cost; and the third stage is multi-agent deep reinforcement learning for multi-objective joint optimization.

[0094] To better understand the present invention, the construction process of the policy network will be described in detail below with reference to the foregoing embodiments.

[0095] 2.1 User Location and User Request Information Update

[0096] 2.1.1 User Location Update

[0097] In dynamic edge environments, the base station location remains constant, while the user's location changes as the user moves. To simulate continuous user movement behavior, movement speed and direction are generated for each user in each time slot. Here, user... In the time slot Taking the movement speed as an example, this speed is subject to a limit. The truncated Gaussian distribution within the range has the following density function:

[0098]

[0099] Among them, the mean To maintain the continuity of speed, Represents the standard normal density function. Represents the cumulative distribution function. The standard deviation of velocity, Indicates the maximum user speed. This indicates the minimum user speed.

[0100] User's movement direction in each time slot ( Then, an inertial perturbation model is used for updating.

[0101] Based on the user's movement speed and direction in each time slot, update the user's position in the next time slot as follows:

[0102] in, Indicates user In the time slot The x-coordinate position, Indicates user In the time slot The ordinate position, Indicates user In the time slot The x-coordinate position, Indicates the user in the time slot The ordinate position, Indicates user In the time slot movement speed, Indicates user In the time slot The direction of movement, Indicates the time slot length.

[0103] By generating movement speed and direction, the user's location coordinates can be continuously updated to obtain the user's location at different times. By measuring the distance between the user and each base station, the base station the user is connected to in each time slot can be determined, thus reflecting the access handover characteristics caused by user movement. At the same time, this process can effectively describe the dynamic changes commonly seen in mobile edge environments and provide necessary location information for caching strategies.

[0104] 2.1.2 User Request Information Update

[0105] In a dynamic edge environment, a user's network model requests in each time slot are divided into two parts: active requests and passive requests. The user's model request behavior in each time slot is determined as follows:

[0106]

[0107] in, Indicates user In the time slot Request network model The probability, Indicates in time slot Active Request Network Model The probability, Indicates user In the time slot Based on social behavior passive request network model The probability, This represents the coefficient of social influence. .

[0108] Active requests are generated based on the popularity distribution of the network model, representing choices made by users based on their own interests. Specifically, the user's active request behavior in each time slot is determined as follows:

[0109]

[0110] in, Representation of network model In the time slot Popularity ranking Representation of network model In the time slot The popularity ranking of network models is based on the frequency of requests for each network model within a historical time window, sorted from highest to lowest. Indicates the distribution skewness parameter. , This indicates the number of network models in the model set. To adapt to dynamic changes in popularity, the popularity ranking of network models is based on request statistics within a historical time window, with random perturbations added, thus maintaining stability while enhancing adaptability to dynamic changes in popularity.

[0111] Passive requests are influenced by a user's social relationships, meaning that the user's choices are influenced by their social connections. For example... Figure 5 As shown, when the distance between two users is less than or equal to the social distance threshold, they form a social relationship. A user's passive requests are determined by the type of task most recently encountered by their social user, giving the request behaviors between users a certain degree of correlation, thus more closely resembling the characteristics of social propagation in real-world scenarios. Specifically, the passive request behavior of a user in each time slot is determined as follows:

[0112]

[0113] Indicates user In the time slot Based on social behavior passive request network model The probability, Indicates user In the time slot A social set that contains users All users whose distance is less than or equal to a preset social distancing threshold, Indicates user Belongs to social collection , This indicates an indicator function. The indicator function returns a value of 1 if the condition within the parentheses is true, and a value of 0 if the condition within the parentheses is false. , Representation of network model The type of task it belongs to, Indicates user In the time slot The network model requested at that time. Representation of network model The type of task it belongs to, Representation of network model With network model When belonging to the same task type, the value is 1; network model With network model The value is 0 when they do not belong to the same task type. Representation and Network Model A collection of all network models belonging to the same task type. Representation and Network Model The number of all network models belonging to the same task type Represents social sets The number of users included Indicates user In the time slot There is social behavior. Indicates user In the time slot There was no social behavior.

[0114] Based on this, it can be seen that if there is social behavior between the user and other users in the previous time slot, the type of task requested by the user in the next time slot will be determined by the social influence, and the specific network model will be selected evenly under that task type. If there is no social behavior between the user and other users in the previous time slot, the network model request of the user in the next time slot will be determined by the popularity of the network model.

[0115] Based on the foregoing, by comprehensively considering the correlation between changes in user preferences and user behavior caused by social relationships, it is possible to simulate the network model selection of mobile users in real-world scenarios, which is beneficial for subsequent caching decision design.

[0116] 2.2 Calculation of Switching Costs and Time Delay Costs

[0117] To achieve stable and efficient model caching services in dynamic edge environments, this invention proposes to improve cache space utilization by using parameter block-level caching, with the optimization goal of reducing latency and switching costs.

[0118] To better understand this invention, the following details how to solve for latency cost and switching cost, and how to construct an optimization objective based on latency cost and switching cost.

[0119] Before calculating the latency cost, a communication rate model for the base station-user link needs to be established. In this invention, a free-space propagation model is used to describe the transmission rate of the base station-user link to reflect the link changes caused by user mobility. Specifically, the transmission rate between the user and the base station in each time slot is determined as follows:

[0120]

[0121]

[0122] in, Indicates user With base station In the time slot transmission rate Indicates bandwidth. Indicates base station The transmission power, Indicates user With base station In the time slot Channel gain, Represents the noise power spectral density. Indicates reference distance Channel gain at that location, Indicates user With base station The distance.

[0123] Once the communication rate model for the base station-to-user link is established, calculations can be performed based on the method of parameter block acquisition. These acquisition methods include three categories: local acquisition, collaborative acquisition, and cloud acquisition. Local acquisition means the user obtains parameter blocks from the nearest base station (local base station); collaborative acquisition means the user obtains parameter blocks from neighboring base stations of the local base station; and cloud acquisition means the user obtains parameter blocks from the cloud. Different acquisition methods correspond to different transmission delays.

[0124] The local acquisition transmission delay is calculated as follows:

[0125]

[0126] in, Indicates user Obtain parameter blocks from the corresponding local base station. Transmission delay, Indicates base station With users In the time slot Link speed, Represents parameter block Size.

[0127] The corresponding transmission delay for collaborative acquisition is calculated as follows:

[0128]

[0129] in, Indicates user Obtain parameter blocks from the set of neighboring base stations corresponding to its own local base station. Transmission delay, Indicates base station To base station The link rate.

[0130] The corresponding transmission latency obtained from the cloud is calculated as follows:

[0131]

[0132] in, Indicates user Retrieve parameter blocks from the cloud Transmission delay, Indicates cloud to base station The link rate.

[0133] Based on the transmission latency corresponding to different acquisition methods and the buffer status of parameter blocks at the base station, the transmission latency for users to access different parameter blocks in each time slot is calculated as follows:

[0134]

[0135] in, Indicates user In the time slot Request Parameters Block Transmission delay, Indicates user Obtain parameter blocks from the corresponding local base station. Transmission delay, Represents parameter block In the time slot Base station cache, Indicates user Obtain parameter blocks from the set of neighboring base stations corresponding to its own local base station. Transmission delay, Represents parameter block In the time slot Not by base station cache, Represents parameter block In the time slot Base station (base station) (Neighboring base stations) cache, Indicates user Retrieve parameter blocks from the cloud Transmission delay, This represents a set of base stations.

[0136] Based on the transmission latency of a user accessing different parameter blocks in each time slot, the transmission latency of a user accessing different network models in each time slot is calculated as follows:

[0137]

[0138] in, Indicates user In the time slot Request the transmission latency of the network model. Indicates user In the time slot Request Parameters Block Transmission delay, Representing the composition network model Required parameter block, Indicates user In the time slot Request network model .

[0139] Based on the transmission latency of users accessing different network models in each time slot, the latency cost is calculated as follows:

[0140]

[0141] in, This represents the latency cost within the simulated time period, and is used to represent the average latency cost for all users within the simulated time range. Indicates the duration of the simulated time period. Indicates the number of users. Indicates user In the time slot Request the transmission latency of the network model.

[0142] Based on the user's network model request in each time slot, the handover cost is calculated as follows:

[0143]

[0144] in, This represents the switching cost within the simulation period, and is used to represent the average parameter block switching cost over the entire simulation period. This represents the cache switching cost for a single parameter block. Indicates in time slot Number of times the parameter block is switched.

[0145] Based on latency cost and switching cost, construct the optimization objective:

[0146]

[0147] in, Indicates the optimization objective. Indicates base station Cache capacity limit, Represents a set of simulated time periods. Represents a set of parameter blocks. This means that the cache capacity of each base station in each time slot does not exceed its own storage limit. The table cache decision variable is a binary variable. Indicates base station In the time slot Uncached parameter block , Indicates base station In the time slot Cache parameter block .

[0148] 2.3 Multi-agent Deep Reinforcement Learning for Multi-Objective Joint Optimization

[0149] The optimization objective is essentially a high-dimensional, time-varying, and strongly coupled joint optimization task, and traditional methods have significant limitations in terms of algorithmic performance and efficiency. Therefore, this invention employs the Multi-Agent Proximal Policy Optimization (MAPPO) framework to solve this problem. The above problem can be abstracted as a Markov process, and the quintuple is defined as... The specific definition is as follows:

[0150] Observation space : Includes cache configuration of the previous time slot Popularity of parameter blocks .

[0151] State space: The aggregated state of each base station, obtained by stitching together local observations from each base station, i.e. .

[0152] Action space :action , representing the caching decisions of each base station, where, Represents a binary cache vector .

[0153] reward function Global instant reward is defined as: ,in, Indicates user In the time slot Request the transmission latency of the network model. This represents the cache switching cost for a single parameter block. Indicates in time slot Number of times the parameter block is switched.

[0154] State transition probability It implicitly reflects the dynamic environment brought about by user movement, changes in request patterns, and cache updates. Indicates the time slot of each base station state, Indicates the time slot of each base station The state.

[0155] In this algorithm framework, each base station is treated as an independent agent, generating cache actions based on its local cache state and model popularity information. Multiple agents are trained by sharing the same global reward, thereby achieving collaborative optimization among multiple base stations to obtain the policy network. It should be noted that during the training phase, to improve robustness under different edge environments, training is performed in rotation under various base station cache capacity conditions to enhance the generalization ability of the policy network under heterogeneous resource constraints. Furthermore, to address the problem of high dimensionality and difficulty in directly solving the discrete cache action space, a continuous action space is used to output the action probability distribution, and the final cache decision is determined through sampling. Further, to avoid getting trapped in local optima during training, a σ-greedy exploration mechanism is introduced in the action selection phase, randomly selecting non-optimal actions within a certain probability range to improve the globality and stability of the policy search.

[0156] Based on the foregoing, the model caching method proposed in this invention for dynamic edge environments aims to reduce latency and handover costs. It improves cache space utilization through parameter block-level caching, accurately reflects user behavior characteristics through user mobility and social relationship modeling, and generates caching strategies using a multi-agent near-end strategy optimization algorithm. This enables stable and efficient model caching services in dynamic multi-base station scenarios.

[0157] To verify the superiority of the method of this invention, experiments were conducted to verify its performance under different conditions. In the experiments, the average user cost (the sum of latency cost and switching cost) was used as the evaluation metric. Several existing typical caching algorithms were selected as comparison schemes, including Independent Proximal Policy Optimization (IPPO), LRU, LFU, FIFO, and Random algorithms. Performance tests were performed under the same network environment, and the results are as follows: Figures 6-9 The experimental results are shown.

[0158] Experimental results show that, under different time slots, cache capacities, and number of users, the method of this invention achieves better experimental results in terms of average user cost (the sum of latency cost and switching cost).

[0159] Among them, Figure 6 This indicates the cost breakdown of each algorithm when the cache capacity is 1200MB. The results show that the method of this invention performs best in terms of average user cost, achieving a better balance between latency and switching costs. In contrast, while some algorithms may excel in a single cost aspect, their overall performance is limited due to excessively high switching or latency costs. This result demonstrates the advantage of the method of this invention in multi-objective trade-offs.

[0160] Figure 7 The paper demonstrates the changes in average user cost for each algorithm across different time slots. Specifically, as the time slot increases, the average user cost for each algorithm decreases rapidly in the initial stage and then stabilizes, indicating that they can gradually learn user request patterns and form effective caching strategies. Among them, the method of this invention maintains the lowest average user cost throughout the entire process, and its average user cost is significantly lower than that of other algorithms in the stable phase, demonstrating its advantage in collaborative decision-making under dynamic user behavior.

[0161] Figure 8 The figure illustrates the changes in average user cost for each algorithm under different cache capacities. As the cache capacity increases, the average user cost for each algorithm generally shows a downward trend, but the rate of decrease and stability vary. The method of this invention maintains the lowest cost throughout the entire capacity range, demonstrating good stability and robustness, and reflecting its ability to effectively learn global request patterns under different cache resource conditions.

[0162] Figure 9The paper compares the average user cost of each algorithm under different numbers of users. It can be seen that the average user cost of all algorithms decreases overall as the number of users increases. This is because the total switching cost is spread across more users, reducing the average cost borne by each individual user. Compared with other algorithms, the method of this invention consistently maintains the lowest and most stable average user cost under different numbers of users, demonstrating its excellent scalability.

[0163] Combination Figures 6-9 As can be seen, the method of the present invention exhibits stable performance improvement under different parameters, verifying its applicability and effectiveness in dynamic edge environments.

[0164] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By adopting a parameter block-level caching method, the base station does not need to cache the complete network model, but only needs to cache the necessary parameter blocks. This caching design reduces the storage of duplicate parameter blocks, reduces the cache occupation of the model on the edge side, and reduces the amount of data transmission during the network model download process, thereby reducing user latency and improving the efficiency of AI services on the edge side; (2) By simulating user behavior through mobility modeling and user request modeling based on social relationships, the dynamic changes of user location and user preferences are reflected to obtain the correlation information of different user requests, so that the subsequent caching strategy can more accurately predict user needs, thereby better supporting multi-base station collaborative caching; (3) By combining switching costs and latency costs, while taking into account user experience quality and operating costs, a more reasonable and stable caching decision is achieved.

[0165] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0166] This invention can be a system, method, electronic device, computing device, computer-readable medium, and / or computer program product. A computer program product primarily refers to a software product that implements this solution through a computer program.

[0167] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0168] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A model caching method for dynamic edge environments, used to cache parameter blocks according to a generated caching strategy, wherein the dynamic edge environment includes a cloud, multiple base stations, and multiple users, wherein: The cloud is used to store parameter block sets, which include multiple backbone network parameter blocks and multiple task header parameter blocks. A task header parameter block and a backbone network parameter block together form a network model. The base station is used to retrieve and cache partial parameter blocks from the cloud, as well as to respond to network model requests from different users at different times; The method involves generating a caching strategy for each time step to guide the base station's cache parameter blocks as follows: Step S1: Obtain the parameter block information cached by each base station in the previous time step, and the popularity of each parameter block, wherein the popularity of the parameter block reflects the probability that the parameter block is requested by the user. Step S2: Based on the parameter block information cached by each base station in the previous time step and the popularity of each parameter block, a pre-trained policy network is used to generate a caching policy for the current time step. The caching policy indicates the parameter block information that each base station needs to cache at the current time step, so that each base station can obtain and update the parameter blocks that it needs to cache from the cloud according to the caching policy at the current time step. The observation space consists of the parameter block information cached by each base station at the previous time step and the popularity of each parameter block. The action space consists of the parameter block information that each base station needs to cache at the current time step. The switching cost and latency cost constructed under the influence of user mobile behavior and social behavior are used as optimization objectives. A preset reinforcement learning algorithm is used to obtain the policy network.

2. The method of claim 1, wherein, Obtain the pre-trained policy network as follows: Obtain the parameter block set, model set, and simulation time period of the dynamic edge environment. The model set includes network models of various task types, with multiple network models corresponding to each task type. The simulation time period includes multiple time slots. Determine the set of neighboring base stations corresponding to each base station in the dynamic edge environment, the initial location information of each user, and the movement speed and direction of each user in each time slot. The set of neighboring base stations includes one or more base stations, and the distance between all base stations in the set and the corresponding base station is less than or equal to a preset threshold. Based on each user's initial location information and each user's movement speed and direction in each time slot, the location information of each user in each time slot is calculated according to the preset mobility strategy. Based on the model set and the location information of each user in each time slot, the probability of each user requesting each network model in each time slot is calculated according to the preset probability calculation rules under the influence of user mobile behavior and social behavior. Based on the probability of each user requesting each network model in each time slot, a network model is randomly selected as the network model request for that user in the current time slot. In each time slot, each user selects the base station closest to itself as the local base station and sends the network model request to the local base station. Based on each user's network model request in each time slot, a caching strategy for each time slot is generated. Based on each user's network model request in each time slot, the set of neighboring base stations corresponding to each base station, and the caching strategy for each time slot, the transmission latency for each user to obtain the corresponding network model in each time slot and the latency cost within the simulated time period are calculated according to the preset latency calculation rules. Based on the caching strategy for each time slot, the switching cost within the simulated time period is calculated according to the preset cost calculation rules; The observation space is the parameter block information cached by each base station at the previous time step and the popularity of each parameter block. The action space is the parameter block information that each base station needs to cache at the current time step. The optimization objective is the handover cost and latency cost within the simulated time step constructed under the influence of user mobile behavior and social behavior. A preset reinforcement learning algorithm is used to obtain the policy network.

3. The method of claim 2, wherein, The preset mobility strategy calculates the location information of each user in each time slot as follows: in, Indicates user In the time slot The x-coordinate position, Indicates user In the time slot The ordinate position, Indicates user In the time slot The x-coordinate position, Indicates the user in the time slot The ordinate position, Indicates user In the time slot movement speed, Indicates user In the time slot The direction of movement, Indicates the time slot length.

4. The method according to claim 2, characterized in that, The preset probability calculation rule is as follows: the probability of each user requesting each network model in each time slot under the influence of user mobile behavior and social behavior is calculated as follows: in, in, Indicates user In the time slot Request network model The probability, Indicates in time slot Active Request Network Model The probability, Indicates user In the time slot Based on social behavior passive request network model The probability, Indicates the strength coefficient of social influence. Representation of network model In the time slot The popularity ranking of network models is based on the frequency of requests for each network model within a historical time window, sorted from highest to lowest. Indicates the distribution skewness parameter. This indicates the number of network models in the model set. Indicates user In the time slot A social set that contains users All users whose distance is less than or equal to a preset social distancing threshold, Indicates user Belongs to social collection , This indicates an indicator function. The indicator function returns a value of 1 if the condition within the parentheses is true, and a value of 0 if the condition within the parentheses is false. , Representation of network model The type of task it belongs to, Indicates user In the time slot The network model requested at that time. Representation of network model The type of task it belongs to, Representation of network model With network model When belonging to the same task type, the value is 1; network model With network model The value is 0 when they do not belong to the same task type. Representation and Network Model A collection of all network models belonging to the same task type. Representation and Network Model The number of all network models belonging to the same task type Represents social sets The number of users included Indicates user In the time slot There is social behavior. Indicates user In the time slot There was no social behavior.

5. The method according to claim 2, characterized in that, The preset latency calculation rule is as follows: the transmission latency of each user in each time slot for obtaining the corresponding network model and the latency cost within the simulated time period are calculated in the following manner: in, in, in, Indicates user In the time slot Request the transmission latency of the network model. Indicates user In the time slot Request Parameters Block Transmission delay, Representing the composition network model Required parameter block, This represents the latency cost over the simulated time period. Indicates the duration of the simulated time period. Indicates the number of users. Indicates user Obtain parameter blocks from the corresponding local base station. Transmission delay, Represents parameter block In the time slot Base station cache, Indicates user Obtain parameter blocks from the set of neighboring base stations corresponding to its own local base station. Transmission delay, Represents parameter block In the time slot Not by base station cache, Represents parameter block In the time slot Base station cache, Indicates user Retrieve parameter blocks from the cloud Transmission delay, Represents a set of base stations. Represents parameter block Size, Indicates base station With users In the time slot Link speed, Indicates base station To base station Link speed, Indicates cloud to base station The link rate.

6. The method according to claim 5, characterized in that, The preset cost calculation rule is to calculate the switching cost within the simulated time period according to the following method: in, This represents the switching cost within the simulated time period. This represents the cache switching cost for a single parameter block. Indicates in time slot The number of times the parameter block is switched.

7. The method according to claim 6, characterized in that, The optimization objective is: in, Indicates the optimization objective. Indicates base station Cache capacity limit, Represents a set of simulated time periods. This represents a set of parameter blocks.

8. The method according to claim 2, characterized in that, The preset reinforcement learning algorithm is the MAPPO algorithm.

9. A computer device comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1-8.

10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, It stores a computer program / instruction thereon, which is executed by a processor to implement the steps of the method according to any one of claims 1-8.

Images