Service caching method for caching decision in edge environment
By combining federated learning and deep reinforcement learning, caching decisions in edge computing environments are optimized, solving the problems of service popularity time-varying and user preference, achieving efficient cache resource utilization and energy management, and improving service quality.
Patent Information
- Application Number
- CN202311214603.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-20
- Publication Date
- 2026-01-30
AI Technical Summary
Existing service caching methods in edge computing environments fail to fully consider the time-varying nature of overall service popularity and local user perception preferences, cannot guarantee personal information privacy, and fail to effectively reduce energy consumption impact.
A federated learning algorithm is used to obtain a service popularity prediction queue. A caching decision model is built by combining deep reinforcement learning. Through the collaboration of edge servers and the assistance of central cloud servers, the caching strategy is optimized to reduce latency and energy consumption.
It achieves accurate prediction of service popularity changes, improves cache hit rate, makes full use of edge server resources, reduces energy consumption, and improves service quality while protecting user privacy.
Smart Images
Figure CN121441998A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of service caching technology in edge computing, and particularly to a service caching method for caching decisions in an edge environment. Background Technology
[0002] In recent years, the explosive growth of mobile applications and the ever-increasing demand for low-latency, high-bandwidth services have placed enormous pressure on traditional cloud-centric network infrastructure. To address these challenges, edge computing has emerged as a promising paradigm, bringing computing and storage capabilities closer to end users, resulting in faster response times, reduced network congestion, and improved Quality of Service (QoS). Edge service caching is a crucial aspect of edge computing and plays a vital role in improving the performance of mobile applications. Edge service caching refers to strategically storing frequently accessed data and services on edge fog access point servers (FAPs) to reduce response times and alleviate the burden on central cloud servers. This approach enables faster access to content and services, especially for latency-sensitive applications such as augmented reality, video streaming, and real-time data processing. However, as edge computing continues to develop, some new difficulties and challenges have gradually emerged. Edge servers typically have limited computing and storage resources. Furthermore, due to the diversity and time-varying nature of services, static caching strategies often cannot adapt to the constantly changing overall service popularity and the specificities of local user-perceived preferences, leading to decreased cache hit rates and impacting user experience and service quality. Moreover, due to the limited storage space of edge servers, the local hit rate has an upper limit, and the collaboration between edge nodes becomes particularly important.
[0003] Existing service caching in edge computing environments still has many shortcomings: (1) Existing methods rarely consider the continuous changes in overall service popularity and the particularity of local user perception preferences. (2) Most existing methods make corresponding predictions and caching decisions by sharing user service request information, but this method cannot protect the privacy of users' personal information in service requests. (3) In the process of collaborative caching decision-making, existing methods often only consider the user's service quality from the perspective of minimizing the average latency of service processing, without considering the impact of energy consumption on network service providers.
[0004] Therefore, a service caching method for caching decisions in edge environments is needed. Summary of the Invention
[0005] To address the shortcomings of existing technologies, such as insufficient consideration of user-perceived preferences, inability to guarantee personal information privacy, and lack of consideration for the impact of energy consumption on network service providers, this invention provides a service caching method for caching decisions in edge environments. This method fully considers factors such as the limited computing and storage resources of edge servers, the time-varying nature of overall service popularity, and the specific characteristics of local user-perceived preferences, thereby reducing service processing latency and energy consumption. The specific technical solution is as follows:
[0006] A service caching method for caching decisions in an edge environment includes the following steps:
[0007] S1: Obtain the service status of the system model and the cache status of the edge server;
[0008] S2: Obtain the popularity prediction queue of edge servers based on federated learning algorithm;
[0009] S3: Calculate the latency and energy consumption of the local edge server execution, the latency and energy consumption of the local edge server execution through collaboration with nearby edge servers, and the latency and energy consumption of the local edge server execution through the central cloud server, respectively.
[0010] S4: Based on the delay and energy consumption of the three cases in step S3, respectively, the optimization index for reducing energy consumption of notifications to ensure the service quality of users is obtained, and then the overall optimization index value is obtained.
[0011] S5: Combining popularity prediction queues and reward functions, a deep neural network is constructed using the DRL algorithm for learning, thereby autonomously obtaining the cache decision with the minimum overall optimization index value;
[0012] S6: Edge servers update and replace cached services based on caching decisions.
[0013] Preferably, the cache status of the edge server in step S1 is as follows:
[0014] L n ={x n,1 ,x n,2 ,…,x n,l}
[0015] Where, x n,l This is a binary variable used to indicate whether the application service is cached on the edge server. middle;
[0016] Furthermore, cached application services cannot exceed the edge server. cache space S n Meanwhile, the total computing resources required cannot exceed those of the edge server. Computing resources C n The expression is as follows:
[0017]
[0018] in, This is the set of all service types.
[0019] Preferably, step S2 is as follows:
[0020] S201: Each edge server downloads global model parameters W from the central cloud server. t ;
[0021] S202: Upon receiving the global model parameter W t Then, each edge server combines the global model parameters W t Iterative training and updating of the local model The updated local model was then uploaded to the central cloud server;
[0022] S203: After receiving the local model uploaded by the edge server, the central cloud server will update the global model W. t+1 At the same time, extract the trained local model. The hidden features in the data are mapped using a sigmoid function to determine the relationship between the hidden features and the popularity labels of user service request preferences, thereby obtaining the predicted values of the popularity of all services on the local edge server.
[0023] S204: Sort the predicted service popularity values from largest to smallest, and extract the top c services as the local service popularity prediction queue.
[0024] Preferably, the updated global model W t+1 The specific expression is as follows:
[0025]
[0026] in Represents the edge server at time t Gradient step size, Represents the edge server at time t The number of service requests received.
[0027] Preferably, the local edge server Execution delay and energy consumption Specifically as follows:
[0028]
[0029]
[0030] Local edge server via nearby edge servers Delay in collaborative execution and energy consumption Specifically as follows:
[0031]
[0032]
[0033] Local edge server Delays executed through the central cloud server and energy consumption Specifically as follows:
[0034]
[0035]
[0036] Among them, L base and E base This represents the basic latency and basic energy consumption of the edge server in receiving service requests and delivering service execution results; E x,l Indicates server Energy consumption of cache service l; q x Represents the side server The set of tasks in the queue with the shortest waiting time; ζ l This represents the computational workload of application service l; f x Indicates server Computational ability; ∈ x Indicates server Energy consumption coefficient; d l P represents the bit size of the calculation result after processing service request l; x Representative server Transmission power consumption; r m,n The data transmission rate of servers m and n under Gaussian noise environment is expressed by the following formula:
[0037] Among them B m For bandwidth rate, |h m | 2 For channel gain, σ 2 This represents the variance of additive white Gaussian noise (AWGN).
[0038] Preferably, the optimization index in step S4 is as follows:
[0039]
[0040] Where η l and η E L represents the influencing factors of delay and energy consumption, respectively. min L max and E min E max These represent the minimum and maximum theoretical delay and energy consumption, respectively.
[0041] Preferably, the overall optimization index value is as follows:
[0042]
[0043] in These are binary variables, representing edge server x executing user service requests locally, edge server x executing user service requests with the assistance of a neighboring edge server y, and edge server x executing user service requests via CS, while also meeting constraints. Ensure that service requests are executed in only one manner.
[0044] Preferably, step S5 is as follows:
[0045] S501: Take the services currently cached by each edge server as the initial state, take how to update the service cache of the edge server in each time step as the action, and take the policy optimization target in this state as the reward.
[0046] S502: By constructing a deep neural network as an agent network θ, the Q-value and policy probability distribution of each action are output. The agent network is used to interact in the environment, collect state, action and reward data, and use the DRL algorithm to train the agent network.
[0047] S503: Maximize the cumulative reward and optimize the value function so that the agent network learns to choose the best action a(t) = argmax(Q(s(t), a; θ)) in each state, thereby obtaining a caching decision that minimizes the overall optimization index value V of the edge server in the current state.
[0048] A computer-readable storage medium includes a stored program, wherein, when the program is executed, a service caching method controls the device where the computer-readable storage medium resides to perform caching decisions in an edge environment as described above.
[0049] A processor for running a program, wherein the program, when running, executes a service caching method for caching decisions in an edge environment as described above.
[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0051] (1) Using federated learning algorithms, while protecting user privacy, we fully consider the time-varying nature of the overall service popularity and the particularity of local user perception preferences, and obtain the cache prediction priority queue of each edge server in real time.
[0052] (2) After obtaining a more accurate prediction queue of service popularity, the changes in the edge network at the next moment are accurately predicted, so as to make full use of the collaboration between edge servers and greatly alleviate the limited storage and computing resources of edge servers.
[0053] (3) The impact of latency on user service quality and energy consumption on edge servers have been fully considered.
[0054] (4) Using deep reinforcement learning, after the network model is trained, it can better adapt to the ever-changing network scenarios, effectively improve the hit rate of edge services, make full use of the limited storage and computing resources of edge servers, and reduce the required energy consumption, which can greatly reduce costs. Attached Figure Description
[0055] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.
[0056] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation
[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0058] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0059] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0060] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0061] Combination Figure 1 The embodiments of the present invention will be further described below.
[0062] This embodiment provides a service caching method for caching decisions in an edge environment, including the following steps:
[0063] S1: Obtain the service status of the system model and the cache status of the edge server;
[0064] Regarding the service status of the system model, consider a service system of an edge network, which includes a central cloud server (CS) and multiple edge servers. Multiple users and all service types Each edge server is equipped with storage, computing, and communication capabilities.
[0065] Regarding the caching status of edge servers, due to limited storage space and computing power, edge servers can only cache a subset of application services. These cached application services need to be updated and replaced according to specific strategies. Edge servers employ containerization technology (e.g., Docker) to store computing services. This technology can quickly adapt to changes in request patterns and supports rapid querying of current resource utilization. We use a binary variable x. n,l This indicates whether the application service is cached on the edge server. In the case of a binary variable x n,l A value of 1 indicates that service l is cached on the edge server. Therefore, edge servers The cache status is:
[0066] L n ={x n,1 ,x n,2 ,…,x n,l}
[0067] The cached application service cannot exceed the edge server. cache space Sn Meanwhile, the total computing resources required cannot exceed those of the edge server. Computing resources C n .
[0068]
[0069] S2: Obtain the popularity prediction queue of edge servers based on federated learning algorithm;
[0070] Based on the federated learning algorithm, each edge server first downloads the global model parameters W from the central cloud server. t The model parameters enable edge servers to extract hidden features of globally popular services, thereby obtaining the global service popularity at the current moment.
[0071] Upon receiving the global model parameter W t Then, each edge server combines the global model parameters W t Iterative training and updating of the local model The updated local model was then uploaded to the central cloud server. It combines hidden features of global service popularity with hidden features of local service-aware preferences with local edge servers, fully considering the universality of global popularity and the particularity of local popularity. Then, it extracts the trained local model. The hidden features are mapped using a sigmoid function to represent the relationship between the hidden features and the popularity labels of user service request preferences. This yields predicted popularity values for all services on the local edge server. The predicted popularity values are then sorted from largest to smallest, and the top c services are extracted as the local service popularity prediction queue. During training, we use binary cross-entropy to calculate the log loss for multi-class predictions. This loss function measures the true label π and the predicted label π. Misclassification between them:
[0072]
[0073] We use the mean squared error (MSE) as the loss function for estimation:
[0074]
[0075] After receiving the local model uploaded by the edge server, the central cloud server will update the global model W. t+1This ensures that the global model contains the hidden features of all local models at the current time, thus guaranteeing that the hidden features extracted by the global model are always those that can predict the service popularity at the current time, fully considering the time-varying nature of global popularity. To address the imbalance problem of each local model, different aggregation weights are assigned to the local models uploaded from different edge servers, employing a weighted federated aggregation method. The updated global model W is then... t+1 for:
[0076]
[0077] in Represents the edge server at time t Gradient step size, Represents the edge server at time t The number of service requests received.
[0078] S3: Calculate the latency and energy consumption of the local edge server execution, the latency and energy consumption of the local edge server execution through collaboration with nearby edge servers, and the latency and energy consumption of the local edge server execution through the central cloud server, respectively.
[0079] This section primarily considers a classic collaborative service caching mechanism, when the edge server... When a user's service request is received, the system first checks if the service is cached locally. If it is, the local edge server handles the request directly; otherwise, it requests the service from a neighboring edge server. If a request for assistance is made, and multiple edge servers have cached service l, the request will be directed to the edge server with the lowest transmission cost; otherwise, the service request will be delivered to the central cloud server (CS) for execution. Therefore, the corresponding delay... and energy consumption and They are represented as follows:
[0080] Local edge server Execution delay and energy consumption
[0081]
[0082]
[0083] Local edge server via nearby edge servers Delay in collaborative execution and energy consumption
[0084]
[0085]
[0086] Local edge server Delays executed through the central cloud server and energy consumption
[0087]
[0088]
[0089] Where L base and E base E represents the basic latency and basic energy consumption of the edge server in receiving service requests and delivering service execution results. x,l Indicates server Energy consumption of cache service l, q x Represents the side server The set of tasks in the queue with the shortest waiting time, ζ l f represents the computational workload of application service l. x Indicates server Computational power, ∈ x Indicates server The energy consumption coefficient, d l P represents the bit size of the calculation result after processing service request l. x Representative server Transmission power consumption, r m,n This represents the data transmission rate of servers m and n under Gaussian noise conditions.
[0090]
[0091] Among them B m For bandwidth rate, |h m | 2 For channel gain, σ 2 This represents the variance of additive white Gaussian noise (AWGN).
[0092] S4: Based on the latency and energy consumption of the three scenarios in step S3, optimization indicators for reducing energy consumption in notifications to ensure user service quality are derived, and then the overall optimization indicator value is obtained; specifically as follows:
[0093] For users, service quality often depends on service execution latency. Therefore, we use service latency to represent the service quality of service l. Where β l This represents the impact factor of service latency on user service quality. We set a service quality threshold. To ensure that the quality of service provided to users is not lower than their minimum expectations.
[0094] For network service providers, it is necessary to minimize energy consumption while ensuring service quality for users. Therefore, after fully considering the impact of latency and energy consumption, our optimization metric can be expressed as:
[0095]
[0096] Where η l and η E L represents the influencing factors of delay and energy consumption, respectively. min L max and E min E max These represent the minimum and maximum theoretical delay and energy consumption, respectively.
[0097] Therefore, our goal is to find a caching strategy that minimizes the overall optimization metric V:
[0098]
[0099] in These are binary variables, representing edge server x executing user service requests locally, edge server x executing user service requests with the assistance of a neighboring edge server y, and edge server x executing user service requests via CS, while also meeting constraints. Ensure that service requests are executed in only one manner.
[0100] S5: Combining popularity prediction queues and reward functions, a deep neural network is constructed using the DRL algorithm for learning, thereby autonomously obtaining the cache decision with the minimum overall optimization index value; specifically as follows:
[0101] After receiving the service popularity prediction queue for each edge server in the current time slot, the service popularity prediction queue and the cache state of the edge servers are used as inputs to the neural network. First, the services currently cached by each edge server are taken as the initial state, how to update the service cache of the edge servers in each time step is taken as the action space, and the policy optimization objective in this state is taken as the reward. A deep neural network θ is constructed as an agent network, outputting the Q-value and policy probability distribution for each action. The agent network interacts in the environment, collecting state, action, and reward data, and is trained using the DRL algorithm. By maximizing the cumulative reward and optimizing the value function, the agent network learns to select the optimal action a(t) = argmax(Q(s(t), a; θ)) in each state, thus obtaining the cache decision that minimizes the overall optimization index V of the edge servers in the current state.
[0102] S6: Edge servers update and replace cached services based on caching decisions.
[0103] In summary, this invention employs a federated learning algorithm, which, while protecting user privacy, fully considers the time-varying nature of overall service popularity and the specificities of local user perception preferences, and obtains the cache prediction priority queue for each edge server in real time. After obtaining a more accurate service popularity prediction queue, we can more accurately predict the changes in the edge network at the next moment, thereby fully utilizing the collaboration between edge servers and greatly alleviating the limited storage and computing resources of edge servers. Furthermore, this invention fully considers the impact of latency on user service quality and the impact of energy consumption on edge servers. In addition, this invention uses deep reinforcement learning, which, after network model training, can better adapt to constantly changing network scenarios, effectively improving the edge service hit rate. Based on our technology, we can fully utilize the limited storage and computing resources of edge servers and reduce energy consumption, significantly reducing costs and solving the problems raised in the background art.
[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.
Claims
1. A service cache method for cache decision in edge environment, characterized in that, The method comprises the following steps: S1: obtaining service states of a system model and cache states of edge servers; S2: obtaining a popularity prediction queue of the edge servers based on a federated learning algorithm; S3: respectively calculating delay and energy consumption of a local edge server, delay and energy consumption of the local edge server in cooperation with a nearby edge server, and delay and energy consumption of the local edge server in cooperation with a central cloud server; S4: based on the delay and energy consumption of the three cases in step S3, obtaining an optimization index for reducing energy consumption for notifying to ensure service quality of users, and then obtaining a total optimization index value; S5: combining the popularity prediction queue and a reward function, constructing a deep neural network through a DRL algorithm to learn, so as to autonomously obtain a cache decision with the minimum total optimization index value; S6: the edge server updates and replaces the cached services according to the cache decision.
2. The service caching method for cache decision in edge environment according to claim 1, characterized in that, The cache states of the edge servers in step S1 are as follows: L n = {x n,1 , x n,2 , …, x n，l} wherein x n,l is a binary variable indicating whether the application service / is cached in the edge server or not; Moreover, the cached application services cannot exceed the cache space S of the edge server n while the sum of the computing resources they require cannot exceed the computing resources C of the edge server n expressed by the following formula: wherein, is the set of all service types.
3. The service caching method for cache decision in edge environment according to claim 1, characterized in that, Step S2 is specifically as follows: S201: Each edge server downloads global model parameters W from the central cloud server t ; S202: after receiving the global model parameters W t After that, each edge server combines the global model parameters W t Iterative training updates the local model The updated local model is then uploaded to the central cloud server; S203: After the center cloud server receives the local model uploaded by the edge server, the global model W is updated t+1 Meanwhile, the hidden features in the trained local model are extracted The relationship between the hidden features and the popularity labels of the user service request is mapped by using a Sigmoid function, thereby obtaining the prediction value of the popularity of all services of the local edge server. S204: arrange the predicted values of service popularity from large to small, extract the first c services as the local service popularity prediction queue 4. The service caching method of caching decision in edge environment according to claim 3, characterized in that, the updated global model W t+1 The specific expression is as follows: wherein denotes the edge server at time t gradient step size, denotes the edge server at time t number of received service requests.
5. The service caching method for cache decision in edge environment according to claim 1, characterized in that, The local edge server Latency of execution And energy consumption In detail as follows: Local edge server Through a proximate edge server Latency of the collaborative execution And energy consumption In particular as follows: Local edge server Latency by central cloud server And energy consumption In detail as follows: wherein, L base and E base represent the basic delay and basic energy consumption of the edge server receiving the service request and delivering the service execution result; E x,l represents the computing capability of the server ; q x represents the queue task set with the shortest waiting time on the edge server ; ζ l represents the computing workload of the application service l; f x represents the computing capability of the server ; ∈ x represents the energy consumption coefficient of the server ; d l represents the bit size of the computing result after processing the service request l; P x represents the transmission energy consumption power of the server ; r m,n represents the transmission data rate of the server m and the server n in the Gaussian noise environment, and the calculation formula is as follows: where B m is the bandwidth rate, h m | 2 is the channel gain, σ 2 is the variance of the additive white Gaussian noise (AWGN).
6. The service caching method for cache decision in edge environment according to claim 1, characterized in that, The optimization index in step S4 is specifically as follows: where η l and η E represent the impact factors of delay and energy consumption, respectively, L min , L max and E min , E max represent the minimum and maximum values of theoretical delay and energy consumption, respectively.
7. The service caching method of caching decision in edge environment according to claim 6, characterized in that, The total optimization index value is specifically as follows: wherein is a binary variable, respectively, indicating that the edge server x executes the user's service request by local execution, the edge server x executes the user's service request with the assistance of a proximate edge server y, the edge server x executes the user's service request by CS, while ensuring that the service request is executed in only one way by the constraint which ensures that the service request is executed in only one way.
8. The service caching method for cache decision in edge environment according to claim 1, characterized in that, Step S5 is specifically as follows: S501: taking the services cached by the current edge servers as an initial state, taking how to update the service cache of the edge server in each time step as an action, and taking a policy optimization target in the state as a reward value; S502: constructing a deep neural network as an agent network θ, outputting a Q value and a policy probability distribution of each action, interacting with the environment using the agent network, collecting state, action and reward data, and training the agent network using a DRL algorithm; S503: maximizing the cumulative reward and optimizing the value function, so that the agent network learns to select the best action a(t)=argmax(Q(s(t),a;θ) in each state, thereby obtaining a cache decision with the minimum total optimization index value V of the edge server in the current state.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein when the program runs, the device where the computer-readable storage medium is located executes the service cache method for cache decision in an edge environment according to any one of claims 1 to 8.
10. A processor, comprising: The processor is used to run a program, wherein when the program runs, the service cache method for cache decision in an edge environment according to any one of claims 1 to 8 is executed.