Cache optimization model training method, cooperative edge caching method and apparatus

Through the collaboration between edge nodes and user terminals in the federated learning system, the grid cache strategy is optimized, and the adaptability problems of grid data dynamics and diversity are solved, and the cache efficiency and response speed are improved.

CN119088553APending Publication Date: 2024-12-06GUANGZHOU POWER SUPPLY BUREAU GUANGDONG POWER GRID CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411136107.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing power grid caching strategy is difficult to make full use of the storage resources of edge devices, resulting in low cache efficiency and slow response speed, and is unable to effectively adapt to the dynamics and diversity of power grid data.

Method used

Using the cache optimization model training method, through the collaboration between edge nodes and user terminals in the federated learning system, each user terminal determines whether to participate in iteration based on local data and bandwidth, performs model parameter training and cache replacement strategy optimization, form global model parameters, and updates cache Optimize the model.

Benefits of technology

Improve cache efficiency and response speed, optimize training resource allocation, and enhance the cache hit rate and content acquisition delay reduction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088553A_ABST
    Figure CN119088553A_ABST
Patent Text Reader

Abstract

The invention provides a cache optimization model training method and device and a cooperative edge caching method and device, and the method comprises the steps: enabling each user side to determine the predicted income in the iteration according to the local data, determining the communication cost in the iteration according to the distributed bandwidth, and determining whether to participate in the iteration or not, the user side participating in the iteration trains the model parameters sent by the edge node according to the local state information of the user side and uploads the trained model parameters to the edge node, and the edge node aggregates the received model parameters uploaded by the user side to obtain global model parameters so as to update the model parameters of the pre-training model. Therefore, each user side can determine whether to participate in iteration or not according to the predicted income and the communication cost, so that the user side with relatively high contribution degree can be selected to participate in the training of the cache optimization model, the distribution of training resources is optimized, the model is helped to better predict user requirements, the cache efficiency is improved, and the time delay of content acquisition is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of smart grid technology, and in particular to a cache optimization model training method, a collaborative edge caching method and a device. Background Art

[0002] In the field of smart grid, with the widespread application of smart meters, sensors and renewable energy systems, the amount of related data has also shown explosive growth. The large amount of real-time data generated by these devices is essential for the monitoring, maintenance and optimization of the power grid. The application of edge computing technology in the power grid can significantly reduce data transmission latency and improve the response speed and operation efficiency of the power grid by processing and analyzing data at the edge nodes of the power grid (such as substations or distributed energy resources).

[0003] However, the storage resources of grid edge devices are also limited, and the dynamic and uncertain nature of grid data further increases the complexity of cache strategy design. This makes it difficult for existing grid cache strategies to fully utilize these resources to adapt to the dynamic and diverse nature of grid data, resulting in low cache efficiency and slow response speed. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defects that the existing power grid caching strategies in the prior art are difficult to fully utilize these resources to adapt to the dynamics and diversity of power grid data, resulting in low caching efficiency and slow response speed.

[0005] In a first aspect, the present application provides a cache optimization model training method, which is applied to a federated learning system, wherein the federated learning system includes an edge node and multiple user terminals; the method includes:

[0006] The edge node obtains a pre-trained model and sends model parameters of the pre-trained model to each user terminal;

[0007] Each user terminal determines the expected revenue in this iteration according to its local data, and determines the communication cost in this iteration according to its local data and the allocated bandwidth;

[0008] Each user terminal determines whether to participate in this iteration based on its expected revenue and communication cost;

[0009] Each target user terminal uses local state information as training data to train the received model parameters, and uploads the trained model parameters to the edge node; wherein the target user terminal is the user terminal determined to participate in this iteration;

[0010] The edge node aggregates the model parameters uploaded by each target user terminal to form the global model parameters of this iteration, and updates the model parameters of the pre-trained model with the global model parameters, and returns to the step of sending the model parameters of the pre-trained model to each user terminal to continue executing until the preset training conditions are met, and the trained pre-trained model is determined as the cache optimization model.

[0011] In one embodiment, the step of each target user terminal using local state information as training data to train the received model parameters includes:

[0012] Each target user terminal updates its local model according to the received model parameters;

[0013] Each target user terminal inputs its local state information into its local model to determine the cache replacement strategy of each target user terminal in this iteration;

[0014] Each target user terminal determines the reward function corresponding to this iteration according to its cache replacement strategy in this iteration;

[0015] Each target user terminal determines the loss function of each target user terminal in this iteration according to the local state information, cache replacement strategy and reward function of each target user terminal, and minimizes the loss function of each target user terminal in this iteration to update the local model of each target user terminal, and use the model parameters of the updated local model as the trained model parameters.

[0016] In one embodiment, each target user terminal determines the reward function corresponding to the current iteration according to its cache replacement strategy in the current iteration, including:

[0017] Each target user terminal sends its cache replacement strategy in this iteration to the edge node to execute each cache replacement strategy and determine the first delay and the second delay of each target user terminal; the first delay is the delay of obtaining content at the cloud node, and the second delay is the average delay required to obtain content;

[0018] Each target user terminal determines its corresponding reward function according to its first time delay and second time delay.

[0019] In one embodiment, the step of each user terminal determining whether to participate in the iteration according to its expected revenue and communication cost includes:

[0020] Each user terminal determines its utility coefficient in this iteration based on its expected revenue and communication cost;

[0021] For any one of the user terminals, when the utility coefficient of the user terminal is greater than the preset threshold corresponding to the user terminal, the user terminal participates in this iteration; when the utility coefficient of the user terminal is not greater than the preset threshold corresponding to the user terminal, the user terminal does not participate in this iteration.

[0022] In one embodiment, the steps of each user terminal determining an estimated revenue in this iteration according to its local data, and determining a communication cost in this iteration according to its local data and the allocated bandwidth include:

[0023] Each user terminal calculates the local data gradient based on its local data, and determines the expected benefit of each user terminal in this iteration in combination with the preset benefit factor;

[0024] Each user terminal determines the communication cost of each user terminal in this iteration according to the bandwidth allocated by the edge node, in combination with the local data gradient of each user terminal and a preset cost consumption coefficient.

[0025] In one embodiment, the step of aggregating the model parameters received and uploaded by each target user terminal by the edge node includes:

[0026] The edge node determines an average value of each model parameter according to the received model parameters uploaded by each target user terminal, and determines the average value as the global model parameter.

[0027] In a second aspect, the present application provides a collaborative edge caching method, which is applied to an edge node in a federated system; the federated system includes an edge node and multiple user terminals; the method includes:

[0028] receiving a content acquisition request initiated by the user terminal, and determining target content according to the content acquisition request;

[0029] When the target content does not exist in the cache space of the edge node and the cache space is insufficient, determining the state information according to the content acquisition request;

[0030] Obtaining a cache optimization model; the cache optimization model is obtained according to the cache optimization model training method described in any of the above embodiments;

[0031] Inputting the state information into the cache optimization model to determine a cache replacement strategy corresponding to the state information;

[0032] According to the cache replacement strategy, the target content is used to replace the corresponding content in the cache space of the edge node.

[0033] In a third aspect, the present application provides a cache optimization model training device, which is applied to a federated learning system, wherein the federated learning system includes an edge node and multiple user terminals; the device includes:

[0034] A parameter sending module, used for the edge node to obtain a pre-trained model and send the model parameters of the pre-trained model to each user terminal;

[0035] A data determination module, used for each user terminal to determine the expected revenue in this iteration according to its local data, and to determine the communication cost in this iteration according to its local data and the allocated bandwidth;

[0036] The participation determination module is used for each user terminal to determine whether to participate in this iteration based on its expected revenue and communication cost;

[0037] The model training module is used for each target user terminal to train the received model parameters using the local state information as training data, and upload the trained model parameters to the edge node; wherein the target user terminal is the user terminal determined to participate in this iteration;

[0038] The parameter updating module is used for the edge node to aggregate the model parameters uploaded by each target user terminal to form the global model parameters of this iteration, and update the model parameters of the pre-trained model with the global model parameters, and return to the step of sending the model parameters of the pre-trained model to each user terminal to continue executing until the preset training conditions are met, and the trained pre-trained model is determined as the cache optimization model.

[0039] In a fourth aspect, the present application provides a collaborative edge cache device, which is applied to an edge node in a federated system; the federated system includes an edge node and multiple user terminals; the device includes:

[0040] A request receiving module, used to receive a content acquisition request initiated by the user terminal, and determine target content according to the content acquisition request;

[0041] An information determination module, configured to determine status information according to the content acquisition request when the target content does not exist in the cache space of the edge node and the cache space is insufficient;

[0042] A model acquisition module, used to acquire a cache optimization model; the cache optimization model is obtained according to the cache optimization model training method described in any of the above embodiments;

[0043] A strategy determination module, used for inputting the state information into the cache optimization model to determine a cache replacement strategy corresponding to the state information;

[0044] The cache replacement module is used to replace the corresponding content in the cache space of the edge node with the target content according to the cache replacement strategy.

[0045] In a fifth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0046] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the method described in any one of the above embodiments are performed.

[0047] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0048] The cache optimization model training method, collaborative edge cache method and device provided in the present application, the training method includes: each user terminal in the federated system determines the expected revenue in this iteration according to its local data, and determines the communication cost in this iteration in combination with the allocated bandwidth to determine whether to participate in this iteration, the user terminal participating in this iteration uses its local state information as training data, trains the model parameters sent by the edge node, and uploads the trained model parameters to the edge node, after the edge node receives the model parameters uploaded by the user terminal, it aggregates these model parameters to obtain the global model parameters, and then updates the model parameters of the pre-trained model to enter the next iteration, until the preset training conditions are met, and the cache optimization model is obtained. In this process, each user terminal can determine whether to participate in each iteration according to its own expected revenue and communication cost, so that the user terminal with a larger contribution can be selected to participate in the training of the cache optimization model, optimize the allocation of training resources, and each user terminal can contribute to the training of the cache optimization model according to its own data characteristics under the premise of ensuring the privacy of user data. This personalization can help the model better predict user needs and improve the cache hit rate of cache optimization model output results, thereby improving cache efficiency and reducing content acquisition latency.

[0049] Furthermore, the cache optimization model is applied to the collaborative edge caching method. When the edge node receives a content acquisition request when the cache space is insufficient, the cache optimization model can be called to determine the cache replacement strategy to replace the corresponding content in the cache space. Due to the above advantages of the cache optimization model, the cache hit rate of the collaborative edge caching method is higher, thereby improving the cache efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A flowchart of a cache optimization model training method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of the steps of training received model parameters using local state information as training data for each target user terminal provided in the embodiment of the present application;

[0053] Figure 3 An example diagram of a process for a target user terminal to train received model parameters provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a process flow of a collaborative edge caching method provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a cache optimization model training device provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of a collaborative edge cache device provided in an embodiment of the present application;

[0057] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] In one embodiment, the present application provides a cache optimization model training method, and the following embodiment is described by applying the method to a server. It can be understood that the cache optimization model training method can be executed by a single server or a server cluster composed of multiple servers, and the present application does not make specific restrictions on this.

[0060] like Figure 1As shown, the present application provides a cache optimization model training method, which is applied to a federated learning system, wherein the federated learning system includes an edge node and multiple user terminals; the method includes:

[0061] Step S101: The edge node obtains a pre-trained model.

[0062] Among them, edge nodes represent edge devices, which are devices located at the edge of the network. They are usually close to the data source and can quickly process and analyze data. For example, smart meters, sensors, embedded systems, etc. Pre-trained models refer to pre-built initial models.

[0063] Step S102: The edge node sends the model parameters of the pre-trained model to each user terminal.

[0064] In this step, after the edge node obtains the pre-trained model, it sends the model parameters of the pre-trained model to each user terminal, so that each user terminal can determine whether to participate in the training of the received model parameters in this round according to its own situation.

[0065] Step S103: Each user terminal determines the estimated revenue in this iteration according to its local data, and determines the communication cost in this iteration according to its local data and the allocated bandwidth.

[0066] The expected benefit is used to measure the impact of the gradient of local data relative to the model parameters on the accuracy improvement of the pre-trained model, representing the contribution of local data to the improvement of the cache optimization model. The communication cost refers to the resources and expenses required to send and receive model parameters.

[0067] In this step, each user terminal can calculate the expected revenue of this iteration based on its own local data, and then calculate the communication cost of this iteration based on its own local data and the bandwidth allocated to each user terminal by the edge node. This allows each user terminal to determine whether to participate in this iteration based on the expected revenue and communication cost of this iteration. In this process, each user terminal aims to maximize its own interests. Through this competition mechanism, user terminals with greater contributions can be selected to participate in the training of the cache optimization model, thereby optimizing the allocation of training resources, and also accelerating the convergence of the model and improving the performance of the final trained model.

[0068] Specifically, each user terminal has its corresponding local data, which includes local state information, and the local state information includes requested content acquisition information and requested content cache information. In one example, the local state information s t It can be defined as:

[0069]

[0070] In the formula, Represents the request content to obtain information, Indicates that user u has obtained the request content on the terminal. f∈F means that the request content f is requested by user u; Indicates request content cache information. when When , it means that the request content f does not exist in the edge node. , it indicates that the request content f exists in the edge node.

[0071] Step S104: Each user terminal determines whether to participate in this iteration based on its expected revenue and communication cost.

[0072] In this step, each user terminal can determine whether to participate in this iteration based on its preset benefits and communication costs in this iteration. In this process, each user terminal makes determinations and choices with the goal of maximizing its own interests. In this way, user terminals with smaller expected benefits and higher communication costs can consider not participating in this iteration, saving system resources and thus optimizing the allocation of training resources.

[0073] Step S105: Each target user terminal uses the local state information as training data to train the received model parameters, and uploads the trained model parameters to the edge node.

[0074] The target user terminal is the user terminal determined to participate in this iteration. The local state information refers to the acquisition state of the content acquisition request, and the local state information includes the requested content acquisition information and the requested content cache information.

[0075] Specifically, the Markov Decision Process (MDP) can be used to model the process of responding to content acquisition requests, and the state is defined as the requested content acquisition information and the requested content cache information, the action is defined as the cache replacement strategy, and the function corresponding to the minimized acquisition delay is defined as the reward function. Based on this, in this step, each target user terminal uses its own local state information as training data to train the model parameters sent by the edge node, and then returns the trained model parameters to the edge node, so that the edge node aggregates the received model parameters.

[0076] Step S106: the edge node aggregates the model parameters received and uploaded by each target user terminal to form the global model parameters of this iteration, and updates the model parameters of the pre-trained model with the global model parameters.

[0077] Step S107: Determine whether the preset training conditions are met.

[0078] Step S108: Return to the step of sending the model parameters of the pre-trained model to each user terminal and continue executing.

[0079] Step S109: Determine the trained pre-trained model as the cache optimization model.

[0080] The cache optimization model training method, collaborative edge cache method and device provided in the present application, the training method includes: each user terminal in the federated system determines the expected revenue in this iteration according to its local data, and determines the communication cost in this iteration in combination with the allocated bandwidth to determine whether to participate in this iteration, the user terminal participating in this iteration uses its local state information as training data, trains the model parameters sent by the edge node, and uploads the trained model parameters to the edge node, after the edge node receives the model parameters uploaded by the user terminal, it aggregates these model parameters to obtain the global model parameters, and then updates the model parameters of the pre-trained model to enter the next iteration, until the preset training conditions are met, and the cache optimization model is obtained. In this process, each user terminal can determine whether to participate in each iteration according to its own expected revenue and communication cost, so that the user terminal with a larger contribution can be selected to participate in the training of the cache optimization model, optimize the allocation of training resources, and each user terminal can contribute to the training of the cache optimization model according to its own data characteristics under the premise of ensuring the privacy of user data. This personalization can help the model better predict user needs and improve the cache hit rate of cache optimization model output results, thereby improving cache efficiency and reducing content acquisition latency.

[0081] like Figure 2 As shown, in one embodiment, each target user terminal uses local state information as training data to train the received model parameters, including:

[0082] Step S201: Each target user terminal updates its local model according to the received model parameters.

[0083] In this step, the user terminals determined to participate in this iteration can update the model parameters of their respective local models according to the model parameters received from the edge nodes, so that the model parameters of the local models of the target user terminals are synchronized with the model parameters of the global model.

[0084] Step S202: Each target client inputs its local state information into its local model to determine the cache replacement strategy of each target client in this iteration.

[0085] The cache replacement strategy refers to a cache content replacement method that minimizes the content acquisition delay under the corresponding state information. In an example, the expression of the cache replacement strategy can be expressed as follows:

[0086]

[0087] In the formula, Indicates that the content has been obtained on the edge node. f∈F means that content f will be replaced by the cache. This means that collaborative caching is not used for content acquisition. Collaborative caching is used to obtain content. This means that the cloud is not used for content acquisition. The cloud is used to obtain content.

[0088] In this step, each target user terminal inputs its own local state information into its corresponding local model, thereby obtaining the output result of the local model of each target user terminal, that is, the cache replacement strategy.

[0089] Step S203: Each target user terminal determines the reward function corresponding to this iteration according to its cache replacement strategy in this iteration.

[0090] Among them, the reward function is used to evaluate the content acquisition delay of executing the cache replacement strategy under the corresponding state information.

[0091] In one example, the reward function is expressed as:

[0092]

[0093] In the formula, r t represents the reward function, Represents the latency of content acquisition requests in the cloud. Represents the average latency required for content acquisition requests to obtain content.

[0094] According to the reward function, the target optimization function can be determined to optimize the long-term return in model training. Assuming that the state information is s and the cache replacement strategy is π(s), the long-term benefit from the beginning to the end of training is maximized, which is expressed as follows:

[0095]

[0096] In the formula, r L represents the target optimization function, r(s t ,π(s t )) indicates that under the status information t Execute cache replacement strategy π(s t) received.

[0097] According to the Bellman equation, the average reward is calculated as follows:

[0098]

[0099] Where V π (s) represents the average reward, γ t-1 =[0,1] represents the discount factor, which is used to measure the impact of the reward value on the current state information. For the explanation of other symbols, please refer to the explanation of the above formula.

[0100] Step S204: Each target user terminal determines the loss function of each target user terminal in this iteration according to the local state information, cache replacement strategy and reward function of each target user terminal, and minimizes the loss function of each target user terminal in this iteration to update the local model of each target user terminal, and use the model parameters of the updated local model as the trained model parameters.

[0101] The loss function is used to evaluate the gap between the local model of the target user end and the model parameters of the corresponding target Q network. In one example, the loss function can be expressed as follows:

[0102]

[0103] Where L(θ) represents the loss function, and Q represents the action-value function, Indicates that the target Q network has model parameters The expected return of executing cache replacement strategy a under state information s, Q(s,a|θ) represents the expected return of the local model (evaluation Q network) executing cache replacement strategy a under model parameters θ and state information s.

[0104] Specifically, each user terminal has an evaluation Q network and a target Q network, where the evaluation Q network serves as the local model of the user terminal. The evaluation Q network and the target Q network are both composed of three convolutional layers and two fully connected layers.

[0105] In this step, each target user terminal inputs its local state information and its corresponding cache replacement strategy and reward function into its local model and the corresponding target Q network to determine the output of the local model and the target Q network, and then each target user terminal determines the corresponding loss function of the target user terminal in this iteration based on the output of the two.

[0106] like Figure 3 As shown, Figure 3An example diagram of the process of training the received model parameters by the target user end provided in the embodiment of the present application. In the figure, the Dueling DQN loss function is the loss function in this embodiment, the evaluation Q network is the local model in the target user end, and the environmental information can be understood as the environmental information related to the edge node. The action refers to the cache replacement strategy, and the gradient of the loss function can be understood as the gradient of the local data of the target user end.

[0107] It can be understood that through pre-defined action information, cache replacement strategy and reward function, the caching process is modeled as a Markov decision process, and then the cumulative reward is maximized based on the above steps, so that the final cache optimization model can accurately output the cache replacement strategy with the highest reward, improve cache efficiency and reduce the latency of content acquisition.

[0108] In one embodiment, each target user terminal determines the reward function corresponding to the current iteration according to its cache replacement strategy in the current iteration, including:

[0109] Each target user terminal sends its cache replacement strategy in this iteration to the edge node to execute each cache replacement strategy and determine the first delay and second delay of each target user terminal; the first delay is the delay of obtaining content at the cloud node, and the second delay is the average delay required to obtain content;

[0110] Each target user terminal determines its corresponding reward function according to its first time delay and second time delay.

[0111] Among them, the second delay is used to measure the average time delay for users to obtain content.

[0112] In an example, the process of this embodiment can be expressed using the following expression:

[0113]

[0114] In the formula, r t represents the reward function, Represents the latency of content acquisition requests in the cloud. Represents the average latency required for content acquisition requests to obtain content.

[0115] In this embodiment, the first delay and the second delay of each target user terminal are collected after executing the cache replacement strategy, and then based on the first delay and the second delay of each target user terminal, the reward function corresponding to each target user terminal in this iteration is calculated, so that the training model is closer to the training target direction, and finally the purpose of improving the cache hit rate to improve the cache efficiency is achieved.

[0116] In one embodiment, the step of each user terminal determining whether to participate in the current iteration according to its expected revenue and communication cost includes:

[0117] Each user terminal determines its utility coefficient in this iteration based on its expected revenue and communication cost;

[0118] For any one of the user terminals, when the utility coefficient of the user terminal is greater than the preset threshold corresponding to the user terminal, the user terminal participates in this iteration; when the utility coefficient of the user terminal is not greater than the preset threshold corresponding to the user terminal, the user terminal does not participate in this iteration.

[0119] The utility coefficient is used to evaluate the effective benefits of the user in this iteration. The preset threshold is an empirical value.

[0120] In this embodiment, the utility coefficient is determined by calculating the difference between the expected benefit and the communication cost, and then the utility coefficient is compared with the preset threshold corresponding to the user. In this process, the utility coefficient is used to evaluate the effective benefit of the user terminal participating in this iteration to determine whether to participate in this iteration, and the preset thresholds compared by each user terminal are not necessarily equal. The preset threshold of each user terminal can be set according to the actual situation and needs of the user terminal. In this way, the flexibility of the user terminal to participate in training is improved to help the model better predict user needs and improve the cache hit rate of the cache optimization model output results, thereby improving cache efficiency and reducing the delay of content acquisition.

[0121] In one embodiment, the steps of each user terminal determining an estimated revenue in this iteration according to its local data, and determining a communication cost in this iteration according to its local data and the allocated bandwidth include:

[0122] Each user terminal calculates the local data gradient based on its local data, and determines the expected benefit of each user terminal in this iteration in combination with the preset benefit factor;

[0123] Each user terminal determines the communication cost of each user terminal in this iteration according to the bandwidth allocated by the edge node, combined with the local data gradient of each user terminal and the preset cost consumption coefficient.

[0124] The local data gradient refers to the gradient of the local data corresponding to the user end.

[0125] In an example, the expression of the utility coefficient of a certain user terminal in this iteration can be expressed as follows:

[0126]

[0127] Where U n represents the utility coefficient, ρGn Indicates the expected revenue, represents the communication cost, ρ is the benefit factor, β is the cost consumption coefficient, both the benefit factor and the cost consumption coefficient are preset values ​​greater than 0, G n is the local data gradient, B n The bandwidth allocated to the client n by the edge node.

[0128] It is understandable that when considering the benefits of participating in federated learning, the user side also needs to weigh the communication resources consumed by participating in federated learning. By calculating the local data gradient, it is possible to estimate the contribution of local data to model improvement, and then determine the improvement of model accuracy by local data in this iteration through the estimated benefits, and then determine the communication cost consumed in this iteration through the ratio of local data gradient and bandwidth. In this way, it is determined whether to participate in this iteration to optimize the allocation of training resources, accelerate model convergence, and improve model performance and accuracy.

[0129] In one embodiment, the step of aggregating the model parameters received and uploaded by each target user terminal by the edge node includes:

[0130] The edge node determines the average value of each model parameter according to the received model parameters uploaded by each target user terminal, and determines the average value as the global model parameter.

[0131] It is understandable that average aggregation helps reduce overfitting during model training because it reduces the impact of individual user-side data characteristics on the global model by integrating parameters from multiple users. In addition, this approach can also improve the robustness of the model because it is resistant to outliers or noise in individual user-side data.

[0132] In one embodiment, the present application provides a collaborative edge caching method, and the following embodiment is described by applying the method to a server. It can be understood that the collaborative edge caching method can be performed by a single server or a server cluster composed of multiple servers, and the present application does not make specific restrictions on this.

[0133] like Figure 4 As shown, the present application provides a collaborative edge caching method, which is applied to an edge node in a federated system; the federated system includes an edge node and multiple user terminals; the method includes:

[0134] Step S301: receiving a content acquisition request initiated by a user terminal, and determining target content according to the content acquisition request.

[0135] The target content refers to the content that the user terminal wants to obtain.

[0136] In this step, when the user terminal wants to obtain certain data content, it can initiate a content acquisition request to its corresponding edge node. When the edge node receives the content acquisition request, it determines the target content by parsing the content acquisition request. If the target content exists in the cache space of the edge node, the content data corresponding to the target content is directly obtained from the cache space of the edge node. If the target content does not exist in the cache space of the edge node and the cache space of the edge node is sufficient, the content data corresponding to the target content is obtained from the cloud and stored in the cache space of the edge node.

[0137] Step S302: when the target content does not exist in the cache space of the edge node and the cache space is insufficient, the state information is determined according to the content acquisition request.

[0138] The status information refers to the acquisition status of the content acquisition request, and the status information includes requested content acquisition information and requested content cache information.

[0139] In this step, when the target content does not exist in the cache space of the edge node and the cache space is insufficient, it is necessary to consider replacing the data in the cache space of the edge node to replace it with the content data corresponding to the target content obtained from the cloud. Therefore, it is necessary to determine the status information based on the content acquisition request to call the cache optimization model to determine the cache replacement strategy.

[0140] Step S303: Obtain a cache optimization model.

[0141] The cache optimization model is obtained according to the cache optimization model training method of any of the above embodiments.

[0142] Step S304: input the state information into the cache optimization model to determine a cache replacement strategy corresponding to the state information.

[0143] Step S305: According to the cache replacement strategy, the target content is used to replace the corresponding content in the cache space of the edge node.

[0144] It is understandable that when the cache optimization model is applied to the collaborative edge caching method, when the edge node receives a content acquisition request when the cache space is insufficient, the cache optimization model can be called at this time to determine the cache replacement strategy to replace the corresponding content in the cache space. Due to the above advantages of the cache optimization model, the cache replacement strategy output by it can be generated according to different user personalization, thereby improving the cache hit rate, and then making the cache hit rate of the collaborative edge caching method higher, so as to improve the cache efficiency.

[0145] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0146] The cache optimization model training device provided in an embodiment of the present application is described below. The cache optimization model training device described below and the cache optimization model training method described above can be referenced to each other.

[0147] like Figure 5 As shown, the present application provides a cache optimization model training device 400, which is applied to a federated learning system, wherein the federated learning system includes an edge node and multiple user terminals; the device includes:

[0148] The parameter sending module 401 is used for the edge node to obtain the pre-trained model and send the model parameters of the pre-trained model to each user terminal;

[0149] The data determination module 402 is used for each user terminal to determine the expected revenue in this iteration according to its local data, and to determine the communication cost in this iteration according to its local data and the allocated bandwidth;

[0150] Participation determination module 403, used for each user terminal to determine whether to participate in this iteration according to its expected revenue and communication cost;

[0151] The model training module 404 is used for each target user end to train the received model parameters using the local state information as training data, and upload the trained model parameters to the edge node; wherein the target user end is the user end determined to participate in this iteration;

[0152] The parameter updating module 405 is used for the edge node to aggregate the model parameters uploaded by each target user terminal to form the global model parameters of this iteration, and to update the model parameters of the pre-trained model with the global model parameters, and to return to the step of sending the model parameters of the pre-trained model to each user terminal to continue executing until the preset training conditions are met, and the trained pre-trained model is determined as the cache optimization model.

[0153] In one embodiment, the model training module includes:

[0154] The model updating submodule is used for each target user end to update its local model according to the received model parameters;

[0155] A strategy determination submodule is used for each target user end to input its local state information into its local model to determine the cache replacement strategy of each target user end in this iteration;

[0156] The function determination submodule is used for each target user end to determine the reward function corresponding to this iteration according to its cache replacement strategy in this iteration;

[0157] The model training submodule is used for each target user terminal to determine the loss function of each target user terminal in this iteration according to the local state information, cache replacement strategy and reward function of each target user terminal, and minimize the loss function of each target user terminal in this iteration, so as to update the local model of each target user terminal and use the model parameters of the updated local model as the trained model parameters.

[0158] In one embodiment, the function determination submodule includes:

[0159] A delay determination unit is used for each target user terminal to send its cache replacement strategy in this iteration to the edge node to execute each cache replacement strategy and determine the first delay and the second delay of each target user terminal; the first delay is the delay of obtaining content at the cloud node, and the second delay is the average delay required to obtain content;

[0160] The function determination unit is used for each target user terminal to determine its corresponding reward function according to its first delay and second delay.

[0161] In one embodiment, the participation determination module includes:

[0162] The coefficient determination submodule is used for each user terminal to determine its utility coefficient in this iteration according to its expected revenue and communication cost;

[0163] The participation determination submodule is used for, for any one of the user terminals, when the utility coefficient of the user terminal is greater than the preset threshold corresponding to the user terminal, the user terminal participates in this iteration; when the utility coefficient of the user terminal is not greater than the preset threshold corresponding to the user terminal, the user terminal does not participate in this iteration.

[0164] In one embodiment, the data determination module includes:

[0165] The revenue determination submodule is used for each user terminal to calculate the local data gradient according to its local data, and determine the expected revenue of each user terminal in this iteration in combination with the preset revenue factor;

[0166] The cost determination submodule is used for each user terminal to determine the communication cost of each user terminal in this iteration according to the bandwidth allocated by the edge node, combined with the local data gradient of each user terminal and the preset cost consumption coefficient.

[0167] In one embodiment, the parameter updating module includes:

[0168] The parameter aggregation submodule is used for the edge node to determine the average value of each model parameter according to the model parameters uploaded by each target user terminal, and determine the average value as the global model parameter.

[0169] The following describes a collaborative edge cache device provided in an embodiment of the present application. The collaborative edge cache device described below and the collaborative edge cache method described above can be referenced to each other.

[0170] like Figure 6 As shown, the present application provides a collaborative edge cache device 500, which is applied to an edge node in a federated system; the federated system includes an edge node and multiple user terminals; the device includes:

[0171] The request receiving module 501 is used to receive a content acquisition request initiated by a user terminal and determine the target content according to the content acquisition request;

[0172] An information determination module 502, configured to determine status information according to a content acquisition request when the target content does not exist in the cache space of the edge node and the cache space is insufficient;

[0173] The model acquisition module 503 is used to acquire a cache optimization model; the cache optimization model is obtained according to the cache optimization model training method of any of the above embodiments;

[0174] A strategy determination module 504, configured to input the state information into a cache optimization model to determine a cache replacement strategy corresponding to the state information;

[0175] The cache replacement module 505 is used to replace the corresponding content in the cache space of the edge node with the target content according to the cache replacement strategy.

[0176] The division of each module in the above device is only for illustration. In other embodiments, the device can be divided into different modules as needed to complete all or part of the functions of the above device. Each module in the above device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0177] In one embodiment, the present application further provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method described in any one of the above embodiments.

[0178] Indicatively, if Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 600 may be provided as a server. Figure 7 , the computer device 600 includes a processing component 602, which further includes one or more processors, and a memory resource represented by a memory 601, for storing instructions that can be executed by the processing component 602, such as an application. The application stored in the memory 601 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 602 is configured to execute instructions to perform the method of any of the above embodiments.

[0179] The computer device 600 may further include a power supply component 603 configured to perform power management of the computer device 600, a wired or wireless network interface 604 configured to connect the computer device 600 to a network, and an input / output (I / O) interface 605. The computer device 600 may operate based on an operating system stored in the memory 601, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0180] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0182] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0183] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cache optimization model training method, characterized in that: Applied to a federated learning system, the federated learning system includes an edge node and multiple user terminals; the method includes: The edge node obtains a pre-trained model and sends model parameters of the pre-trained model to each user terminal; Each user terminal determines the expected revenue in this iteration according to its local data, and determines the communication cost in this iteration according to its local data and the allocated bandwidth; Each user terminal determines whether to participate in this iteration based on its expected revenue and communication cost; Each target user terminal uses local state information as training data to train the received model parameters, and uploads the trained model parameters to the edge node; wherein the target user terminal is the user terminal determined to participate in this iteration; The edge node aggregates the model parameters uploaded by each target user terminal to form the global model parameters of this iteration, and updates the model parameters of the pre-trained model with the global model parameters, and returns to the step of sending the model parameters of the pre-trained model to each user terminal to continue executing until the preset training conditions are met, and the trained pre-trained model is determined as the cache optimization model.

2. The cache optimization model training method according to claim 1, characterized in that: The step of each target user terminal using local state information as training data to train the received model parameters includes: Each target user terminal updates its local model according to the received model parameters; Each target user terminal inputs its local state information into its local model to determine the cache replacement strategy of each target user terminal in this iteration; Each target user terminal determines the reward function corresponding to this iteration according to its cache replacement strategy in this iteration; Each target user terminal determines the loss function of each target user terminal in this iteration according to the local state information, cache replacement strategy and reward function of each target user terminal, and minimizes the loss function of each target user terminal in this iteration to update the local model of each target user terminal, and use the model parameters of the updated local model as the trained model parameters.

3. The cache optimization model training method according to claim 2, characterized in that: Each target user terminal determines the steps of the reward function corresponding to this iteration according to its cache replacement strategy in this iteration, including: Each target user terminal sends its cache replacement strategy in this iteration to the edge node to execute each cache replacement strategy and determine the first delay and the second delay of each target user terminal; the first delay is the delay of obtaining content at the cloud node, and the second delay is the average delay required to obtain content; Each target user terminal determines its corresponding reward function according to its first time delay and second time delay.

4. The cache optimization model training method according to claim 1, characterized in that: The step of each user terminal determining whether to participate in this iteration according to its expected revenue and communication cost includes: Each user terminal determines its utility coefficient in this iteration based on its expected revenue and communication cost; For any one of the user terminals, when the utility coefficient of the user terminal is greater than the preset threshold corresponding to the user terminal, the user terminal participates in this iteration; when the utility coefficient of the user terminal is not greater than the preset threshold corresponding to the user terminal, the user terminal does not participate in this iteration.

5. The cache optimization model training method according to claim 1, characterized in that: The steps of each user terminal determining the estimated revenue in this iteration according to its local data, and determining the communication cost in this iteration according to its local data and the allocated bandwidth include: Each user terminal calculates the local data gradient based on its local data, and determines the expected benefit of each user terminal in this iteration in combination with the preset benefit factor; Each user terminal determines the communication cost of each user terminal in this iteration according to the bandwidth allocated by the edge node, in combination with the local data gradient of each user terminal and a preset cost consumption coefficient.

6. The cache optimization model training method according to any one of claims 1 to 5, characterized in that: The step of aggregating the model parameters received and uploaded by each target user terminal by the edge node includes: The edge node determines an average value of each model parameter according to the received model parameters uploaded by each target user terminal, and determines the average value as the global model parameter.

7. A collaborative edge caching method, characterized in that: An edge node applied to a federated system; the federated system includes an edge node and multiple user terminals; the method includes: receiving a content acquisition request initiated by the user terminal, and determining target content according to the content acquisition request; When the target content does not exist in the cache space of the edge node and the cache space is insufficient, determining the state information according to the content acquisition request; Obtaining a cache optimization model; the cache optimization model is obtained according to the cache optimization model training method according to any one of claims 1 to 6; Inputting the state information into the cache optimization model to determine a cache replacement strategy corresponding to the state information; According to the cache replacement strategy, the target content is used to replace the corresponding content in the cache space of the edge node.

8. A cache optimization model training device, characterized in that: Applied to a federated learning system, the federated learning system includes an edge node and multiple user terminals; the device includes: A parameter sending module, used for the edge node to obtain a pre-trained model and send the model parameters of the pre-trained model to each user terminal; A data determination module, used for each user terminal to determine the expected revenue in this iteration according to its local data, and to determine the communication cost in this iteration according to its local data and the allocated bandwidth; The participation determination module is used for each user terminal to determine whether to participate in this iteration based on its expected revenue and communication cost; The model training module is used for each target user terminal to train the received model parameters using the local state information as training data, and upload the trained model parameters to the edge node; wherein the target user terminal is the user terminal determined to participate in this iteration; The parameter updating module is used for the edge node to aggregate the model parameters uploaded by each target user terminal to form the global model parameters of this iteration, and to update the model parameters of the pre-trained model with the global model parameters, and to return to the step of sending the model parameters of the pre-trained model to each user terminal to continue executing until the preset training conditions are met, and the trained pre-trained model is determined as the cache optimization model.

9. A collaborative edge cache device, characterized in that: An edge node applied to a federated system; the federated system includes an edge node and multiple user terminals; the device includes: A request receiving module, used to receive a content acquisition request initiated by the user terminal, and determine target content according to the content acquisition request; An information determination module, configured to determine status information according to the content acquisition request when the target content does not exist in the cache space of the edge node and the cache space is insufficient; A model acquisition module, used to acquire a cache optimization model; the cache optimization model is obtained according to the cache optimization model training method according to any one of claims 1 to 6; A strategy determination module, used for inputting the state information into the cache optimization model to determine a cache replacement strategy corresponding to the state information; The cache replacement module is used to replace the corresponding content in the cache space of the edge node with the target content according to the cache replacement strategy.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the method according to any one of claims 1 to 7 are performed.