Multi-agent collaborative edge caching method and system based on gated communication mechanism

Abstract

The present invention proposes a multi-agent collaborative edge caching method and system based on a gated communication mechanism. The method deploys agents in multiple edge servers, uses a gated model to judge the cache status of the edge servers, and decides whether to upload the corresponding agent status information to the central server to participate in the global decision-making according to the judgment result. After the central server receives the agent status information, it comprehensively considers the state information of each agent based on the multi-agent decision model, determines the optimal cache decision through a reinforcement learning algorithm, and sends the optimal cache decision to the corresponding edge server. The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management according to the cache decision. The present invention can automatically determine the optimal location of the collaborative edge server, enables selective information exchange between servers, and reduces the waste of bandwidth resources.

Description

Technical Field

[0001] The present invention relates to the field of edge caching, and in particular to a multi-agent collaborative edge caching method and system based on a gated communication mechanism. Background Art

[0002] Currently, many caching schemes developed for single edge servers may be applicable to multi-server scenarios, but their inherent limitations may hinder the effectiveness of optimal collaborative caching. Although techniques such as integer linear programming and dynamic programming can provide dynamic and adaptive caching frameworks, they still focus mainly on optimization within the scope of a single server. Similarly, although good at decision-making and traffic optimization in complex network environments, single-agent reinforcement learning models focus on optimizing caching decisions of a single server rather than taking a holistic collaborative approach. This isolated optimization paradigm can produce different service qualities among edge servers in a region, especially during asynchronous request peaks. In addition, most collaborative edge caching frameworks use distributed learning algorithms for decentralized decision-making, or require all edge nodes to upload data to a decision center for centralized decision-making.

[0003] Therefore, the former independent decision-making method may cause popular content to be cached multiple times by edge servers. This will significantly reduce the cache performance of edge servers, especially during high request traffic. The latter centralized decision-making method may reduce the accuracy and timeliness of decisions due to the delay in information upload. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose a multi-agent collaborative edge caching method and system based on a gated communication mechanism to solve the above technical problems.

[0005] The present invention proposes a multi-agent collaborative edge caching method based on a gated communication mechanism, the method comprising the following steps:

[0006] Step 1: The client sends the request information to the nearest edge server based on its location;

[0007] Step 2: The nearest edge server confirms whether the cache hits according to the description information. If not, it sends a cache request to the adjacent edge server or central server according to the current edge server's own cache status and network load;

[0008] Step 3: deploy an agent for each edge server, and the agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request;

[0009] The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making;

[0010] Step 4: After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal cache decision through the reinforcement learning algorithm, and sends the optimal cache decision to the corresponding edge server;

[0011] Step 5: The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management according to the cache decision.

[0012] The present invention also proposes a multi-agent collaborative edge caching system based on a gated communication mechanism, wherein the system applies the multi-agent collaborative edge caching method based on a gated communication mechanism as described above, and the system comprises:

[0013] User side, used for:

[0014] The client sends the request information to the nearest edge server based on its location;

[0015] Edge side, used for:

[0016] The nearest edge server confirms whether the cache hits based on the description information. If not, it sends a cache request to the adjacent edge server or central server based on the current edge server's own cache status and network load;

[0017] An intelligent agent is deployed for each edge server, and the intelligent agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request;

[0018] The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making;

[0019] Cloud for:

[0020] After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal caching decision through the reinforcement learning algorithm, and sends the optimal caching decision to the corresponding edge server;

[0021] The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management based on the cache decision.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. Through the gated communication mechanism, an adaptive information exchange mechanism is established between edge servers. Different from the traditional fixed information transmission mode, the gated communication mechanism decides whether to upload information based on the current network status and the cache status of the edge server. Through this selective information transmission method, unnecessary communication overhead is effectively reduced, and network load and data transmission delay are reduced. At the same time, the mechanism can flexibly adjust the communication frequency according to the current load status of the server, ensuring that the system maintains a high response speed and service quality while efficiently utilizing resources.

[0024] 2. Adaptive collaboration of edge servers in a dynamic network environment is achieved by using multi-agent reinforcement learning technology. Each edge server acts as an independent agent and dynamically adjusts the cache strategy according to cache status and client requests through collaborative learning with other servers. The reinforcement learning model introduces a reward mechanism to encourage edge servers to share cache resources to maximize the cache hit rate of the overall system while reducing the use of backhaul bandwidth. This self-adaptation based on reinforcement learning enables edge servers to make autonomous decisions, optimize cache allocation, and improve the operating efficiency of the overall network.

[0025] 3. The proposed cache strategy is not limited to the cache optimization of a single edge server, but also realizes the global optimization configuration of cache resources through the coordination of the central server. The central server receives status information from multiple edge servers, formulates cache update strategies using reinforcement learning models, and sends cache instructions to each edge server. This global optimization process ensures balanced resource allocation among multiple servers in the region, avoids cache overload or cache redundancy on some servers, and further improves the cache hit rate and response efficiency of the entire system.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of a multi-agent collaborative edge caching method based on a gated communication mechanism proposed by the present invention;

[0028] Figure 2 This is a schematic diagram of the collaborative edge network structure of the present invention;

[0029] Figure 3 A schematic diagram of a proxy model of an edge server of the present invention;

[0030] Figure 4 A schematic diagram of the model of communication and cache decision-making based on multi-agent reinforcement learning proposed by the present invention;

[0031] Figure 5 This is a schematic diagram of the structure of the multi-agent collaborative edge caching system based on the gated communication mechanism proposed in the present invention. DETAILED DESCRIPTION

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0033] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0034] See also Figure 1 This embodiment provides a multi-agent collaborative edge caching method based on a gated communication mechanism, the method comprising the following steps:

[0035] Step 1: The client sends the request information to the nearest edge server based on its location;

[0036] In this step, the client device determines its current physical location through the built-in positioning system (such as global positioning system GPS, Wi-Fi positioning, cellular base station information, etc.), and selects the nearest edge server based on the location information. This process can be achieved through the following mechanisms:

[0037] Location determination and edge server selection: The client uses positioning technologies such as GPS or obtains location information from nearby cellular base stations, and then calculates the physical distance to multiple edge servers. Based on the principle of the closest distance, the client automatically connects to the nearest edge server. This selection process can effectively reduce the delay of data transmission in the network and improve the response speed of data access.

[0038] In this step, after connecting to the edge server, the client generates and sends a file request message.

[0039] The request information contains the following metadata:

[0040] File ID: It is used to uniquely identify the file or content requested by the client. The file identifier can be a hash value, file name, or other unique identification method of the file so that the server can quickly locate the location of the file in the cache.

[0041] Requested file size: This metadata is used to inform the server how much storage space the requested file takes up. By providing file size information, the server can better determine whether there is enough storage space to cache the file during cache management.

[0042] Priority: Priority information is used to help edge servers decide the order in which to process requests based on the current network and storage load. If some file requests are of high real-time or importance, the client can mark the request as high priority when sending the request, so as to obtain a quick response from the server.

[0043] Optimization of data transmission delay: By automatically selecting the nearest edge server, the data transmission distance is minimized, thereby reducing the data transmission delay in the network. Especially in the edge cache environment of multi-agent collaboration, reducing the communication delay between the client and the server is crucial to improving the cache hit rate and service quality of the entire system.

[0044] Request processing mechanism of edge server: After receiving the request from the client, the edge server first checks whether the file is already cached in the local cache based on the file identifier and file size in the request information. If the file is in the cache, the server will immediately respond to the client's request and transfer the file to the client. Otherwise, the server will further evaluate whether it needs to request the central server for cache update or data forwarding.

[0045] Step 2: The nearest edge server confirms whether the cache hits according to the description information. If not, it sends a cache request to the adjacent edge server or central server according to the current edge server's own cache status and network load;

[0046] In this step, after receiving the file request from the client, the edge server will first check the file in the local cache. If the file is already cached, it will be returned to the client immediately; if it is not cached, the edge server will decide whether to forward the request to a neighboring edge server or central server based on its current cache status, cache hit rate, and network bandwidth. When the neighboring edge server cannot meet the current request or the overall optimization algorithm of the current system determines that the central server is more efficient to respond based on the global status, the edge server will forward the request to the central server, and the central server will complete this data request.

[0047] When an IoT application accesses an edge server within its service area, the interconnected edge servers prioritize retrieving the requested data from their local caches. A central server oversees and coordinates all edge servers. If a requested data item is not found locally, closer edge servers may be able to fulfill the request over internal connections, depending on their cache capacity. The central server maintains a comprehensive view of the caches of all edge servers, distributing and orchestrating caching decisions. If an edge server is unable to fulfill a request, the central server can serve as a fallback mechanism to fetch data from the Internet over a high-bandwidth backhaul link.

[0048] like Figure 2 As shown, this embodiment adopts Represents the set of all edge servers in the region, , Indicates the area N The cache capacity of the edge servers, Indicates the total number of edge servers. Represents edge server i The surrounding client collection, , Represents edge server i The Mth client around, Indicates the number of clients. The central server uses Said that n The round channel gain calculation formula is as follows:

[0049] ;

[0050] in, Indicates the device a and b The distance between represents the effects of large-scale fading, including path loss and shadowing. The path loss is calculated as , the shadows follow a log-normal distribution. is the small-scale fading effect, which is exponentially distributed with unit mean.

[0051] The change in distance between communicating entities causes fluctuations in channel impairments, which leads to dynamic transmission rates between different rounds. Based on Shannon’s theorem, we calculate the transmission rate between the local edge server and the central server using the following formula:

[0052] ;

[0053] in, Indicates the transmission power level used by the central server. represents the noise power, Indicates the channel gain.

[0054] Similarly, the transmission rate between the local edge server and the nearby edge server is calculated as follows:

[0055] ;

[0056] in, Indicates the available bandwidth between the current edge server and the collaborative edge server. Indicates the transmission power level used by the collaborative edge server. Figure 1 .

[0057] Step 3: deploy an agent for each edge server, and the agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request;

[0058] The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making;

[0059] In this step, when the gated model is used to judge the edge server cache status, the current edge server performs a comprehensive analysis and judgment based on the current network load and the current cache status; other edge servers perform analysis and judgment based on the corresponding cache status.

[0060] Before uploading status information, the edge server first uses the gating model to comprehensively analyze its own cache status (such as cache space utilization and cache content popularity) and network load (such as bandwidth availability and latency). The gating model sets thresholds and rules to determine whether the uploaded status information has global decision-making significance, avoid meaningless communication, and reduce communication overhead and upload delay.

[0061] Other edge servers make analysis and judgment based on the corresponding cache status. The gating model evaluates the cache status of other edge servers in the network (such as cache space utilization, cache content popularity, etc.), and combines the current edge server status to determine whether information needs to be uploaded. Status data is uploaded only when the current edge server status is related to the global cache decision, thus achieving efficient collaboration of the system.

[0062] Further preferably, other edge servers analyze and judge according to the corresponding cache status, specifically including the following steps:

[0063] Obtain the absolute location of each edge server and perform location encoding to obtain the encoded location information. The corresponding process has the following relationship:

[0064] ;

[0065] in, Indicates j The absolute location of the edge servers, represents the first linear layer, Indicates j The location information encoded by the edge server;

[0066] Input the cache state of each edge server into the LSTM model to obtain the hidden state;

[0067] The hidden state and the encoded position information are input into the gating model to obtain a binary output. The corresponding process has the following relationship:

[0068] ;

[0069] in, represents the gating model, Indicates the hidden state, Indicates binary output;

[0070] Gated Output The binary value of can be obtained by combining the activation function with the threshold processing to ensure that the output is in the range of {0,1}.

[0071] like If is 0, the current LSTM model does not share self-description state information with other LSTM models at the next moment. In this case, the LSTM model is in an independent self-description state and only records local information. , it means that the current LSTM model allows sharing of its hidden state information. This state information is transmitted to the central server or other edge devices through a specific sharing mechanism, thereby realizing the exchange of state information. This gating mechanism can realize flexible state sharing control in a multi-agent environment. This mechanism enables each edge server to dynamically evaluate its current network status and decide whether to share cached data or hidden state information according to actual needs. Specifically, the mechanism evaluates whether to allow each edge server to continue to upload subsequent state information to the central server based on the state information previously submitted by the server. The main purpose of this mechanism is to minimize the transmission of unnecessary information, thereby reducing network overhead and latency. Each edge server can independently evaluate the current network status and decide whether it is necessary to share its cached data.

[0072] In this embodiment, the agent adopts the LSTM model, and the encoded position information is input into the LSTM model as part of the current environment state. In order to enable the LSTM model to independently decide whether to communicate with other agents in the area, the present invention designs a gating model. If the gating model decides not to communicate, the edge server does not need to upload the local agent state information to participate in the cache decision. By combining the information after the position encoding, the gating mechanism can consider the relative position of the device in space, thereby more intelligently controlling the communication and data transmission decisions.

[0073] Step 4: After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal cache decision through the reinforcement learning algorithm, and sends the optimal cache decision to the corresponding edge server;

[0074] In this embodiment, the multi-agent decision model also adopts the LSTM model. The central server comprehensively considers the agent status information of each edge server based on the multi-agent decision model, and determines the optimal cache update strategy through the reinforcement learning algorithm to maximize the cache hit rate and reduce the return traffic.

[0075] like Figure 3 As shown, in this step, based on the multi-agent decision model, the state information of each agent is comprehensively considered, and the optimal collaborative caching strategy is determined by the reinforcement learning algorithm, which specifically includes the following steps:

[0076] When the central server obtains the agent state information filtered by the gated communication mechanism, it inputs the agent state information into the LSTM model to obtain the next layer of hidden state. The corresponding process has the following relationship:

[0077] ;

[0078] in, represents the hidden state of the next layer, Represents the input data of the model at time t+1, that is, the description information of some edge servers received by the central server, represents the second linear layer, Indicates the cell status, Represents the input data of the model at time t;

[0079] Add the binary output to the weight of the next hidden state, and after a dot product operation, get the next layer of cell state. The corresponding process has the following relationship:

[0080] ;

[0081] in, represents the absolute value of the number of all agents, represents the linear transformation matrix that transforms the gated average hidden state into the communication tensor, Indicates the state of the next layer of cells; since the amount of data of the agent's own weight parameters is much smaller than the amount of data of the network description information, more efficient information exchange between multiple agents is achieved through the aggregation of agent model parameters.

[0082] The hidden state of the next layer and the cell state of the next layer are input into the third linear layer, and then the action vector and position vector are obtained through the softmax function. The corresponding process has the following relationship:

[0083] ;

[0084] in, represents the third linear layer, represents the activation function, represents a position vector, storing the positions of the collaborative edge servers adjacent to the current edge server; Represents the action vector, which stores the actions to be performed by each edge server.

[0085] According to the position vector, the corresponding action vector is sent to the corresponding edge server;

[0086] After receiving the action vector, the edge server performs corresponding cache management operations according to the action vector, and gives corresponding reward values ​​according to the content transmission delay and cache space utilization during the cache management operation;

[0087] The action vector is optimized by maximizing the overall reward value to obtain the optimal collaborative caching strategy.

[0088] Specifically, the corresponding cache management operations performed according to the action vector include:

[0089] When the action vector is 0, the current edge server retains any cache operations on the current request data;

[0090] When the action vector is 1, the current request data is cached locally on the current edge server;

[0091] When the action vector is -1, the nearest edge server is determined as the assisting edge server according to the position vector. The collaborative edge server caches the current request data. When the cache space is insufficient to satisfy the incoming data request, the old data is replaced by the LRU algorithm to free up cache space for caching.

[0092] Specifically, optimizing the action vector by maximizing the overall reward value to obtain the optimal collaborative caching strategy includes the following steps:

[0093] Calculate the reward value of all edge servers in the region;

[0094] Construct an objective function to maximize the overall reward value based on the reward values ​​of all edge servers;

[0095] The objective function is maximized to obtain the optimal collaborative caching strategy.

[0096] like Figure 4 As shown, the theoretical basis of the deep reinforcement learning framework is several basic elements, including state representation, action selection, reward mechanism and target optimization. In order to facilitate direct collaboration between edge servers, the present invention introduces the location element. In each time slot, the edge server determines the action by considering its current state and current environmental conditions, and communicates it to the central server together. The central server acts as a decision engine, generating actions customized for the unique state of each time slot, thereby learning dynamically and adaptively.

[0097] State representation: To fully understand the state of edge servers in the region, a matrix is ​​created that includes a content vector representing the cached data of each edge server. The encoded location information is input into the reinforcement learning model as part of the state representation. To enhance the consistency and compatibility of the state description, zero padding is performed to extend the matrix;

[0098] Location: A new element called location is introduced, which enhances the flexibility of decision-making and improves the accessibility of content. When the client initiates a request, the present invention independently selects the local edge server or the corresponding assisting edge server as the ideal data repository. The collaboration between the local edge server and the assisting edge server can effectively solve the problem of inconsistent number of simultaneous requests to different edge servers in the same region, thereby evenly distributing resource cache requirements among regional edge servers;

[0099] Action selection: The action vector is important in caching decisions and affects the appearance and location of cached data in the edge server. There are three different possibilities in the action space. When the action vector is 0, any cache operation on the current request data is retained, and when the action vector is 1, the current request data is cached to the original local. However, when the action vector is -1, the current request data is cached in the collaborative edge server identified by the location element. When the cache space is insufficient to satisfy the incoming data request, the old data is replaced by the LRU algorithm to free up cache space for caching; the model diagram of communication and cache decision based on multi-agent reinforcement learning can be seen in Figure 3 .

[0100] Reward Mechanism: The reward function is designed with content transmission delay and cache space in mind. Cache location has a significant impact, especially on the transmission delay when retrieving requested content. If the local edge server has already cached the content, , denoted as , then it is responsible for transmitting the content to the client. In this case, the first content transmission delay calculation process has the following relationship:

[0101] ;

[0102] in, Indicates the current transmission rate between the edge server and the client. Indicates the size of the transferred data. If the content is found is cached in the collaborative edge server and is denoted as , then collaborative transmission occurs. The collaborative edge server is responsible for transferring the content to the local edge server, which then takes on the responsibility of delivering the content Therefore, the second content transmission delay calculation process has the following relationship:

[0103] ;

[0104] in, Indicates the transmission rate between the current edge server and the collaborative edge server. , denoted as , then the central server assumes responsibility and Directly pass it to the client to ensure the accessibility of the content. Therefore, the third content transmission delay calculation process has the following relationship:

[0105] ;

[0106] in, Represents the transmission rate between the central server and the local edge server. In order to optimize the use of cache space, the remaining cache space of the edge server is also considered. , and its calculation formula is as follows:

[0107] ;

[0108] in, Indicates the total cache size of the current edge server. Indicates the cache data size of the current edge server. In order to clearly distinguish the content delivery delay under different conditions, the reward , indicating the edge server In time period Successfully retrieved content The rewards obtained are as follows:

[0109] ;

[0110] in, Represents the reward value, represents the trade-off between transmission delay and cache space utilization, Indicates the remaining cache space of the edge server. Indicates the current edge server. represents the collaborative edge server, Indicates the first content transmission delay, Indicates the second content transmission delay, Indicates the third content transmission delay, Indicates the content to be cached. Indicates that the current edge server has cached the content. Indicates that the content to be cached is cached in the collaborative edge server. Represents the base of natural logarithms. The reward function for all edge servers in the region is Calculated as:

[0111] ;

[0112] in, represents the reward value of all edge servers in the region, Represents the set of all edge servers in the region, Indicates the amount of content requested from the client.

[0113] Objective function: To maximize the overall reward value in the region, the objective function is defined as:

[0114] ;

[0115] in, represents the objective function, represents the expected value of the maximum long-term return, represents the maximum moment in a complete training process, represents the discount factor. The optimal collaborative caching strategy that maximizes can be expressed as:

[0116] ;

[0117] in, Represents the optimal collaborative caching strategy.

[0118] Step 5: The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management according to the cache decision.

[0119] Please refer to Figure 5 This embodiment further provides a multi-agent collaborative edge caching system based on a gated communication mechanism, wherein the system applies the multi-agent collaborative edge caching method based on a gated communication mechanism as described above, and the system includes:

[0120] User side, used for:

[0121] The client sends the request information to the nearest edge server based on its location;

[0122] Edge side, used for:

[0123] The nearest edge server confirms whether the cache hits based on the description information. If not, it sends a cache request to the adjacent edge server or central server based on the current edge server's own cache status and network load;

[0124] An intelligent agent is deployed for each edge server, and the intelligent agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request;

[0125] The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making;

[0126] Cloud for:

[0127] After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal caching decision through the reinforcement learning algorithm, and sends the optimal caching decision to the corresponding edge server;

[0128] The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management based on the cache decision.

[0129] The present invention deploys agents in multiple edge servers, and then uses a multi-agent reinforcement learning model with a gating model to effectively coordinate and schedule edge servers in the region, thereby improving the cache hit rate of client request content and reducing user access latency. The key point is that this method can automatically determine the optimal location of the collaborative edge server, allowing the edge server to dynamically adjust cache allocation and data transmission in real time according to the current state of the network. The gated communication mechanism enables selective exchange of information between servers, reducing the waste of bandwidth resources. Therefore, the present invention has better performance in improving cache hit rate and reducing return traffic costs.

[0130] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0131] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0133] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-agent collaborative edge caching method based on a gated communication mechanism, characterized in that: The method comprises the following steps: Step 1: The client sends the request information to the nearest edge server based on its location; Step 2: The nearest edge server confirms whether the cache hits according to the description information. If not, it sends a cache request to the adjacent edge server or central server according to the current edge server's own cache status and network load; Step 3: deploy an agent for each edge server, and the agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request; The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making; Step 4: After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal cache decision through the reinforcement learning algorithm, and sends the optimal cache decision to the corresponding edge server; Step 5: The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management according to the cache decision.

2. The multi-agent collaborative edge caching method based on the gated communication mechanism according to claim 1 is characterized in that: In step 3, when judging the edge server cache status using the gating model, the current edge server performs comprehensive analysis and judgment based on the current network load and the current cache status; other edge servers perform analysis and judgment based on the corresponding cache status.

3. The multi-agent collaborative edge caching method based on the gated communication mechanism according to claim 2 is characterized in that: Other edge servers analyze and judge according to the corresponding cache status, including the following steps: Obtain the absolute location of each edge server and perform location encoding to obtain the encoded location information. The corresponding process has the following relationship: ； in, Indicates j The absolute location of the edge servers, represents the first linear layer, Indicates j The encoded location information of the edge servers; Input the cache state of each edge server into the LSTM model to obtain the hidden state; The hidden state and the encoded position information are input into the gating model to obtain a binary output. The corresponding process has the following relationship: ； in, represents the gating model, Indicates the hidden state, Indicates binary output; like If is 0, the current LSTM model does not share self-description state information with other LSTM models at the next moment. , the current LSTM model shares the self-describing state information.

4. The multi-agent collaborative edge caching method based on the gated communication mechanism according to claim 3 is characterized in that: In step 4, based on the multi-agent decision model, comprehensively considering the state information of each agent, the optimal collaborative caching strategy is determined by the reinforcement learning algorithm, which specifically includes the following steps: When the central server obtains the agent state information filtered by the gated communication mechanism, it inputs the agent state information into the LSTM model to obtain the next layer of hidden state. The corresponding process has the following relationship: ； in, represents the hidden state of the next layer, Represents the input data of the model at time t+1, that is, the description information of some edge servers received by the central server, represents the second linear layer, Indicates the cell status, Represents the input data of the model at time t; Add the binary output to the weight of the next hidden state, and after a dot product operation, get the next layer of cell state. The corresponding process has the following relationship: ； in, represents the absolute value of the number of all agents, represents the linear transformation matrix that transforms the gated average hidden state into the communication tensor, Indicates the state of the next layer of cells; The hidden state of the next layer and the cell state of the next layer are input into the third linear layer, and then the action vector and position vector are obtained through the softmax function. The corresponding process has the following relationship: ； in, represents the third linear layer, represents the activation function, represents the position vector, represents the action vector; According to the position vector, the corresponding action vector is sent to the corresponding edge server; After receiving the action vector, the edge server performs corresponding cache management operations according to the action vector, and gives corresponding reward values ​​according to the content transmission delay and cache space utilization during the cache management operation; The action vector is optimized by maximizing the overall reward value to obtain the optimal collaborative caching strategy.

5. The multi-agent collaborative edge caching method based on the gated communication mechanism according to claim 4 is characterized in that: The corresponding cache management operations according to the action vector include: When the action vector is 0, the current edge server retains any cache operations on the current request data; When the action vector is 1, the current request data is cached locally on the current edge server; When the action vector is -1, the nearest edge server is determined as the assisting edge server according to the position vector. The collaborative edge server caches the current request data. When the cache space is insufficient to satisfy the incoming data request, the old data is replaced by the LRU algorithm to free up cache space for caching.

6. The multi-agent collaborative edge caching method based on gated communication mechanism according to claim 5 is characterized in that: Optimizing the action vector by maximizing the overall reward value to obtain the optimal collaborative caching strategy includes the following steps: Calculate the reward value of all edge servers in the region. The corresponding process has the following relationship: ； in, represents the reward value of all edge servers in the region, Represents the set of all edge servers in the region, Indicates the amount of content requested from the client. Indicates the reward value; The objective function of maximizing the overall reward value is constructed based on the reward values ​​of all edge servers. The corresponding process has the following relationship: ； in, represents the objective function, represents the expected value of the maximum long-term return, represents the maximum moment in a complete training process, represents the discount factor; Maximize the objective function to obtain the optimal collaborative caching strategy. The corresponding process has the following relationship: ； in, Represents the optimal collaborative caching strategy.

7. The multi-agent collaborative edge caching method based on gated communication mechanism according to claim 6 is characterized in that: According to the content transmission delay and cache space utilization during the cache management operation, the corresponding reward value is given by the following relationship: ； in, represents the trade-off between transmission delay and cache space utilization, Indicates the remaining cache space of the edge server. Indicates the current edge server. represents the collaborative edge server, Indicates the first content transmission delay, Indicates the second content transmission delay, Indicates the third content transmission delay, Indicates the content to be cached. Indicates that the current edge server has cached the content. Indicates that the content to be cached is cached in the collaborative edge server. Represents the base of natural logarithms.

8. The multi-agent collaborative edge caching method based on gated communication mechanism according to claim 7, characterized in that: The first content transmission delay calculation process has the following relationship: ； in, Indicates the current transmission rate between the edge server and the client. Indicates the size of the transmitted data; The second content transmission delay calculation process has the following relationship: ； in, Indicates the transmission rate between the current edge server and the collaborative edge server. The third content transmission delay calculation process has the following relationship: ； in, represents the transmission rate between the central server and the local edge server; The calculation process of the remaining cache space of the edge server has the following relationship: ； in, Indicates the total cache size of the current edge server. Indicates the current cache data size of the edge server.

9. The multi-agent collaborative edge caching method based on gated communication mechanism according to claim 8, characterized in that: The transmission rate calculation process between the central server and the local edge server has the following relationship: ； in, Indicates the transmission power level used by the central server. represents the noise power, represents the channel gain; The transmission rate calculation process between the current edge server and the collaborative edge server has the following relationship: ； in, Indicates the available bandwidth between the current edge server and the collaborative edge server. Indicates the transmit power level used by the collaborative edge server; The channel gain calculation process has the following relationship: ； in, Indicates the distance between two devices. represents the large-scale fading effect, represents the small-scale fading effect, Represents edge server i The surrounding client collection, , Represents edge server i The Mth client around, Indicates the number of clients; Represents the set of all edge servers in the region, , Indicates the area N The cache capacity of the edge servers, Indicates the total number of edge servers; Indicates a central server.

10. A multi-agent collaborative edge caching system based on a gated communication mechanism, characterized in that: The system applies the multi-agent collaborative edge caching method based on a gated communication mechanism as described in any one of claims 1 to 9, and the system comprises: User side, used for: The client sends the request information to the nearest edge server based on its location; Edge side, used for: The nearest edge server confirms whether the cache hits based on the description information. If not, it sends a cache request to the adjacent edge server or central server based on the current edge server's own cache status and network load; An intelligent agent is deployed for each edge server, and the intelligent agent generates a corresponding cache management strategy for the edge server. When the adjacent edge server or central server receives a cache request; The gated model is used to judge the cache status of the edge server. Based on the judgment result, it is decided whether to upload the corresponding agent status information to the central server to participate in the global decision-making; Cloud for: After receiving the agent status information, the central server comprehensively considers the status information of each agent based on the multi-agent decision model, determines the optimal caching decision through the reinforcement learning algorithm, and sends the optimal caching decision to the corresponding edge server; The edge server receives the optimal cache decision, transmits the file data to the client, and performs cache management based on the cache decision.

Images