A distributed two-level cache optimization method for metaverse scenarios combining MADDPG and GCN

Abstract

The present invention proposes a distributed two-level cache optimization method for a metaverse scene by combining MADDPG and GCN. The method comprises: dividing the metaverse scene in horizontal and vertical directions to obtain a preset number of sub-scenes, organizing and compressing the resources required for rendering each sub-scene, and compressing the sub-scenes through an edge network, and calculating and obtaining the similarity of user trajectories using a fast dynamic time warping algorithm; based on the similarity of different trajectory fragments, obtaining a sub-scene popularity ranking by weighted aggregation and sorting of the number of GPU server accesses, and pre-requesting and caching those sub-scenes that are expected to be frequently accessed according to the popularity ranking of the sub-scenes, and using a two-level cache replacement framework combined with GCN and MADDPG to adjust the server's cache strategy. The present invention can realize intelligent cache allocation and adjustment according to sub-scene popularity prediction, improve cache hit rate, and reduce delayed response to user requests.

Description

Technical Field

[0001] The present invention relates to the field of metaverse edge caching, and in particular to a metaverse scenario distributed two-level cache optimization method combining MADDPG and GCN. Background Art

[0002] Currently, the Metaverse is considered to be an advanced stage of digital transformation and is seen as a vision with long-term development potential. Although the Metaverse is seen as the successor to the Internet, the realization of its full vision is still in its early stages. The main challenge in the early development of the Metaverse is how to effectively transmit and render complex 3D scenes, which requires high bandwidth and ultra-reliable low-latency communication (URLLC) to support the transmission of immersive content and real-time interaction. Especially in tactile artificial intelligence applications, the transmission rate must reach at least 1Mbit / s and the delay must not exceed 1ms. Although 5G technology has alleviated these problems to a certain extent, as the amount of data and model complexity grow exponentially, the Metaverse's requirements for the network are becoming increasingly stringent.

[0003] Trajectory prediction technology provides key support for cache optimization. By analyzing the user's historical behavior, the system can predict future access behavior and cache related content in advance, thereby improving the cache hit rate. Trajectory prediction, viewpoint prediction, and eye movement prediction technologies have been widely used in VR and AR scenarios. Clustering, as an unsupervised learning method, is often used in classification, recommendation, anomaly detection and other fields, but has not yet been used as a trajectory sequence prediction method in the edge cache field. Traditional caching strategies such as LRU and LFU are simple and easy to implement, but have limited performance when processing complex scenarios and dynamic data. With the development of deep learning and reinforcement learning, DRL is increasingly used in edge cache optimization. MADDPG improves the cache hit rate by training multiple agents to work together and optimizes the cache replacement strategy in a distributed environment. However, the computational complexity of MADDPG is high, which can cause the training process to be unstable.

[0004] Although distributed edge caching improves the efficiency of resource transmission, it still has obvious limitations. In the existing distributed cloud-edge collaborative caching architecture, the edge cache is responsible for satisfying user requests nearby to reduce dependence on cloud servers, while the cloud cache is used to store global data. However, in complex and highly dynamic application scenarios such as the Metaverse, a large amount of scene data still needs to be requested from the cloud server, which increases the delay processing time and resource allocation efficiency. Summary of the invention

[0005] In view of the above situation, the main purpose of the present invention is to propose a distributed two-level cache optimization method for metaverse scenarios combining MADDPG and GCN to solve the above technical problems.

[0006] The present invention proposes a distributed two-level cache optimization method for a metaverse scenario combining MADDPG and GCN, the method comprising the following steps:

[0007] Step 1: Divide the metaverse scene horizontally and vertically to obtain a preset number of sub-scenes, and there is at least overlap between adjacent sub-scenes to ensure a smooth transition between sub-scenes;

[0008] Organize and compress the resources required for rendering each sub-scene, and transmit and cache the sub-scene compressed packages through the edge network;

[0009] Step 2: When a user visits a metaverse scene, a sub-scene access trajectory segment is formed at a preset time interval, and the similarity of the user trajectory is calculated and obtained using a fast dynamic time warping algorithm; based on the similarity of the user trajectory, the clustering model is used to predict the next trajectory segment of the user;

[0010] Step 3: When the client predicts the sub-scenario visited by the user through the real-time trajectory prediction model, a sub-scenario probability sequence is generated, and the sub-scenario number and probability in the sub-scenario prediction probability sequence are selected and transmitted to the GPU server. The sub-scenario visit times of the GPU server and the edge server are weighted and summarized and arranged in descending order to obtain the popularity ranking of the sub-scenario;

[0011] Step 4: The client sends a cache request to the GPU server for the sub-scenes that are estimated to be frequently accessed based on the popularity ranking of the sub-scenes;

[0012] Cache space is set up on both the GPU server and the edge server, and sub-scenes are cached according to the popularity ranking of the sub-scenes. The cache space adopts the intelligent agent to adjust the cache strategy of the cache space to achieve dynamic cache management.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] 1. The present invention can realize intelligent cache allocation and adjustment according to the sub-scenario popularity prediction, improve the cache hit rate, and reduce the delayed response of user requests;

[0015] 2. The present invention has high real-time performance and accuracy in similarity analysis of user trajectories, thereby effectively optimizing sub-scene resource allocation, significantly reducing transmission delays, and providing users with a smoother virtual experience;

[0016] 3. The present invention adds cache replacement penalties in GPU servers and edge servers to optimize the cache strategy of cache hits, reduce frequent cache updates, effectively reduce dependence on cloud servers, reduce network latency, and improve the overall stability of the system. It is suitable for metaverse application scenarios with high real-time requirements, and provides a solid foundation and technical support for the future expansion of the generative metaverse.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the cache optimization steps of a distributed two-level cache optimization method for a metaverse scenario that combines MADDPG and GCN proposed in the present invention.

[0019] Figure 2 This is a schematic diagram of the overall solution for Metaverse scene transmission in a Metaverse scene distributed two-level cache optimization method that combines MADDPG and GCN proposed in the present invention.

[0020] Figure 3 Schematic diagram of the "cloud-edge-end" edge cache network framework in the distributed two-level cache optimization method for metaverse scenarios that combines MADDPG and GCN proposed in the present invention.

[0021] Figure 4 This is a schematic diagram of a two-level cache optimization scheme in a distributed two-level cache optimization method for a metaverse scenario that combines MADDPG and GCN proposed in the present invention.

[0022] Figure 5 Schematic diagram of real-time trajectory segment prediction proposed by the distributed two-level cache optimization method for metaverse scenarios combining MADDPG and GCN proposed in the present invention. DETAILED DESCRIPTION

[0023] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout are the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0024] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to provide some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0025] See also Figure 1 The embodiment of the present invention proposes a distributed two-level cache optimization method for a metaverse scenario combining MADDPG and GCN, the method comprising the following steps:

[0026] Step 1: Divide the metaverse scene horizontally and vertically to obtain a preset number of sub-scenes, and there is at least overlap between adjacent sub-scenes to ensure a smooth transition between sub-scenes;

[0027] Organize and compress the resources required for rendering each sub-scene, and transmit and cache the sub-scene compressed packages through the edge network;

[0028] In order to ensure that users can seamlessly enter the next exhibition area and obtain a high-quality virtual experience, the present invention proposes an overall solution for Metaverse scene transmission, relying on distributed rendering and edge computing technology to reduce the load of centralized cloud rendering; its purpose is to provide clients with stable Metaverse scene rendering services through a distributed rendering solution of multiple GPU servers, alleviating the pressure of cloud rendering. The solution includes the following aspects:

[0029] On the premise of ensuring the continuity of the scene, the metaverse scene is divided into multiple sub-scenes. Each sub-scene overlaps with the adjacent sub-scene by about 30% to ensure smooth scene transition. The scene is divided mainly in the horizontal and vertical directions without cutting in the depth direction. This division strategy not only facilitates the management of scene resources, but also provides convenience for the rendering process;

[0030] Organize and compress the rendering resources required for each sub-scene, upload them to the cloud server, and then transmit and cache them through edge devices;

[0031] According to the access path of nearby users, the GPU server requests the corresponding resource compression package from the edge server and transmits it to the client for Metaverse sub-scene rendering;

[0032] On the client side, the received resource compression package is rendered as a 3D sub-scene to ensure that when the user enters a new scene, there is a prepared 3D environment, thereby reducing the time the user waits for rendering; through VR devices, users can smoothly immerse themselves in 360° panoramic videos; in terms of visual performance, a slight distant view fog effect and depth of field blur in non-focus areas are added to ensure a smooth transition of unrendered or less detailed distant views, further enhancing the viewing experience.

[0033] Step 2: When a user visits a metaverse scene, a sub-scene access trajectory segment is formed at a preset time interval, and the similarity of the user trajectory is calculated and obtained using a fast dynamic time warping algorithm; based on the similarity of the user trajectory, the clustering model is used to predict the next trajectory segment of the user;

[0034] In order to evaluate the similarity of sub-scene sequences of different lengths, the present invention introduces a fast dynamic time warping algorithm for identifying the similarity of sub-scene sequences of different lengths, and in order to improve the clustering accuracy, a combination of a custom distance function and a fast dynamic time warping algorithm is used for measurement. When calculating the similarity of trajectory segments, in addition to the matching of spatial positions, other behavioral factors are comprehensively considered, such as the direction when leaving the sub-scene and the dwell time. Therefore, a defined custom distance function can comprehensively measure the spatial position, trajectory direction, and dwell time, thereby more accurately measuring the similarity of the trajectory, and the matching accuracy can be effectively improved through the custom distance function.

[0035] Step 3: When the client predicts the sub-scenario visited by the user through the real-time trajectory prediction model, a sub-scenario probability sequence is generated, and the sub-scenario number and probability in the sub-scenario prediction probability sequence are selected and transmitted to the GPU server. The sub-scenario visit times of the GPU server and the edge server are weighted and summarized and arranged in descending order to obtain the popularity ranking of the sub-scenario;

[0036] The number of visits to the GPU server sub-scenario is weighted and summarized and arranged in descending order to obtain the popularity ranking of the sub-scenario. The corresponding relationship is:

[0037] ;

[0038] in, For sub-scene The number of times the edge server is requested within a preset time interval, For sub-scene The number of requests made to the edge server at a preset time;

[0039] In the step of obtaining the popularity ranking of the sub-scenario, the corresponding process has the following relation:

[0040] ;

[0041] in, The popularity ranking of GPU server sub-scenario, For GPU server statistics matrix, To sort in descending order, is the sorting function;

[0042] Based on the popularity ranking of sub-scenario, the popularity ranking of each sub-scenario is obtained, and the relationship between the corresponding process is:

[0043] ;

[0044] in, is the popularity ranking of edge server sub-scenario, For sub-scene The number of times the edge server is requested during the time interval, Number the popularity ranking of the sub-scene.

[0045] Please refer to Figures 3 to 5 ,This paper proposes a "cloud-edge-end" sub-scene edge cache framework to optimize the processing ,and distribution of scene data ,and achieve efficient transmission and processing ,of complex scenes. All levels of the framework work together ,to provide support for efficient data management and scene rendering.

[0046] The cloud layer, as the network hub, is responsible for overall management and provides functions such as application center, long-term storage, and real-time data synchronization through storage servers and synchronization servers.

[0047] At the edge layer, the edge server collaborates with the GPU server it manages to complete the sub-scenario real-time caching solution; based on the GPU server's summary of the client's prediction results, the real-time popularity of the sub-scenario is calculated separately, and the sub-scenario with the most user requests is cached first; in the edge server, the sub-scenario that is not hit on any GPU server is cached first. If the edge server does not hit, it requests the sub-scenario data from the cloud service to meet the user's real-time needs;

[0048] At the terminal layer, GPU servers are configured in the request-intensive areas to collect, encrypt, render and cache sub-scene data. In the two-level cache optimization solution, multiple GPU servers form a MADDPG cache strategy to give priority to caching the sub-scenes with the highest real-time popularity.

[0049] If the local GPU server does not hit, it will first request sub-scene data from other nearby GPU servers; if all GPU servers do not have cache, it will request resources from the edge server;

[0050] In areas where requests are not dense, the client directly requests the sub-scene from the edge server and renders it locally. This not only takes into account factors such as cost and network connection, but also ensures full utilization of resources, providing an efficient and economical solution.

[0051] In order to optimize the caching strategy and improve system performance, it is crucial to calculate the popularity of sub-scenes. The popularity calculation helps to identify the most frequently visited sub-scenes, so that these scenes can be cached first, improving the cache hit rate, reducing data transmission latency, and thus improving user experience.

[0052] Step 4: The client sends a cache request to the GPU server for the sub-scenes that are estimated to be frequently accessed based on the popularity ranking of the sub-scenes;

[0053] Cache space is set up on both the GPU server and the edge server, and sub-scenes are cached according to the popularity ranking of the sub-scenes. The cache space uses the intelligent agent to adjust the cache space cache strategy to achieve dynamic cache management;

[0054] The cache strategy of the agent to adjust the cache space includes multi-agent training on GPU servers and single-agent training on edge servers;

[0055] The multi-agent training of GPU server specifically includes the following steps:

[0056] The client sends cache requests to the GPU server for sub-scenes that are estimated to be frequently accessed based on the popularity ranking of the sub-scenes. The GPU server performs real-time cache according to the received cache requests and calculates and generates the status of the GPU server at the same time.

[0057] The GPU server has a binary decision operation corresponding to the currently generated state, and generates a cache replacement action through the binary decision operation;

[0058] The GPU server replaces the cached sub-scene according to the cache replacement action, calculates the cache hit rate of the replaced cache, and rewards the GPU server according to the calculated change in the GPU server cache hit rate;

[0059] The GPU server calculates the replacement penalty based on the impact of the cache replacement action on the GPU server cache hit rate;

[0060] The GPU server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the experience replay buffer in the form of a four-tuple;

[0061] The GPU server uses a policy network and value network architecture. The value network calculates the change of the target value based on the experience playback of the GPU server to evaluate the current policy.

[0062] Single agent training specifically includes the following steps:

[0063] The edge server calculates and generates the state of the edge server based on the cache requests that the GPU server has not received;

[0064] The edge server has a binary selection operation corresponding to the currently generated state, and generates a cache replacement action through the binary selection operation;

[0065] The edge server calculates the sub-scenario popularity for requests that do not hit the sub-scenario cache of the GPU server based on the cache replacement action, and calculates the hit rate of the replaced cache, and rewards the edge server based on the calculated change in the edge server cache hit rate;

[0066] The edge server calculates the replacement penalty based on the impact of the cache replacement action on the edge server hit rate;

[0067] The edge server stores the current state, the selected cache replacement action, the reward obtained by performing the cache replacement action, and the new state after obtaining the reward in the experience replay buffer in the form of a four-tuple;

[0068] The edge server adopts the policy network and value network architecture. The value network calculates the change of the target value based on the experience playback of the edge server to evaluate the current policy;

[0069] In the step of calculating and generating the state of the GPU server, the calculation process of the GPU server state corresponds to the following relationship:

[0070] ;

[0071] in, Indicates the GPU server status. Indicates at time Moment GPU server cached sub-scenes, Indicates the popularity of each sub-scenario of the current GPU server;

[0072] In the step where the edge server calculates and generates the state of the edge server according to the cache request that the GPU server has not received, the calculation process of the edge server state corresponds to the following relationship:

[0073] ;

[0074] in, Indicates the edge server status. Indicates the cache status of the edge server;

[0075] In the step where the GPU server has a binary decision operation corresponding to the currently generated state, the calculation process of the binary decision of the GPU server corresponds to the following relationship:

[0076] ;

[0077] in, represents a binary decision, Indicates GPU server Perform cache replacement. Indicates that cache replacement is not performed;

[0078] In the step where the edge server has a binary selection operation corresponding to the currently generated state, the binary selection calculation process of the edge server corresponds to the following relationship:

[0079] ;

[0080] in, represents a binary choice, Indicates that the edge server performs cache replacement based on the GPU server's miss request, giving priority to the cache miss sub-scenario. Indicates that cache replacement is not performed;

[0081] In the step of generating a cache replacement action by a binary selection operation, the calculation process corresponding to the cache replacement of the GPU server corresponds to the following relationship:

[0082] ;

[0083] in, For cache replacement actions, The policy output by the policy network;

[0084] In the step of generating a cache replacement action by a binary selection operation, the calculation process of the cache replacement action of the edge server corresponds to the following relationship:

[0085] ;

[0086] in, is the replacement action of the edge server, The policy exported by the edge server policy network, Indicates The exploration probability during training.

[0087] In the step of generating a cache replacement action by a binary decision operation, the method further comprises:

[0088] The exploration probability is calculated using a dynamic greedy strategy, and action selection is performed based on the calculated exploration probability;

[0089] Among them, the calculation process of the dynamic greedy strategy corresponds to the following relationship:

[0090] ;

[0091] in, represents the initial exploration probability, represents the minimum exploration probability, represents the attenuation coefficient;

[0092] In the step where the GPU server calculates the replacement penalty according to the impact of the cache replacement action on the cache hit rate of the GPU server, the penalty mechanism calculation process of the GPU server corresponds to the following relationship:

[0093] ;

[0094] in, Indicates the average cache hit rate of the GPU server, Indicates the client request Sub-scenes, represents the indicator function, Indicates the GPU server cache content. Indicates the number of client requests within a preset time;

[0095] In the step where the edge server calculates the replacement penalty according to the impact of the cache replacement action on the edge server hit rate, the penalty mechanism calculation process of the edge server corresponds to the following relationship:

[0096] ;

[0097] in, represents the cache hit rate of the edge server, Indicates the number of missed requests forwarded to the edge server. Indicates the edge server processes Requests, Indicates whether the request hits the server edge Indicates that the edge server caches content.

[0098] In the step of rewarding the GPU server according to the calculated change in the GPU server cache hit rate, the GPU server reward calculation process corresponds to the following relationship:

[0099] ;

[0100] in, Indicates whether the requested content hits the GPU server near the client. Indicates whether other GPU servers under the same edge server are hit. Indicates whether the request content hits the edge server. Whether the requested content is matched in the cloud server. Indicates whether the requested content hits the GPU server near the client. Indicates whether other GPU servers under the same edge server hit the reward. Indicates the reward for whether the requested content hits the edge server. Indicates whether the requested content hits the cloud server. Indicates the change in cache hit rate. Represents GPU server rewards, represents the replacement penalty coefficient of the GPU server, Indicates that the preset constraints are met.

[0101] In the step of rewarding the edge server according to the calculated change in the cache hit rate of the edge server, the reward calculation process of the edge server corresponds to the following relationship:

[0102] ;

[0103] in, is the edge server reward function, is the cache hit rate change of the edge server, is the replacement penalty coefficient of the edge server.

[0104] In the step where the GPU server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the form of a quadruple in the experience replay buffer, the experience replay calculation process of the GPU server corresponds to the following relationship:

[0105] ;

[0106] in, is the target value of the GPU server, Rewards for GPU servers, is the discount factor, is the output value of the target value network of the GPU server in the next stage, The cache status of each GPU server in the next stage, New actions for GPU servers, For the given conditions, Output policy for the next stage of the edge server policy network, The local state observed by the GPU server.

[0107] In the step where the edge server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the form of a four-tuple in the experience replay buffer, the experience replay calculation process of the edge server corresponds to the following relationship:

[0108] ;

[0109] in, is the edge server target value, Reward for edge servers, is the output of the target value network, is the cache status of the edge server in the next stage. Output policies for the next stage of the edge server policy network;

[0110] In the step where the value network calculates the change of the target value based on the experience playback of the GPU server, the target value network calculation process of the GPU server corresponds to the following relationship:

[0111] ;

[0112] in, Indicates The loss function of a GPU server is represents the parameters of the value network, Indicates that the GPU server is running at the preset time. The number of samples in represents the index of the sample, Indicates the GPU server target value, Indicates The value network of GPU servers, Indicates the cache status of each GPU server. Indicates GPU servers in the sample action.

[0113] In the step where the value network calculates the change of the target value based on the experience playback of the edge server, the target value calculation process of the edge server corresponds to the following relationship:

[0114] ;

[0115] in, represents the loss function of the edge server, Indicates that The parameters of the value network, Indicates that the edge server is at the preset time The number of samples in represents the index of the sample, represents the edge server target value, Represents the edge server Value Network, Indicates the cache status of the edge server. Indicates preset time The action taken by the edge server.

[0116] Please refer to Figure 5,The client first requests and caches the sub-scenes with higher predicted access frequency based on the real-time trajectory prediction. If the GPU server misses, the request is forwarded to the edge server for caching. In order to effectively manage cache resources, the two-level cache optimization scheme introduces graph convolutional networks and cache replacement penalty mechanisms in the cache replacement framework to accurately extract sub-scene features, optimize cache replacement strategies, and reduce frequent cache updates. Through this collaborative approach, the two-level cache optimization scheme effectively reduces dependence on cloud servers, reduces network latency, and improves the overall stability of the system. It is suitable for metaverse application scenarios with high real-time requirements;

[0117] The two-level cache optimization solution combines GCN with the MADDPG architecture to optimize the cache replacement strategy. Graph convolutional networks are used in both GPU servers and edge servers to extract sub-scene features. On the GPU server side, the MADDPG algorithm is used. Multiple GPU servers work together as multi-agents to dynamically adjust the cache content using the sub-scene features extracted by the graph convolutional network. The graph convolutional network aggregates node features layer by layer through the graph convolution layer to help the system capture the global and local information of the sub-scene, so that each GPU server can optimize the cache strategy based on the real-time requests of the client and give priority to caching high-popularity sub-scenes.

[0118] In the evaluator architecture of MADDPG, the policy network generates cache replacement policies, and the value network evaluates the value of the current state to ensure the rationality of cache replacement decisions, ultimately improving cache hit rate and system performance. The edge server uses the deep deterministic policy gradient algorithm alone because of its different roles and tasks, thereby avoiding unnecessary agent communication and synchronization overhead and improving cache replacement efficiency. The edge server specializes in processing miss requests transmitted from the GPU server, so its cache management is relatively independent and does not need to collaborate with other agents. Compared with the complex multi-agent collaboration of the GPU server, the edge server only needs to make independent decisions based on its own cache status. Therefore, the deep deterministic policy gradient algorithm using a single agent is more suitable for this scenario. GCN is also used for sub-scenario feature extraction of the edge server, but the deep deterministic policy gradient algorithm dynamically manages cache content through an independent evaluator architecture to ensure that the edge server prioritizes caching the sub-scenario that is most likely to be requested again, thereby improving the overall cache efficiency of the system.

[0119] State, in a multi-GPU server, the state consists of the sub-scene state currently cached by each GPU server and the real-time popularity of the sub-scene; specifically, the state includes the sub-scene number cached by the current server and the access frequency of each sub-scene. In an edge server, the state consists of the cache state of the edge server and the real-time popularity of the sub-scene.

[0120] In order to enhance the exploration capability of the model, a dynamic greedy strategy is introduced in the action selection process of GPU servers and edge servers. The exploration probability will gradually decay over time. More exploration will be performed in the early stage, and the optimal action will be selected in the later stage.

[0121] Reward,In order to balance the consumption caused by cache hit rate and cache replacement,a reward function with a dynamic penalty mechanism is designed.,This function links the change of cache hit rate with the replacement,operation. When frequent replacement leads to negative impact, the reward function will automatically increase the,penalty, and vice versa. Through this mechanism, the cache replacement frequency and cache hit rate can be effectively,balanced, thereby optimizing system performance.

[0122] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0123] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0124] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A distributed two-level cache optimization method for metaverse scenarios combining MADDPG and GCN, characterized in that: The method comprises the following steps: Step 1: Divide the metaverse scene horizontally and vertically to obtain a preset number of sub-scenes, and there is at least overlap between adjacent sub-scenes to ensure a smooth transition between sub-scenes; Organize and compress the resources required for rendering each sub-scene, and transmit and cache the sub-scene compressed packages through the edge network; Step 2: When a user visits a metaverse scene, a sub-scene access trajectory segment is formed at a preset time interval, and the similarity of the user trajectory is calculated and obtained using a fast dynamic time warping algorithm; based on the similarity of the user trajectory, the clustering model is used to predict the next trajectory segment of the user; Step 3: When the client predicts the sub-scenario visited by the user through the real-time trajectory prediction model, a sub-scenario probability sequence is generated, the sub-scenario number and probability in the sub-scenario probability sequence are selected and transmitted to the GPU server, and the sub-scenario visit times of the GPU server and the edge server are weighted and summarized and arranged in descending order to obtain the popularity ranking of the sub-scenario; Step 4: The client sends a cache request to the GPU server for the sub-scenes that are estimated to be frequently accessed based on the popularity ranking of the sub-scenes; Cache space is set up on both the GPU server and the edge server, and sub-scenes are cached according to their popularity ranking. The cache space uses an intelligent agent to adjust the cache space's cache strategy to achieve dynamic cache management; In step 4, the cache strategy of the agent adjusting the cache space includes multi-agent training of the GPU server and single-agent training of the edge server; Among them, the client first requests and caches the sub-scenes with higher predicted access frequency based on the real-time trajectory prediction. If the GPU server does not hit it, it forwards the request to the edge server for caching; The two-level cache optimization solution combines GCN with the MADDPG architecture to optimize the cache replacement strategy, and uses graph convolutional networks for sub-scene feature extraction in both GPU servers and edge servers. On the GPU server side, the MADDPG algorithm is used, and multiple GPU servers work together as multi-agents.

2. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 1, characterized in that: The multi-agent training of GPU server includes the following steps: The client sends cache requests to the GPU server for sub-scenes that are estimated to be frequently accessed based on the popularity ranking of the sub-scenes. The GPU server performs real-time cache according to the received cache requests and calculates and generates the status of the GPU server at the same time. The GPU server has a binary decision operation corresponding to the currently generated state, and generates a cache replacement action through the binary decision operation; The GPU server replaces the cached sub-scene according to the cache replacement action, calculates the cache hit rate of the replaced cache, and rewards the GPU server according to the calculated change in the GPU server cache hit rate; The GPU server calculates the replacement penalty based on the impact of the cache replacement action on the GPU server cache hit rate; The GPU server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the form of a four-tuple in the experience replay buffer; The GPU server uses a policy network and value network architecture. The value network calculates the change of the target value based on the experience playback of the GPU server to evaluate the current policy. Single agent training specifically includes the following steps: The edge server calculates and generates the state of the edge server based on the cache requests that the GPU server has not received; The edge server has a binary selection operation corresponding to the currently generated state, and generates a cache replacement action through the binary selection operation; The edge server calculates the sub-scenario popularity for requests that do not hit the sub-scenario cache of the GPU server based on the cache replacement action, and calculates the hit rate of the replaced cache, and rewards the edge server based on the calculated change in the edge server cache hit rate; The edge server calculates the replacement penalty based on the impact of the cache replacement action on the edge server hit rate; The edge server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the experience replay buffer in the form of a four-tuple; The edge server adopts a policy network and value network architecture. The value network calculates the change of the target value based on the experience playback of the edge server for evaluating the current strategy.

3. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 2, characterized in that: In the step of calculating and generating the state of the GPU server, the calculation process of the GPU server state corresponds to the following relationship: ； in, Indicates the GPU server status. Indicates at time Moment GPU server cached sub-scenes, Indicates the popularity of each sub-scenario of the current GPU server; In the step where the edge server calculates and generates the state of the edge server according to the cache request that the GPU server has not received, the calculation process of the edge server state corresponds to the following relationship: ； in, Indicates the edge server status. Indicates the cache status of the edge server.

4. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 3 is characterized in that: In the step where the GPU server has a binary decision operation corresponding to the currently generated state, the calculation process of the binary decision of the GPU server corresponds to the following relationship: ； in, represents a binary decision, Indicates that the GPU server performs cache replacement. Indicates that cache replacement is not performed; In the step where the edge server has a binary selection operation corresponding to the currently generated state, the binary selection calculation process of the edge server corresponds to the following relationship: ； in, represents a binary choice, Indicates that the edge server performs cache replacement based on the GPU server's miss request. Indicates that no cache replacement is performed.

5. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 4, characterized in that: In the step of generating a cache replacement action by a binary selection operation, the calculation process corresponding to the cache replacement of the GPU server corresponds to the following relationship: ； in, For cache replacement actions, is the policy output by the policy network, Indicates The exploration probability during training; In the step of generating a cache replacement action by a binary selection operation, the calculation process of the cache replacement action of the edge server corresponds to the following relationship: ； in, is the replacement action of the edge server, Policy for the Edge Server Policy Network output.

6. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 5, characterized in that: In the step of generating a cache replacement action by a binary decision operation, the method further comprises: The exploration probability is calculated using a dynamic greedy strategy, and action selection is performed based on the calculated exploration probability; Among them, the calculation process of the dynamic greedy strategy corresponds to the following relationship: ； in, Indicates The exploration probability at training time, represents the initial exploration probability, represents the minimum exploration probability, Represents the attenuation coefficient.

7. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 6, characterized in that: In the step where the GPU server calculates the replacement penalty according to the impact of the cache replacement action on the cache hit rate of the GPU server, the penalty mechanism calculation process of the GPU server corresponds to the following relationship: ； in, Indicates the average cache hit rate of the GPU server, Indicates the client request Sub-scenes, represents the indicator function, Indicates the GPU server cache content. Indicates the number of client requests within a preset time; In the step where the edge server calculates the replacement penalty according to the impact of the cache replacement action on the edge server hit rate, the penalty mechanism calculation process of the edge server corresponds to the following relationship: ； in, represents the cache hit rate of the edge server, Indicates the number of missed requests forwarded to the edge server. Indicates the edge server processes Requests, Indicates whether the request hits the server edge Indicates that the edge server caches content.

8. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 7, characterized in that: In the step of rewarding the GPU server according to the calculated change in the GPU server cache hit rate, the GPU server reward calculation process corresponds to the following relationship: ； in, Indicates whether the requested content hits the GPU server near the client. Indicates whether other GPU servers under the same edge server are hit. Indicates whether the request content hits the edge server. Indicates whether the requested content is matched in the cloud server. Indicates whether the requested content hits the GPU server near the client. Indicates whether other GPU servers under the same edge server hit the reward. Indicates the reward for whether the requested content hits the edge server. Indicates whether the requested content hits the cloud server. Indicates the change in cache hit rate. Represents GPU server rewards, represents the replacement penalty coefficient of the GPU server, Indicates that the preset constraints are met; In the step of rewarding the edge server according to the calculated change in the cache hit rate of the edge server, the reward calculation process of the edge server corresponds to the following relationship: ； in, is the edge server reward function, is the cache hit rate change of the edge server, is the replacement penalty coefficient of the edge server.

9. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 8, characterized in that: In the step where the GPU server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the form of a quadruple in the experience replay buffer, the experience replay calculation process of the GPU server corresponds to the following relationship: ； in, is the target value of the GPU server, Rewards for GPU servers, is the discount factor, is the output value of the target value network of the GPU server in the next stage, The cache status of each GPU server in the next stage, New actions for GPU servers, For the given conditions, Output policy for the next stage of the edge server policy network, The local state observed by the GPU server; In the step where the edge server stores the current state, the selected cache replacement action, the reward obtained by executing the cache replacement action, and the new state after obtaining the reward in the form of a four-tuple in the experience replay buffer, the experience replay calculation process of the edge server corresponds to the following relationship: ； in, is the edge server target value, Reward for edge servers, is the output of the target value network, is the cache status of the edge server in the next stage. Outputs policy for the next phase of the edge server policy network.

10. The method for optimizing the metaverse scenario distributed two-level cache by combining MADDPG and GCN according to claim 9, characterized in that: In the step where the value network calculates the change of the target value based on the experience playback of the GPU server, the target value network calculation process of the GPU server corresponds to the following relationship: ； in, Indicates The loss function of a GPU server is represents the parameters of the value network, Indicates that the GPU server is running at the preset time. The number of samples in represents the index of the sample, Indicates the GPU server target value, Indicates The value network of GPU servers, Indicates the cache status of each GPU server. Indicates GPU servers in the sample Actions; In the step where the value network calculates the change of the target value based on the experience playback of the edge server, the target value calculation process of the edge server corresponds to the following relationship: ； in, represents the loss function of the edge server, Indicates that The parameters of the value network, Indicates that the edge server is at the preset time The number of samples in represents the index of the sample, represents the edge server target value, Represents the edge server Value Network, Indicates the cache status of the edge server. Indicates preset time The action taken by the edge server.

Images