Metacosm scene caching method combining DRL and AFL

Abstract

The invention provides a DRL and AFL combined meta-universe scene caching method. The method specifically comprises the steps that a meta-universe scene is segmented in a horizontal and vertical segmentation mode, and a predetermined number of sub-scenes are obtained; constructing an asynchronous federated learning training trajectory prediction model, and predicting the trajectory of the user through the asynchronous federated learning training trajectory prediction model; when a user enters a sub-scene, generating a prediction result through an asynchronous federal learning training trajectory prediction model, and generating a cache hit rate and a sub-scene probability sequence of an edge end of a target sub-scene at the same time; on the basis of an Actor-Critic framework, a cache replacement framework is formed by combining three basic elements including states, actions and rewards, so that a cache replacement strategy of an edge server is realized. The user behavior track is accurately predicted through the distributed real-time track prediction system, the sub-scene caching strategy is dynamically adjusted in combination with the Actor-Critic algorithm, and the caching hit rate of the target sub-scene is remarkably increased.

Description

Technical Field

[0001] The present invention relates to the field of metaverse and edge caching, and in particular to a metaverse scene caching method combining DRL and AFL. Background Art

[0002] Driven by significant progress in information technology, especially 5G, artificial intelligence (AI) and cloud computing, we are entering a new digital era. In this context, the Metaverse, which integrates technologies such as virtual reality (VR), augmented reality (AR) and mixed reality (MR), has become a hot topic in the technology and digital entertainment industries. However, the construction and development of the Metaverse is facing many challenges, especially in the efficient transmission and rendering of complex 3D scenes. The limitations of traditional network transmission, such as bandwidth limitations, buffering and jamming, seriously affect the user's immersion.

[0003] In addition, high latency and network instability also pose challenges to users' real-time interactive experience. In terms of data privacy and security, it is crucial to protect the security of users' personal information and virtual assets, which requires effective encryption and security measures during data transmission. At the same time, in order to achieve cross-platform compatibility and flexibility, the Metaverse scene must adapt to the performance and input methods of different devices while maintaining high quality.

[0004] So far, most metaverses use centralized data processing methods, and rendering work is completed on cloud servers. The existing 5G network often cannot meet the needs of large-scale users for real-time immersive metaverse interaction. The emergence of Neural Radiance Fields (NeRF) has realized the rendering of realistic 3D scenes from a set of sparse views, but NeRF currently faces problems such as long rendering time for 3D scenes, high rendering configuration requirements, and unsatisfactory rendering quality. Block-NeRF and Grid-NeRF have realized the rendering of sub-scenes in modules, providing a research direction for the rendering of 3D sub-scenes in modules in the metaverse. When rendering large-scale scenes, it still faces problems such as long rendering time and high equipment requirements.

[0005] Currently, the Metaverse is seen as an advanced stage of digital transformation and a long-term development vision. Although the Metaverse is seen as the successor to the Internet, the realization of the full vision is still in the early stages of development. The main challenge in the early development of the Metaverse is the efficient transmission and rendering of 3D virtual scenes. Metaverse scenes are rendered and transmitted in the form of 3D virtual scenes, which requires high bandwidth and ultra-reliable low-latency communication (URLLC) to support immersive content delivery and real-time interaction. These requirements pose significant challenges to 5G networks. Summary of the invention

[0006] In view of the above situation, the main purpose of the present invention is to propose an interpretable human brain visual semantic cognition classification method to solve the above technical problems.

[0007] The present invention proposes a metaverse scene caching method combining DRL and AFL, the method comprising the following steps: Step 1: Segment the metaverse scene horizontally and vertically to obtain a predetermined number of sub-scenes, and adjacent sub-scenes have overlapping areas to ensure continuity between sub-scenes; Organize and compress the resources required for each sub-scenario and upload them to the edge network for caching; Step 2: Build an asynchronous federated learning (AFL) training trajectory prediction model, and design a custom comprehensive loss function based on the asynchronous federated learning training trajectory prediction model to optimize the model parameters and improve the accuracy of the prediction results of the asynchronous federated learning training trajectory prediction model. The custom comprehensive loss function consists of a standard cross entropy loss and a distance penalty term. Step 3: When the user enters a sub-scenario, the trajectory prediction model is trained through asynchronous federated learning to generate prediction results, and the cache hit rate and sub-scenario probability sequence of the edge end of the target sub-scenario are generated at the same time; Evaluate the performance of the asynchronous federated learning training trajectory prediction model based on the cache hit rate at the edge of the target sub-scenario; Based on the sub-scenario probability sequence, the scene number in the sub-scenario probability sequence and the probability corresponding to the sub-scenario number are selected and uploaded to the edge server through the GPU server; The prediction results generated by the asynchronous federated learning training trajectory prediction model are accumulated through the edge server, and the popularity of the sub-scenario is obtained by sorting the prediction results. Step 4: Combine the cache state in the edge server and the prediction matrix of the edge server sub-scenario to obtain the state; Combined with deep learning (DRL), the cache hit rate and transmission delay at the edge of the target sub-scenario are used for weighted summation to obtain rewards; Based on the Actor-Critic framework, a cache replacement framework is constructed by combining the three basic elements of state, action and reward to implement the cache replacement strategy of the edge server.

[0008] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention uses a distributed real-time trajectory prediction system to accurately predict user behavior trajectories, and dynamically adjusts the sub-scene cache strategy in combination with the Actor-Critic algorithm, significantly improving the cache hit rate of the target sub-scene; 2. The present invention calculates and predicts popularity on the edge server, optimizes the cache strategy, reduces the data transmission path and time, reduces the transmission delay, and meets the requirements of high immersion and low delay in the metaverse scene; 3. The present invention adopts asynchronous federated learning technology to achieve efficient user behavior prediction and ensure user information security while protecting user data privacy; 4. The present invention dynamically adjusts the cache strategy, reasonably allocates network resources, improves the overall efficiency of the cache system, and avoids resource waste.

[0009] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a metaverse scene caching method combining DRL and AFL proposed by the present invention; Figure 2 A schematic diagram of the overall solution of Metaverse scene transmission of a Metaverse scene caching method combining DRL and AFL proposed by the present invention; Figure 3 A schematic diagram of an edge cache network framework of a metaverse scene caching method combining DRL and AFL proposed in the present invention; Figure 4 A schematic diagram of the data transmission process of a metaverse scene caching method combining DRL and AFL proposed in the present invention; Figure 5 A schematic diagram of a cache optimization solution for a metaverse scene cache method combining DRL and AFL proposed by the present invention; Figure 6 A schematic diagram of an asynchronous federated learning algorithm for a metaverse scene caching method combining DRL and AFL proposed in the present invention; Figure 7 Schematic diagram of the Bi-LSTM neural network structure of the metaverse scene caching method combining DRL and AFL proposed in the present invention. DETAILED DESCRIPTION

[0011] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0012] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0013] See also Figure 1 , an embodiment of the present invention proposes a metaverse scene caching method combining DRL and AFL, the method comprising the following steps: Step 1: Segment the metaverse scene horizontally and vertically to obtain a predetermined number of sub-scenes, and adjacent sub-scenes have overlapping areas to ensure continuity between sub-scenes; Organize and compress the resources required for each sub-scenario and upload them to the edge network for caching; Please refer to Figure 2 ,Furthermore, adjacent sub-scenes have 30% overlapping areas to ensure smooth transition between ,sub-scenes.,The division strategy in the horizontal and vertical directions not only facilitates ,the efficient management of scene resources but also facilitates ,rendering work.

[0014] Step 2: Build an asynchronous federated learning training trajectory prediction model, and design a custom comprehensive loss function based on the asynchronous federated learning training trajectory prediction model to optimize the model parameters and improve the accuracy of the prediction results of the asynchronous federated learning training trajectory prediction model. The custom comprehensive loss function consists of a standard cross entropy loss and a distance penalty term. Please refer to Figure 5 In step 2, the custom comprehensive loss function is composed of the standard cross entropy loss and the distance penalty term, where the expression of the custom comprehensive loss function is: ; in, represents a custom comprehensive loss function, represents the penalty weight, Indicates The actual coordinates of the sub-scenes, Indicates The predicted coordinates of each sub-scene, Indicates The Euclidean distance between the actual coordinates and the predicted coordinates of each sub-scene, represents the total number of predicted sub-scenarios, represents the cross entropy loss function, Indicates The model after local training iterations, represents the number of learning rounds, Indicates the number of local iterations; The custom comprehensive loss function consists of the standard cross entropy loss and the distance penalty term, where the expression of the standard cross entropy loss is: ; in, Indicates the sub-scene number. represents the total number of training data, Indicates GPU servers, Indicates belonging to The probability of represents training data; Furthermore, this comprehensive loss function not only considers the classification accuracy, but also penalizes the deviation of spatial prediction, which makes the model training more accurate. Through this custom loss function, the consideration of classification and spatial information is balanced, thus significantly improving the overall performance of the model; For further information, please refer to Figure 7 , Bi-LSTM is selected as the sequence modeling tool. The bidirectional structure of Bi-LSTM can fully capture the relationship between temporal dependencies and sub-scenarios, thereby improving the model's ability to model complex time series and accurately predict user behavior trajectories; For further information, please refer to Figure 6 ,The asynchronous federated learning trajectory prediction strategy mainly includes the following steps: a. The edge server trains a global Bi-LSTM model to predict the next sub-scenario that the user is about to enter. The global model is downloaded to the terminal device or transmitted to the terminal device via the GPU server for user trajectory prediction and local training of the model; b. The global model downloaded by the local GPU server is set as the initial local model, and the local model is updated through iterative training. In each iteration, the GPU server randomly selects part of the user trajectory data for training, aiming to predict the next scene that the user will visit by analyzing the access sequence of the metaverse sub-scenes. The updated local model will be uploaded to the edge server for aggregation; c. After completing the prediction model training on the local GPU server, upload the training results to the edge server for aggregation. In order to reduce communication costs and enhance efficiency, choose to upload parameter changes instead of the entire model to the edge server for aggregation; d. Merge local update results from different GPU servers to generate a comprehensive and accurate global model to improve the overall prediction performance.

[0015] Step 3: When the user enters a sub-scenario, the trajectory prediction model is trained through asynchronous federated learning to generate prediction results, and the cache hit rate and sub-scenario probability sequence of the edge end of the target sub-scenario are generated at the same time; Evaluate the performance of the asynchronous federated learning training trajectory prediction model based on the cache hit rate at the edge of the target sub-scenario; Based on the sub-scenario probability sequence, the scene number in the sub-scenario probability sequence and the probability corresponding to the sub-scenario number are selected and uploaded to the edge server through the GPU server; The prediction results generated by the asynchronous federated learning training trajectory prediction model are accumulated through the edge server, and the popularity of the sub-scenario is obtained by sorting the prediction results. In step 3, when the user enters a sub-scenario, the trajectory prediction model is trained through asynchronous federated learning to generate prediction results, and the cache hit rate and sub-scenario probability sequence of the edge of the target sub-scenario are generated at the same time. The calculation relationship of the cache hit rate of the edge of the target sub-scenario is: ; in, represents the indicator function, Indicates the total number of times users log into the Metaverse system. represents the cache hit rate at the edge of the target sub-scenario, Indicates the number of times a user logs into the Metaverse system; The prediction results generated by the asynchronous federated learning training trajectory prediction model are accumulated by the edge server, and the popularity of the sub-scenario is obtained by sorting the prediction results. The corresponding relationship in the process is: ; in, Indicates a point in time, Indicates the number of sub-scenes, Indicates at a point in time No. The access weight of each sub-scenario, express The prediction matrix of the edge server sub-scenario at the moment, represents an edge server; In the step of obtaining the popularity of sub-scenario by sorting the prediction results, the relationship between the corresponding process is: ; in, express Sub-scenario prediction matrix of sub-scenario summary at each moment; Furthermore, in order to accurately reflect the popularity of sub-scenarios in the metaverse, the predicted probability of each sub-scenario in the probability sequence is used as the weight of the corresponding sub-scenario. The uploaded data generated after each prediction can be regarded as a vector, and the weight vector calculated after each prediction will be appended to the matrix in the edge server to record the historical data received by the edge server.

[0016] Step 4: Combine the cache state in the edge server and the prediction matrix of the edge server sub-scenario to obtain the state; Combined with deep learning, the cache hit rate and transmission delay at the edge of the target sub-scenario are used for weighted summation to obtain rewards; Based on the Actor-Critic framework, the cache replacement framework is constructed by combining the three basic elements of state, action and reward to implement the cache replacement strategy of the edge server; Please refer to Figure 4 In step 4, the cache state in the edge server and the prediction matrix of the edge server sub-scenario are combined to obtain the state. The relationship between the corresponding process is: ; in, Indicates the status, Indicates the cache status in the edge server. represents an edge server; The reward is obtained by weighted summing the cache hit rate and transmission delay at the edge of the target sub-scenario. The corresponding relationship is: ; in, and Both represent weights, represents a natural constant, Indicated in The transmission delay of the time, Indicates action; Based on the Actor-Critic framework, the cache replacement framework is composed of three basic elements: state, action, and reward. The expression of the action is: ; in, Represents a sub-scene, Indicates whether the sub-scenario performs cache replacement on the edge server. Represents a collection of subscenes.

[0017] Please refer to Figure 3 Furthermore, the framework proposed by the present invention can provide efficient data management and scene rendering services: Cloud layer: As the network hub, it is responsible for comprehensive management and provides functions such as application center, long-term storage, and real-time data synchronization through storage servers and synchronization servers; Edge layer: The edge server collaborates with the neighboring GPU server to train the prediction model using asynchronous federated learning and calculate the popularity of the scene based on historical data. The TAC solution is used to determine the sub-scene resource compression packages that should be cached first to meet the user's immediate needs; Terminal layer: GPU servers are configured near terminal devices to be responsible for data collection, request encryption, and local computing, and to assist nearby edge servers in performing local asynchronous federated learning to achieve personalized data services that protect user data privacy.

[0018] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0019] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0020] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A metaverse scene caching method combining DRL and AFL, characterized in that: The method comprises the following steps: Step 1: Segment the metaverse scene horizontally and vertically to obtain a predetermined number of sub-scenes, and adjacent sub-scenes have overlapping areas to ensure continuity between sub-scenes; Organize and compress the resources required for each sub-scenario and upload them to the edge network for caching; Step 2: Build an asynchronous federated learning training trajectory prediction model, and design a custom comprehensive loss function based on the asynchronous federated learning training trajectory prediction model to optimize the model parameters and improve the accuracy of the prediction results of the asynchronous federated learning training trajectory prediction model. The custom comprehensive loss function consists of a standard cross entropy loss and a distance penalty term. Step 3: When the user enters a sub-scenario, the trajectory prediction model is trained through asynchronous federated learning to generate prediction results, and the cache hit rate and sub-scenario probability sequence of the edge end of the target sub-scenario are generated at the same time; Evaluate the performance of the asynchronous federated learning training trajectory prediction model based on the cache hit rate at the edge of the target sub-scenario; Based on the sub-scenario probability sequence, the scene number in the sub-scenario probability sequence and the probability corresponding to the sub-scenario number are selected and uploaded to the edge server through the GPU server; The prediction results generated by the asynchronous federated learning training trajectory prediction model are accumulated through the edge server, and the popularity of the sub-scenario is obtained by sorting the prediction results. Step 4: Combine the cache state in the edge server and the prediction matrix of the edge server sub-scenario to obtain the state; Combined with deep learning, the cache hit rate and transmission delay at the edge of the target sub-scenario are used for weighted summation to obtain rewards; Based on the Actor-Critic framework, a cache replacement framework is constructed by combining the three basic elements of state, action and reward to implement the cache replacement strategy of the edge server.

2. A method for caching a metaverse scene by combining DRL and AFL according to claim 1, characterized in that: In step 2, the custom comprehensive loss function is composed of a standard cross entropy loss and a distance penalty term, wherein the expression of the custom comprehensive loss function is: ； in, represents a custom comprehensive loss function, represents the penalty weight, Indicates The actual coordinates of the sub-scenes, Indicates The predicted coordinates of each sub-scene, Indicates The Euclidean distance between the actual coordinates and the predicted coordinates of each sub-scene, represents the total number of predicted sub-scenarios, represents the cross entropy loss function, Indicates The model after local training iterations, represents the number of learning rounds, Indicates the number of local iterations.

3. A method for caching a metaverse scene by combining DRL and AFL according to claim 2, characterized in that: In step 2, the custom comprehensive loss function is composed of a standard cross entropy loss and a distance penalty term, wherein the expression of the standard cross entropy loss is: ； in, Indicates the sub-scene number. represents the total number of training data, Indicates GPU servers, Indicates belonging to The probability of Represents the training data.

4. The method for caching a metaverse scene by combining DRL and AFL according to claim 3, characterized in that: In step 3, when the user enters a sub-scenario, the trajectory prediction model is trained through asynchronous federated learning to generate a prediction result, and the cache hit rate and sub-scenario probability sequence of the edge of the target sub-scenario are generated at the same time. The calculation relationship of the cache hit rate of the edge of the target sub-scenario is: ； in, represents the indicator function, Indicates the total number of times users log into the Metaverse system. represents the cache hit rate at the edge of the target sub-scenario, Indicates the number of times a user logs into the Metaverse system.

5. A method for caching a metaverse scene by combining DRL and AFL according to claim 4, characterized in that: In step 3, the prediction results generated by the asynchronous federated learning training trajectory prediction model are accumulated by the edge server, and the sub-scenario popularity is obtained by sorting the prediction results. The relationship between the corresponding process is: ; in, Indicates a point in time, Indicates the number of sub-scenes, Indicates at a point in time No. The access weight of each sub-scenario, express The prediction matrix of the edge server sub-scenario at the moment, Represents an edge server.

6. A method for caching a metaverse scene by combining DRL and AFL according to claim 5, characterized in that: In the step of obtaining the popularity of sub-scenario by sorting the prediction results, the relationship between the corresponding process is: ； in, express The sub-scenario prediction matrix of the sub-scenario summary at each moment.

7. The method for caching a metaverse scene by combining DRL and AFL according to claim 6, characterized in that: In step 4, the cache state in the edge server and the prediction matrix of the edge server sub-scenario are combined to obtain the state, and the relationship between the corresponding process is: ； in, Indicates the status, Indicates the cache status in the edge server. Represents an edge server.

8. The method for caching a metaverse scene by combining DRL and AFL according to claim 7, characterized in that: In step 4, the cache hit rate and transmission delay at the edge of the target sub-scenario are weighted and summed to obtain the reward. The corresponding relationship in the process is: ； in, and Both represent weights, represents a natural constant, Indicated in The transmission delay of the moment, Indicates action.

9. The method for caching a metaverse scene by combining DRL and AFL according to claim 8, characterized in that: In step 4, based on the Actor-Critic framework, the cache replacement framework is formed by combining the three basic elements of state, action and reward, where the expression of the action is: ； in, Represents a sub-scene, Indicates whether the sub-scenario performs cache replacement on the edge server. Represents a collection of subscenes.

Images