Metaverse virtual reality scene collaborative rendering method and related device based on edge computing

Abstract

This invention discloses a method and related device for collaborative rendering of metaverse virtual reality scenes based on edge computing. This method utilizes a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built metaverse user experience quality model to solve a partially observable Markov decision process problem at the metaverse user level, obtaining the optimal edge server for access by the metaverse user and its optimal GPU, CPU, and bandwidth resource requirements. Each metaverse user calculates a bid based on the current resource pricing corresponding to the optimal edge server accessed and its own optimal GPU, CPU, and bandwidth resource requirements, and sends it to the corresponding edge server. After receiving the bids from all metaverse users, the edge server updates its own resource price, calculates the actual amount of resources available for each metaverse user to lease, and leases the resources to the corresponding metaverse user. This invention introduces an edge server-metaverse user collaborative pre-rendering framework and establishes a new delay-sensitive and interest-aware user experience quality model within this framework. Furthermore, the method utilizes a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and the metaverse user experience quality model to solve the partially observable Markov decision process problem at the metaverse user level.

Description

Technical Field

[0001] The present invention belongs to the field of metaverse technology, and specifically relates to a metaverse virtual reality scene collaborative rendering method and related devices based on edge computing. Background Art

[0002] The Metaverse creates a three-dimensional virtual shared space, providing users with an immersive social experience. This new model offers a revolutionary direction for the future of social networking. Current Metaverse applications, including virtual reality games, immersive work environments, and virtual concerts, have demonstrated tremendous value and potential and have attracted significant attention from both academia and industry. During Metaverse activities, users experience rich sensory stimulation, making human-centered quality of experience a key performance metric. User experience quality is primarily influenced by the quality of the VR scene, including the richness of visual detail and the smoothness of scene transitions.

[0003] There are two key challenges in improving the quality of user experience: 1) Current wearable VR devices typically lack sufficient multi-dimensional rendering resources, such as GPUs and CPUs, to support high-quality visual experiences and low-latency complex scene generation. 2) Each metaverse user needs to make decisions that include both discrete actions (such as choosing which edge server to access) and continuous actions (such as determining the rendering quality and required resources for the VR scene). At the same time, the edge server level needs to determine how to reasonably price rendering resources and lease these resources to users. Regarding 1, in existing resource allocation research, using the same resources to achieve higher visual quality can enhance user immersion, but this often results in longer rendering delays, which in turn reduces the quality of user experience. Regarding 2, in existing collaborative VR rendering research, how to optimize discrete-continuous hybrid action decisions in a two-tier distributed system to achieve efficient resource allocation remains an underexplored issue. Summary of the Invention

[0004] In response to the problems existing in the prior art, the present invention provides a collaborative rendering method and related devices for metaverse virtual reality scenes based on edge computing, introduces an edge server-metaverse user collaborative pre-rendering framework, and establishes a new delay-sensitive and interest-aware user experience quality model under this framework; in addition, a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a metaverse user experience quality model are used to solve the partially observable Markov decision process problem of the metaverse user layer.

[0005] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0006] According to a first aspect of the present invention, a method for collaborative rendering of a metaverse virtual reality scene based on edge computing is provided, which is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. The rendering method includes:

[0007] The Metaverse virtual reality scene is divided into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer using resources leased from the edge server, while the foreground interaction is rendered locally by the Metaverse user layer.

[0008] The dynamic resource leasing problem between multiple edge servers and multiple metaverse users is formulated as a two-level decision problem, which includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level.

[0009] Using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built metaverse user experience quality model, we solve a partially observable Markov decision process problem at the metaverse user level, and determine the optimal edge server for metaverse user access, as well as the optimal GPU, CPU, and bandwidth resource requirements.

[0010] Each Metaverse user calculates a bid based on the current resource pricing corresponding to the optimal edge server they access, as well as their own optimal GPU, CPU, and bandwidth resource requirements, and sends it to the corresponding edge server. After receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the actual amount of resources that each Metaverse user can rent, and rents the resources to the corresponding Metaverse user.

[0011] In a possible implementation of the first aspect, the metaverse user experience quality model is constructed by using the Weber-Fechner law to characterize the metaverse user's subjective evaluation of picture quality, and multiplying it by the objective pre-rendering delay.

[0012] In a possible implementation of the first aspect, the Weber-Fechner law is used to characterize the subjective evaluation of image quality by Metaverse users, specifically:

[0013]

[0014] Where, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of metaverse user i in object n; r i,n k represents the resolution density that the metaverse user i requires the edge server to render object n; irepresents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio respectively; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering latency on the quality of Metaverse user experience.

[0015] In a possible implementation manner of the first aspect, the objective pre-rendering delay is specifically expressed as:

[0016]

[0017] Where, Indicates the objective pre-rendering delay; T unit Represents the time it takes for a metaverse user to move between adjacent grid points; represents the time required for Metaverse user i to request edge server j to pre-render the panorama.

[0018] In a possible implementation of the first aspect, the time required for the metaverse user i to request the edge server j to pre-render the panorama is specifically:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] Where, represents the request time that Metaverse user i needs to request edge server j to render the corresponding background environment; d ij represents the network distance between metaverse user i and edge server j; β' represents the time it takes for a request to pass through a single router;

[0025] represents the rendering time of Metaverse user i renting edge server j to render the background environment using its GPU resources; g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that metaverse user i requires the edge server to render object n;

[0026] represents the encoding time of Metaverse user i renting edge server j to encode the background environment frame using its CPU resources; f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j;

[0027] represents the transmission time of edge server j transmitting the encoded background environment frame to metaverse user i; w i,j represents the amount of output bandwidth resources of edge server j rented by Metaverse user i; h represents the size of a single pixel.

[0028] In a possible implementation of the first aspect, the metaverse user experience quality model is specifically expressed as follows:

[0029]

[0030] Where, QoE i Indicates the quality of Metaverse user experience.

[0031] In a possible implementation of the first aspect, the multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables includes:

[0032] Construct a latent representation space, map the original discrete-continuous hybrid actions into the latent representation space, then use the classic multi-agent reinforcement learning algorithm to learn the latent strategy in the latent representation space, and then decode the latent strategy into actions in the original space to interact with the environment;

[0033] The construction of the latent representation space is specifically as follows:

[0034] In a multi-agent environment, the action space faced by each metaverse user includes both discrete choices of which edge server to connect to and continuous decisions on the rendering quality and amount of resources to request;

[0035] The strategy learning in the latent space is:

[0036] A multi-agent dual-delay deep deterministic policy gradient algorithm is introduced for policy learning in the latent space. By optimizing resource leasing and allocation strategies, it ensures that metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience.

[0037] According to a second aspect of the present invention, a device for collaborative rendering of a metaverse virtual reality scene based on edge computing is provided, which is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. A rendering method includes:

[0038] A partitioning module is used to divide the Metaverse virtual reality scene into a foreground interaction and a background environment. The background environment is remotely rendered by the Metaverse user layer using resources leased from the edge server, while the foreground interaction is rendered locally by the Metaverse user layer.

[0039] A construction module for constructing a dynamic resource leasing problem between multiple edge servers and multiple metaverse users as a two-level decision problem, wherein the two-level decision problem includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level;

[0040] The solution module is used to solve the partially observable Markov decision process problem of the Metaverse user layer using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements;

[0041] The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected, as well as the Metaverse user's own optimal GPU, CPU and bandwidth resource requirements, and send it to the corresponding edge server; after receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the actual amount of resources that each Metaverse user can rent, and rents the resources to the corresponding Metaverse user.

[0042] According to a third aspect of the present invention, a device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for collaborative rendering of a metaverse virtual reality scene based on edge computing is implemented.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for collaborative rendering of a metaverse virtual reality scene based on edge computing is implemented.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] (1) The present invention proposes a collaborative virtual reality scene pre-rendering framework between metaverse users and edge servers, which allows metaverse users to rent rendering resources of edge servers according to their needs to improve the rendering quality of virtual reality scenes.

[0046] (2) The present invention establishes a comprehensive metaverse user experience quality model that takes into account not only rendering latency but also visual quality, and pays special attention to the user's interest in different virtual objects. The model integrates pre-rendering latency and the visual quality of virtual reality scenes to more comprehensively evaluate the user experience.

[0047] (3) The present invention utilizes a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a metaverse user experience quality model to optimize the discrete-continuous hybrid action decision in the user layer and achieve efficient allocation of resources.

[0048] In summary, the present invention achieves efficient resource allocation and optimization of user experience in the metaverse environment through the application of an innovative collaborative pre-rendering framework, a comprehensive user experience quality model, and a multi-agent reinforcement learning algorithm, thereby enhancing user immersion and satisfaction in the metaverse.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the specific embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of a method for collaborative rendering of a metaverse virtual reality scene based on edge computing in the present invention;

[0052] Figure 2 A specific flow chart of an embodiment of the present invention;

[0053] Figure 3 Design a diagram for a comprehensive Metaverse UX quality model;

[0054] Figure 4 Design diagram for a multi-agent reinforcement learning algorithm with mixed discrete-continuous decision variables. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] Combine Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for collaborative rendering of a metaverse virtual reality scene based on edge computing. The method is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. The method specifically includes the following steps:

[0057] Step 1: Divide the Metaverse virtual reality scene into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer by leasing resources from the edge server, and the foreground interaction is rendered locally by the Metaverse user layer.

[0058] In other words, the Metaverse system divides the Metaverse virtual reality scene into two parts: foreground interaction and background environment. Among them, Metaverse users need to rent the resources of the edge server to complete the remote rendering of the background environment, while Metaverse users only need to render the foreground interaction locally.

[0059] More specifically, the edge server-metaverse user collaborative VR scene pre-rendering framework is as follows:

[0060] The Metaverse network consists of three layers: the central cloud server layer, the edge server layer, and the Metaverse user layer. The central cloud server stores basic Metaverse user information, such as identity and virtual assets. Edge servers, located at the edge of the internet, are closer to Metaverse users than the central cloud servers. They utilize resources like GPUs and CPUs to help Metaverse users build a virtual, immersive, three-dimensional social space. Metaverse users connect to edge servers to engage in Metaverse social activities. When a Metaverse user experiences the Metaverse, the scene they observe can be divided into foreground interactions and background environments. Foreground interactions represent the user's actions and interactions with other Metaverse users and the environment. This requires minimal rendering and high real-time performance. Therefore, the Metaverse system renders foreground interactions locally on the Metaverse user's device. The background environment can be thought of as the user's "map" of the Metaverse experience. It is rich in detail and requires extensive rendering. Therefore, the Metaverse system pre-renders the background environment on the edge server and transmits the rendered results to the Metaverse user in a timely manner. The rendered results are ultimately combined with the user's foreground interactions to form a complete panoramic frame for the Metaverse user's experience.

[0061] In order to reduce the time of collaborative rendering between the edge server and the metaverse user, the metaverse system adopts the background pre-rendering method. That is, before the metaverse user moves to a certain location, the edge server has already rendered, encoded and transmitted the background panoramic frame at that location. The metaverse user only needs to decode the background environment and render the foreground interaction at the same time, and combine the foreground interaction with the background environment.

[0062] Step 2: The Metaverse system uses the Weber-Fechner law to characterize the Metaverse user's subjective evaluation of picture quality, and multiplies it with the objective pre-rendering delay part to establish a comprehensive Metaverse user experience quality model.

[0063] Reference Figure 3 As shown in the figure, the comprehensive Metaverse user experience quality model is described in detail as follows:

[0064] During Metaverse activities, Metaverse users experience a variety of sensory stimulations, so their quality of experience is a key indicator of Metaverse service quality. In the edge server-Metaverse user collaborative VR scene pre-rendering framework, the pre-rendering time for the background environment cannot exceed the time it takes for a Metaverse user to move between adjacent grid points. This is fundamental to the collaborative pre-rendering framework. Otherwise, Metaverse users will experience visual lag, resulting in a reduced quality of experience. Furthermore, within a virtual scene, Metaverse users have varying levels of interest in different virtual items, and this interest guides their line of sight. If high-quality rendering is applied to highly interesting objects, Metaverse users will subjectively assess the image quality higher, resulting in a higher quality of experience within the Metaverse experience.

[0065] The Metaverse system can improve the user experience quality by reducing rendering latency and improving image quality. However, reducing rendering latency and improving image quality conflict with each other because, with the same number of rendering resources, high rendering quality means long rendering latency. In order to fully consider rendering latency and image quality, as well as their contradictory characteristics, the Metaverse system has established a comprehensive user experience quality model, which uses the Weber-Fechner law to characterize the user's subjective evaluation of image quality and multiplies it by the objective pre-rendering delay to obtain the following formula:

[0066]

[0067] Where, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of metaverse user i in object n; r i,n k represents the resolution density that the metaverse user i requires the edge server to render object n; i represents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio respectively; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering latency on the quality of Metaverse user experience.

[0068] 1) Objective pre-rendering delay part

[0069] Request time: The time it takes for Metaverse user i to request edge server j to render the corresponding background environment (including rendering quality and the number of rented resources) is called the request time. The time required for the request to be transmitted to the edge server is called the request time, which is proportional to the number of routers on the transmission path between the Metaverse user and the edge server and can be expressed as the following formula:

[0070]

[0071] Where, d ij represents the network distance between the metaverse user and the edge server; β' represents the time it takes for a request to pass through a single router.

[0072] Rendering time: represents the rendering time required for Metaverse user i to use edge server j's GPU resources to render the background environment. For Metaverse user i, the workload required by the edge server to render is the sum of the number of pixels of all virtual objects in the background environment. The rendering time can be expressed as follows:

[0073]

[0074] Where g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that metaverse user i requires the edge server to render object n.

[0075] Coding time: represents the encoding time it takes for Metaverse user i to use the CPU resources of edge server j to encode the background environment frame. Given the amount of CPU resources, the encoding time is proportional to the total rendering workload and can be expressed as follows:

[0076]

[0077] Where, f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j.

[0078] Transfer time: represents the transmission time from edge server j to Metaverse user i, which can be divided into two parts: propagation time and sending time. Propagation time refers to the time it takes for the first byte of the background frame to propagate through the network, which is consistent with the request time. Sending time is related to the bottleneck bandwidth in the transmission path. Assuming that the server's output bandwidth is the bottleneck bandwidth on the transmission path, the transmission time can be expressed as follows:

[0079]

[0080] Where w i,j represents the amount of output bandwidth resources of edge server j rented by metaverse user i; h represents the size of a single pixel (in bits).

[0081] Adding up the request time, rendering time, encoding time, and transmission time gives the total pre-rendering time, which is represents the time required for Metaverse user i to request edge server j to pre-render the panorama. The pre-rendering time cannot exceed the time it takes for a Metaverse user to move between adjacent grid points. Therefore, the objective pre-rendering delay for Metaverse user experience quality is:

[0082]

[0083] 2) Metaverse users’ subjective evaluation of image quality

[0084] During the pre-rendering process, there are two main factors that affect image quality: resolution and compression ratio. Resolution determines the richness of detail in the image, and compression ratio determines how much original information is retained in the background environment frame during encoding. At the same time, Metaverse users have different levels of interest in different virtual objects. Rendering an object of no interest with high quality may be futile because Metaverse users may not pay much attention to the object. Therefore, interest plays a key role in the subjective evaluation of Metaverse users. The subjective evaluation of image quality by Metaverse users can be expressed as the following formula:

[0085]

[0086] Where, Indicates the subjective evaluation results of the Metaverse users on the picture quality.

[0087] Step 3: The Metaverse system uses a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a Metaverse user experience quality model to solve the partially observable Markov decision process problem at the Metaverse user layer, and obtain the optimal edge server for Metaverse users to access, as well as the requested GPU, CPU, and output bandwidth resources.

[0088] Reference Figure 4 Specifically, the multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables is as follows:

[0089] In the Metaverse system, within the edge server-user collaborative pre-rendering framework, Metaverse users need to lease resources from edge servers to assist in rendering the background environment. The Metaverse system models the dynamic resource leasing problem between multiple edge servers and Metaverse users as a two-tier decision-making problem. At the Metaverse user level, each Metaverse user must make dynamic discrete-continuous hybrid action decisions. At the edge server level, each edge server must decide on the price of its resources. Resource transactions between the two layers are conducted through auctions.

[0090] The Metaverse system models the optimization problem of Metaverse users as a partially observable Markov decision process, which can be represented by a 7-tuple. The action space is a mixture of discrete and continuous. The action taken by Metaverse user i at time round t can be represented as:

[0091]

[0092] It consists of the following two parts:

[0093] Discrete Action: Discrete Action v i,t represents the edge server that Metaverse user i chooses to access at time round t;

[0094] Continuous action: Continuous action C i,t Includes the rendering resolution density of all virtual objects Compression ratio k i,t , and the requested GPU, CPU, and output bandwidth resources g i,t ,f i,t ,w i,t .

[0095] The reward a Metaverse user receives in each time round is defined as the difference between their experience quality and all expenses, including resource rental fees paid to edge servers and bandwidth fees paid to routers. The goal of each Metaverse user is to find an optimal strategy to maximize their long-term returns:

[0096]

[0097] Where γ∈[0,1] represents the discount factor, which determines the importance of future returns.

[0098] The Metaverse system utilizes a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables to solve the partially observable Markov decision process problem at the user level of the Metaverse. The algorithm first constructs a compact, decodable, and semantically smooth latent representation space to map the original discrete-continuous hybrid actions into this space. A classic multi-agent reinforcement learning algorithm is then used within this space to learn the latent policy, which is then decoded into actions in the original space to interact with the environment.

[0099] The following describes in detail the multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables in two parts: the construction of the latent representation space and the learning of strategies within the latent space.

[0100] 1) Construction of latent representation space

[0101] The metaverse system first establishes an embedding table Each line of e ζ,v (v is the row index) is a continuous vector of dimension d1, which is used to represent the discrete action v. Then, the observation o of the metaverse user and the discrete action representation e are used. ζ,v As a condition, a conditional variational autoencoder (CVAE) q φ (φ is a learnable parameter), q φ Map the continuous action C to the latent action in the representation space Specifically, CVAEq φ Will o,e ζ,v and C as input, and then output a Gaussian distribution The mean and standard deviation of , from which latent variables z are then sampled.

[0102] For any hidden action and They should all be able to be decoded into actions in the original action space, so that they can interact with the environment. For continuous actions, a CVAE decoder p with learnable parameters ψ is used ψ Decode, decoder p ψ With o and e ζ,v As a condition, z is deterministically decoded into a continuous action C in the original action space. For discrete actions, a nearest neighbor search is performed in the embedding table to decode it as

[0103] The encoding and decoding process of CVAE can be expressed as the following formula:

[0104] Encoding:e ζ,v =E ζ (v),z~q φ (·|C,o,e ζ,v )

[0105]

[0106]

[0107] The metaverse system learns the embedding table and CVAE by minimizing the following loss function:

[0108]

[0109] Where, L HyAR represents the loss function of the hybrid action representation model; φ represents the encoder parameters of the conditional variational autoencoder; ψ represents the decoder parameters of the conditional variational autoencoder; ζ represents the parameters of the embedding table; is the expected value, which represents the average of all samples; α is the weight hyperparameter used to balance the reconstruction error term; Indicates continuous action C i Instead of the refactored version The L2 norm square between , that is, the reconstruction error; λ represents the weight hyperparameter, which is used to balance the prediction error term; Indicates actual resource price changes; represents the predicted resource price change; η represents the weight hyperparameter used to balance the KL divergence term; D KL represents the Kullback-Leibler divergence, which measures the difference between two probability distributions; represents the encoder of the conditional variational autoencoder, which outputs the parameters of a Gaussian distribution to represent the latent action space;

[0110] Represents the labeled normal distribution, which serves as the reference distribution for the KL divergence.

[0111] That is, the first term represents the mean squared reconstruction error between the original continuous action and the reconstructed continuous action; the third term represents the KL divergence between the latent variable and the standard Gaussian distribution; and the second term represents the mean squared prediction error between the actual metaverse user observation (edge ​​server resource price) and its predicted value.

[0112] 2) Strategy learning in latent space

[0113] The Metaverse system introduces a multi-agent dual-delay deep deterministic policy gradient algorithm for policy learning in latent space. It can not only ensure that Metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience by optimizing resource leasing and allocation strategies, but also comprehensively consider the multi-dimensional resource requirements and limitations such as computing, storage, and network bandwidth required for the operation of the Metaverse to achieve efficient resource utilization.

[0114] In the multi-agent dual-delayed deep deterministic policy gradient algorithm, each metaverse user has a policy network and two value networks. In the metaverse system scenario, all metaverse users have the same observation space, action space, and reward function form, so the metaverse system uses a parameter sharing framework to more efficiently train all networks.

[0115] For the value network, the Metaverse system uses the following Loss function to learn:

[0116]

[0117] Where E is the expectation operator; is the lth value network's potential actions (e 1:I ,z 1:I )’s value estimate; i is the timing difference target, which can be expressed as follows:

[0118]

[0119] Where r i is the immediate reward obtained by the i-th agent at time step t; γ is a discount factor used to determine the importance of future rewards relative to the current reward; is the value network's policy output for the next state s' and all agents estimated value.

[0120] For the policy network, the Metaverse system uses deterministic policy gradient to learn:

[0121]

[0122] Step 4: Each Metaverse user calculates a bid based on the current resource pricing of the optimal edge server they are connected to and the amount of resources they are requesting, and sends it to the corresponding edge server. After receiving all bids, the edge server updates its resource price, calculates the actual amount of resources available for each Metaverse user to lease, and leases the resources to the corresponding Metaverse user.

[0123] Specifically, the edge server rendering resource auction mechanism is as follows:

[0124] Metaverse users' decisions include the quantity of the three rendering resources they request. However, because a large number of Metaverse users simultaneously request resources from edge servers, this can lead to resource contention among Metaverse users. Edge servers need to design a reasonable pricing-based resource leasing mechanism to allocate their limited resources. Furthermore, the resource pricing method must incentivize Metaverse users to engage in long-term rendering collaboration.

[0125] The Metaverse system designs its resource leasing mechanism as an auction, with all edge servers acting as auctioneers and all Metaverse users acting as bidders. Each Metaverse user calculates a bid based on the edge server's current resource pricing and the amount of resources they are requesting, and sends it to the corresponding edge server. After receiving all bids, the edge server updates the resource price using the following resource pricing method, calculates the actual amount of resources each Metaverse user can lease, and then leases the resources to the corresponding Metaverse user:

[0126]

[0127]

[0128] Where, represents the price set by edge server j for resource type h at time t+1; represents the operating cost borne by edge server j for using unit resource h; U j,t represents the set of all users who access edge server j at time t; V represents the bid of Metaverse user i for resource h at time t; j h represents the total capacity of resource type h owned by edge server j.

[0129] This resource pricing method has been shown to be reasonable and incentivize long-term collaboration among Metaverse users by satisfying the following three conditions:

[0130] 1) Individual rationality condition. The profit of the edge server is not less than 0:

[0131]

[0132] Where U j,t represents the set of all users who access edge server j at time t; represents the price set by edge server j for GPU resources at time t; represents the operating cost of edge server j for using unit GPU resources; g′ i，t represents the amount of GPU resources leased by Metaverse user i from edge server j at time t; f′ i，t represents the amount of CPU resources leased by Metaverse user i from edge server j at time t; represents the price set by edge server j for bandwidth resources at time t; represents the operating cost borne by edge server j for using unit bandwidth resources; w′ i，tIt represents the amount of bandwidth resources leased by Metaverse user i from edge server j at time t.

[0133] 2) Resource capacity limitation: For each edge server, the sum of the resources allocated in each time round cannot exceed the total amount of resources it owns.

[0134] 3) Participation incentives. The Metaverse market is still in its infancy. To attract more Metaverse users to participate in social activities within the Metaverse, the resource price of edge servers should be as low as possible, while meeting individual rationality conditions and resource capacity constraints.

[0135] This invention utilizes edge server resources for background rendering while processing foreground interactions locally. The Metaverse system dynamically adjusts resource allocation through user behavior analysis and resource leasing modeling. It also incorporates a user experience quality model based on the Weber-Fechner theorem and a multi-agent reinforcement learning algorithm to optimize resource request and allocation strategies, ensuring users receive a high-quality visual and interactive experience. This allows for efficient resource utilization and reasonable market pricing, thereby improving the performance of the entire Metaverse platform and Metaverse user satisfaction.

[0136] Exemplarily, when this method is applied to a metaverse system, the metaverse system divides the virtual reality scene into two parts: foreground interaction and background environment; the metaverse system performs mobility analysis based on user behavior, and the edge server only needs to pre-render the background environment on the user's movement path; the metaverse system models the resource leasing problem as a two-layer decision problem, which includes a partially observable Markov decision process at the user layer and a resource pricing problem at the edge server layer; the metaverse system uses the Weber-Fechner law to characterize the user's subjective evaluation of picture quality, and multiplies it with the objective pre-rendering delay to establish a user experience quality model; the metaverse system uses a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables to solve the decision problem at the user layer and obtain the optimal edge server and resource quantity.

[0137] Exemplarily, when this method is applied to the metaverse user layer in the metaverse system, the user device calculates a bid based on the resource pricing of the optimal edge server it is connected to and the number of resources it requests; the user device sends the bid to the corresponding edge server; the user device receives resource leasing feedback from the edge server and performs foreground interaction rendering.

[0138] Exemplarily, when this method is applied to the edge server layer in the metaverse system, the edge server updates its resource price after receiving all bids; the edge server calculates the actual amount of resources that each user can rent; the edge server rents the resources to the corresponding metaverse user; and the edge server pre-renders the background environment on the user's movement path based on the user mobility analysis results.

[0139] In another embodiment of the present invention, a device for collaborative rendering of a metaverse virtual reality scene based on edge computing is provided, which is used to implement the above-mentioned method for collaborative rendering of a metaverse virtual reality scene based on edge computing. The device is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. The rendering method includes:

[0140] The division module is used to divide the Metaverse virtual reality scene into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer using the resources of the edge server leased, and the foreground interaction is rendered locally by the Metaverse user layer.

[0141] A building module is used to construct the dynamic resource leasing problem between multiple edge servers and multiple metaverse users as a two-level decision problem, which includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level.

[0142] The solution module is used to use a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a pre-built metaverse user experience quality model to solve the partially observable Markov decision process problem of the metaverse user layer, and obtain the optimal edge server for metaverse user access and the optimal GPU, CPU and bandwidth resource requirements.

[0143] The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected, as well as the Metaverse user's own optimal GPU, CPU and bandwidth resource requirements, and send it to the corresponding edge server; after receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the actual amount of resources that each Metaverse user can rent, and rents the resources to the corresponding Metaverse user.

[0144] All relevant contents of each step involved in the embodiment of the aforementioned method for collaborative rendering of a metaverse virtual reality scene based on edge computing can be referred to the functional description of the functional module corresponding to the device for collaborative rendering of a metaverse virtual reality scene based on edge computing in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present invention can be integrated into one processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0145] In another embodiment of the present invention, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a metaverse virtual reality scene collaborative rendering method based on edge computing.

[0146] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium herein may include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides storage space, which stores the terminal's operating system. In addition, the storage space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device. The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the above-mentioned embodiment regarding a method for collaborative rendering of a metaverse virtual reality scene based on edge computing.

[0147] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0151] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0152] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A collaborative rendering method for metaverse virtual reality scenes based on edge computing, characterized in that: Applied to a metaverse system including an edge server layer and a metaverse user layer, the edge server layer including multiple edge servers, the metaverse user layer including multiple metaverse users, the rendering method comprising: The Metaverse virtual reality scene is divided into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer using resources leased from the edge server, while the foreground interaction is rendered locally by the Metaverse user layer. The dynamic resource leasing problem between multiple edge servers and multiple metaverse users is formulated as a two-level decision problem, which includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level. Using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built metaverse user experience quality model, we solve a partially observable Markov decision process problem at the metaverse user level, and determine the optimal edge server for metaverse user access, as well as the optimal GPU, CPU, and bandwidth resource requirements. Each Metaverse user calculates a bid based on the current resource pricing corresponding to the optimal edge server they access, as well as their own optimal GPU, CPU, and bandwidth resource requirements, and sends it to the corresponding edge server. After receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the actual amount of resources that each Metaverse user can rent, and rents the resources to the corresponding Metaverse user.

2. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 1, characterized in that: The Metaverse User Experience Quality Model is constructed by using the Weber-Fechner law to characterize the Metaverse user's subjective evaluation of picture quality and multiplying it by the objective pre-rendering delay.

3. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 2, characterized in that: The Weber-Fechner law is used to characterize the subjective evaluation of the image quality by Metaverse users, specifically: Where, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of metaverse user i in object n; r i,n k represents the resolution density that the metaverse user i requires the edge server to render object n; i represents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio respectively; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering latency on the quality of Metaverse user experience.

4. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 3, characterized in that: The objective pre-rendering delay is specifically expressed as: Where, Indicates the objective pre-rendering delay; T unit Represents the time it takes for a metaverse user to move between adjacent grid points; represents the time required for Metaverse user i to request edge server j to pre-render the panorama.

5. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 4, characterized in that: The time required for metaverse user i to request edge server j to pre-render the panorama is specifically: Where, represents the request time that Metaverse user i needs to request edge server j to render the corresponding background environment; d ij represents the network distance between metaverse user i and edge server j; β' represents the time it takes for a request to pass through a single router; represents the rendering time of Metaverse user i renting edge server j to render the background environment using its GPU resources; g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that metaverse user i requires the edge server to render object n; represents the encoding time of Metaverse user i renting edge server j to encode the background environment frame using its CPU resources; f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j; represents the transmission time of edge server j transmitting the encoded background environment frame to metaverse user i; w i,j represents the amount of output bandwidth resources of edge server j rented by Metaverse user i; h represents the size of a single pixel.

6. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 5, characterized in that: The Metaverse User Experience Quality Model is specifically expressed as follows: Where, QoE i Indicates the quality of Metaverse user experience.

7. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 1, characterized in that: The multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables includes: Construct a latent representation space, map the original discrete-continuous hybrid actions into the latent representation space, then use the classic multi-agent reinforcement learning algorithm to learn the latent strategy in the latent representation space, and then decode the latent strategy into actions in the original space to interact with the environment; The construction of the latent representation space is specifically as follows: In a multi-agent environment, the action space faced by each metaverse user includes both discrete choices of which edge server to connect to and continuous decisions on the rendering quality and amount of resources to request; The strategy learning in the latent space is: A multi-agent dual-delay deep deterministic policy gradient algorithm is introduced for policy learning in the latent space. By optimizing resource leasing and allocation strategies, it ensures that metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience.

8. A collaborative rendering device for metaverse virtual reality scenes based on edge computing, characterized in that: Applied to a metaverse system including an edge server layer and a metaverse user layer, the edge server layer including multiple edge servers, the metaverse user layer including multiple metaverse users, the rendering method comprising: A partitioning module is used to divide the Metaverse virtual reality scene into a foreground interaction and a background environment. The background environment is remotely rendered by the Metaverse user layer using resources leased from the edge server, while the foreground interaction is rendered locally by the Metaverse user layer. A construction module for constructing a dynamic resource leasing problem between multiple edge servers and multiple metaverse users as a two-level decision problem, wherein the two-level decision problem includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level; The solution module is used to solve the partially observable Markov decision process problem of the Metaverse user layer using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements; The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected, as well as the Metaverse user's own optimal GPU, CPU and bandwidth resource requirements, and send it to the corresponding edge server; after receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the actual amount of resources that each Metaverse user can rent, and rents the resources to the corresponding Metaverse user.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements a method for collaborative rendering of a metaverse virtual reality scene based on edge computing as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for collaborative rendering of a metaverse virtual reality scene based on edge computing as described in any one of claims 1 to 7.

Images