Edge calculation-based meta-universe virtual reality scene collaborative rendering method and related device

Abstract

The invention discloses a meta-universe virtual reality scene collaborative rendering method based on edge calculation and a related device. A multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a pre-constructed meta-universe user experience quality model are utilized; solving a partial observable Markov decision process problem of the meta-universe user layer, and obtaining an optimal edge server accessed by the meta-universe user and an optimal GPU, CPU and bandwidth resource demand quantity; each meta-universe user calculates a bidding price according to the current resource price corresponding to the accessed optimal edge server and the optimal GPU, CPU and bandwidth resource demand quantity of the meta-universe user, and sends the bidding price to the corresponding edge server; and after receiving the bids of all the meta-universe users, the edge server updates own resource prices, calculates the quantity of resources which can be actually rented by each meta-universe user, and rents the resources to the corresponding meta-universe users. According to the method, an edge server-meta universe user cooperation pre-rendering framework is introduced, and a new delay-sensitive and interest-aware user experience quality model is established under the framework; in addition, a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a meta-universe user experience quality model are utilized to solve a partially observable Markov decision process problem of a meta-universe user layer.

Description

Technical Field

[0001] The present invention belongs to the field of metaverse technology, and specifically relates to a metaverse virtual reality scene collaborative rendering method and related devices based on edge computing. Background Art

[0002] By building a three-dimensional virtual shared space, Metaverse brings an immersive social experience to Metaverse users. This new model provides a revolutionary development direction for future social networks. Current Metaverse applications, including virtual reality games, immersive work environments, and virtual concerts, not only demonstrate great value and potential, but also attract great attention from academia and industry. In Metaverse activities, Metaverse users are exposed to rich sensory stimulation, which makes the human-centered experience quality a key indicator for measuring the performance of Metaverse applications. The user's experience quality is mainly affected by the quality of the virtual reality scene, which includes the richness of visual details and the smoothness of scene transitions.

[0003] In the process of improving the quality of user experience, there are two key challenges: 1) Current wearable VR devices usually lack sufficient multi-dimensional rendering resources, such as GPUs and CPUs, to support high-quality visual experience and low-latency complex scene generation. 2) Each metaverse user needs to make decisions that include both discrete actions (such as choosing which edge server to access) and continuous actions (such as determining the rendering quality and required resources of the VR scene), and the edge server level needs to determine how to reasonably price rendering resources and lease these resources to users. Regarding 1, in existing resource allocation research, using the same resources to achieve higher visual quality can improve user immersion, but this usually leads to longer rendering delays, which in turn reduces the quality of user experience. Regarding 2, in existing collaborative VR rendering research, how to optimize discrete-continuous hybrid action decisions in a two-layer distributed system to achieve efficient resource allocation is still an underexplored problem. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a collaborative rendering method and related devices for a metaverse virtual reality scene based on edge computing, introduces an edge server-metaverse user collaborative pre-rendering framework, and establishes a new delay-sensitive and interest-aware user experience quality model under this framework; in addition, a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a metaverse user experience quality model are used to solve the partially observable Markov decision process problem of the metaverse user layer.

[0005] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0006] According to a first aspect of the present invention, a method for collaborative rendering of a metaverse virtual reality scene based on edge computing is provided, which is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes a plurality of edge servers, and the metaverse user layer includes a plurality of metaverse users. The rendering method includes:

[0007] The Metaverse virtual reality scene is divided into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer by leasing resources from the edge server, and the foreground interaction is rendered locally by the Metaverse user layer.

[0008] The dynamic resource leasing problem between multiple edge servers and multiple metaverse users is constructed as a two-level decision problem, which includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level.

[0009] Using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, we solve the partially observable Markov decision process problem at the Metaverse user layer and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements.

[0010] Each Metaverse user calculates a bid based on the current resource pricing of the optimal edge server to which it is connected, as well as the Metaverse user's own optimal GPU, CPU, and bandwidth resource requirements, and sends it to the corresponding edge server. After receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

[0011] In a possible implementation of the first aspect, the Metaverse user experience quality model is constructed by using the Weber-Fechner law to characterize the Metaverse user's subjective evaluation of picture quality and multiplying it by the objective pre-rendering delay.

[0012] In a possible implementation of the first aspect, the Weber-Fechner law is used to characterize the subjective evaluation of the Metaverse user on the picture quality, specifically:

[0013]

[0014] In the formula, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of Metaverse user i in object n; r i,n k represents the resolution density that Metaverse user i requires the edge server to render object n; irepresents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering delay on the quality of Metaverse user experience.

[0015] In a possible implementation manner of the first aspect, the objective pre-rendering delay is specifically expressed as:

[0016]

[0017] In the formula, Indicates the objective pre-rendering delay; T unit Represents the time it takes for a Metaverse user to move between adjacent grid points; It represents the time required for Metaverse user i to request edge server j to pre-render the panorama.

[0018] In a possible implementation manner of the first aspect, the time required for the metaverse user i to request the edge server j to pre-render the panorama is specifically:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] In the formula, represents the request time that Metaverse user i needs to request edge server j to render the corresponding background environment; d ij represents the network distance between Metaverse user i and edge server j; β' represents the time it takes for a request to pass through a single router;

[0025] represents the rendering time of Metaverse user i renting edge server j to render the background environment using its GPU resources; g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that Metaverse user i requires the edge server to render object n;

[0026] represents the encoding time of Metaverse user i renting edge server j to encode the background environment frame using its CPU resources; f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j;

[0027] represents the transmission time of edge server j transmitting the encoded background environment frame to Metaverse user i; w i,j represents the amount of output bandwidth resources of edge server j rented by Metaverse user i; h represents the size of a single pixel.

[0028] In a possible implementation manner of the first aspect, the Metaverse user experience quality model is specifically expressed as:

[0029]

[0030] Where, QoE i Represents the quality of Metaverse user experience.

[0031] In a possible implementation of the first aspect, the multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables includes:

[0032] Construct a latent representation space, map the original discrete-continuous hybrid actions into the latent representation space, then use the classic multi-agent reinforcement learning algorithm to learn the latent strategy in the latent representation space, and then decode the latent strategy into actions in the original space to interact with the environment;

[0033] The construction of the latent representation space is specifically as follows:

[0034] In a multi-agent environment, the action space faced by each metaverse user includes both discrete choices of edge servers to connect to and continuous decisions on the rendering quality and amount of resources requested;

[0035] The strategy learning in the latent space is:

[0036] A multi-agent dual-delay deep deterministic policy gradient algorithm is introduced for policy learning in the latent space. By optimizing resource leasing and allocation strategies, it ensures that Metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience.

[0037] According to a second aspect of the present invention, a metaverse virtual reality scene collaborative rendering device based on edge computing is provided, which is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes a plurality of edge servers, and the metaverse user layer includes a plurality of metaverse users, and a rendering method includes:

[0038] A partitioning module is used to divide the Metaverse virtual reality scene into a foreground interaction and a background environment. The background environment is remotely rendered by the Metaverse user layer by leasing resources from the edge server, and the foreground interaction is rendered locally by the Metaverse user layer.

[0039] A construction module is used to construct a dynamic resource leasing problem between multiple edge servers and multiple metaverse users as a two-level decision problem, wherein the two-level decision problem includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level;

[0040] A solution module is used to solve the partially observable Markov decision process problem of the Metaverse user layer by using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements;

[0041] The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected to it, as well as the Metaverse user's own optimal GPU, CPU, and bandwidth resource requirements, and send it to the corresponding edge server; after the edge server receives the bids from all Metaverse users, it updates its own resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

[0042] According to a third aspect of the present invention, there is provided a device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for collaborative rendering of a metaverse virtual reality scene based on edge computing is implemented.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for collaborative rendering of a metaverse virtual reality scene based on edge computing is implemented.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] (1) The present invention proposes a collaborative virtual reality scene pre-rendering framework between Metaverse users and edge servers, which allows Metaverse users to rent rendering resources of edge servers according to demand to improve the rendering quality of virtual reality scenes.

[0046] (2) The present invention establishes a comprehensive Metaverse user experience quality model, which not only takes into account rendering latency but also visual quality, and pays special attention to the user's interest in different virtual objects. The model integrates pre-rendering latency and visual quality of virtual reality scenes to more comprehensively evaluate the user experience.

[0047] (3) The present invention utilizes a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a metaverse user experience quality model to optimize the discrete-continuous mixed action decisions in the user layer and achieve efficient allocation of resources.

[0048] In summary, the present invention achieves efficient allocation of resources and optimization of user experience in the metaverse environment through the application of an innovative collaborative pre-rendering framework, a comprehensive user experience quality model, and a multi-agent reinforcement learning algorithm, thereby enhancing the user's immersion and satisfaction in the metaverse.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 This is a flow chart of a method for collaborative rendering of a metaverse virtual reality scene based on edge computing of the present invention;

[0052] Figure 2 A specific flow chart of an embodiment of the present invention;

[0053] Figure 3 Design a diagram for a comprehensive Metaverse UX quality model;

[0054] Figure 4 Design diagram for a multi-agent reinforcement learning algorithm with mixed discrete-continuous decision variables. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Combination Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for collaborative rendering of a metaverse virtual reality scene based on edge computing. The method is applied to a metaverse system including an edge server layer and a metaverse user layer. The edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. The method specifically includes the following steps:

[0057] Step 1: Divide the Metaverse virtual reality scene into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer by leasing the resources of the edge server, and the foreground interaction is rendered locally by the Metaverse user layer.

[0058] In other words, the Metaverse system divides the Metaverse virtual reality scene into two parts: foreground interaction and background environment. Among them, Metaverse users need to rent the resources of the edge server to complete the remote rendering of the background environment, and at the same time, Metaverse users only need to render the foreground interaction locally.

[0059] More specifically, the edge server-metaverse user collaborative virtual reality scene pre-rendering framework is as follows:

[0060] The Metaverse network is divided into three layers, namely the central cloud server layer, the edge server layer and the Metaverse user layer. The central cloud server stores the basic information of Metaverse users, such as identity information and virtual assets; the edge server is located at the edge of the Internet, closer to the Metaverse users than the central cloud server, and uses resources such as GPU and CPU to help Metaverse users build a virtual immersive three-dimensional social space; Metaverse users conduct Metaverse social activities by accessing the edge server. When Metaverse users are experiencing the Metaverse, the scenes they observe can be divided into foreground interactions and background environments. Among them, foreground interactions represent the actions of Metaverse users and their interactions with other Metaverse users and the environment, which requires a small amount of rendering and high real-time performance. Therefore, the Metaverse system renders the foreground interactions on the local device of the Metaverse user. The background environment can be understood as the "map" of the Metaverse user in the Metaverse experience. It is rich in details and requires a large amount of rendering. Therefore, the Metaverse system pre-renders the background environment at the edge server, and transmits the rendering results to the Metaverse user in a timely manner, and finally combines it with the foreground interaction of the Metaverse user to form a complete panoramic frame for the Metaverse user to experience the Metaverse.

[0061] In order to reduce the time of collaborative rendering between edge servers and Metaverse users, the Metaverse system adopts the method of background pre-rendering. That is, before the Metaverse user moves to a certain location, the edge server has already rendered, encoded and transmitted the background panoramic frame at that location. The Metaverse user only needs to decode the background environment and render the foreground interaction at the same time, and combine the foreground interaction with the background environment.

[0062] Step 2: The Metaverse system uses the Weber-Fechner law to characterize the Metaverse user's subjective evaluation of picture quality, and multiplies it with the objective pre-rendering delay part to establish a comprehensive Metaverse user experience quality model.

[0063] Reference Figure 3 As shown in the figure, the comprehensive Metaverse user experience quality model is described in detail as follows:

[0064] In Metaverse activities, Metaverse users will receive various sensory stimulations, so their experience quality is an important indicator for measuring the quality of Metaverse services. In the edge server-Metaverse user cooperative virtual reality scene pre-rendering framework, the pre-rendering time for the background environment cannot exceed the time it takes for Metaverse users to move between adjacent grid points. This is the basis of the cooperative pre-rendering framework, otherwise Metaverse users will experience visual freezes, causing their experience quality to decline. In addition, in a virtual scene, Metaverse users have different interests in different virtual items, and the interest will guide the Metaverse user's line of sight. If objects with high interest are rendered with higher quality, then Metaverse users will have a higher subjective evaluation of the picture quality, thereby obtaining a higher quality of experience in the Metaverse experience.

[0065] The Metaverse system can improve the user experience quality by reducing rendering latency and improving picture quality. However, reducing rendering latency and improving picture quality conflict with each other, because with the same number of rendering resources, high rendering quality means long rendering latency. In order to fully consider rendering latency and picture quality, as well as the contradictory characteristics of the two, the Metaverse system has established a comprehensive user experience quality model, which uses the Weber-Fechner law to characterize the user's subjective evaluation of picture quality, and multiplies it with the objective pre-rendering delay to obtain the following formula:

[0066]

[0067] In the formula, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of Metaverse user i in object n; r i,n k represents the resolution density that Metaverse user i requires the edge server to render object n; i represents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering delay on the quality of Metaverse user experience.

[0068] 1) Objective pre-rendering delay part

[0069] Request time: It represents the request time that Metaverse user i needs to request edge server j to render the corresponding background environment (including rendering quality and the number of rented resources). The time required for the request to be transmitted to the edge server is called the request time, which is proportional to the number of routers on the transmission path between the Metaverse user and the edge server, and is expressed as the following formula:

[0070]

[0071] Where, d ij represents the network distance between the Metaverse user and the edge server; β' represents the time it takes for a request to pass through a single router.

[0072] Rendering time: It represents the rendering time of Metaverse user i renting edge server j to render the background environment using its GPU resources. For Metaverse user i, the workload required by the edge server to render is the sum of the number of pixels of all virtual objects in its background environment. The rendering time can be expressed as the following formula:

[0073]

[0074] In the formula, g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that Metaverse user i requires the edge server to render object n.

[0075] Coding time: It represents the encoding time of Metaverse user i renting edge server j to encode the background environment frame using its CPU resources. Given the amount of CPU resources, the encoding time is proportional to the total rendering workload, and the encoding time can be expressed as the following formula:

[0076]

[0077] In the formula, f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j.

[0078] Transfer time: It represents the transmission time of edge server j transmitting the encoded background environment frame to Metaverse user i, which can be divided into two parts: propagation time and sending time. The propagation time refers to the propagation time of the first byte of the background environment frame in the network, which is consistent with the request time; the sending time is related to the bottleneck bandwidth in the transmission path. Assuming that the output bandwidth of the server is the bottleneck bandwidth on the transmission path, the transmission time can be expressed as the following formula:

[0079]

[0080] In the formula, w i,j represents the amount of output bandwidth resources of edge server j rented by Metaverse user i; h represents the size of a single pixel (in bits).

[0081] Adding the request time, rendering time, encoding time, and transmission time together gives the total pre-rendering time, which is It represents the time required for Metaverse user i to request edge server j to pre-render the panorama. The pre-rendering time cannot exceed the time it takes for Metaverse users to move between adjacent grid points, so the objective pre-rendering delay part of the Metaverse user experience quality is:

[0082]

[0083] 2) Metaverse users’ subjective evaluation of image quality

[0084] In the pre-rendering process, there are two main factors that affect the image quality, namely resolution and compression ratio. Resolution determines the richness of details in the picture, and compression ratio determines how much original information is retained in the background environment frame when encoding. At the same time, Metaverse users have different interests in different virtual objects. It may be futile to render an object of no interest with high quality, because Metaverse users may not pay much attention to the object. Therefore, interest plays a key role in the subjective evaluation of Metaverse users. The subjective evaluation of Metaverse users on image quality can be expressed as the following formula:

[0085]

[0086] In the formula, Represents the subjective evaluation results of Metaverse users on picture quality.

[0087] Step 3: The Metaverse system uses a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a Metaverse user experience quality model to solve the partially observable Markov decision process problem at the Metaverse user layer, and obtain the optimal edge server accessed by the Metaverse user and the requested GPU, CPU, and output bandwidth resources.

[0088] Reference Figure 4 Specifically, the multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables is as follows:

[0089] In the edge server-metaverse user cooperative pre-rendering framework of the Metaverse system, Metaverse users need to rent edge server resources to help them complete the rendering of the background environment. The Metaverse system models the dynamic resource leasing problem between multiple edge servers and multiverse users as a two-layer decision problem. For the Metaverse user layer, each Metaverse user needs to make a dynamic discrete-continuous hybrid action decision. For the edge server layer, each edge server needs to decide its resource price. The resource transaction between the two layers is carried out through auctions.

[0090] The Metaverse system models the optimization problem of Metaverse users as a partially observable Markov decision process, which can be represented by a 7-tuple. Among them, its action space is a discrete-continuous mixture, and the action taken by Metaverse user i at time round t can be represented as:

[0091]

[0092] It consists of the following two parts:

[0093] Discrete Action: Discrete Action v i,t represents the edge server that Metaverse user i chooses to access at time round t;

[0094] Continuous action: Continuous action C i,t Includes the rendering resolution density of all virtual objects Compression ratio k i,t , and the requested GPU, CPU, and output bandwidth resources g i,t ,f i,t ,w i,t .

[0095] The reward received by a Metaverse user in each time round is defined as the difference between its experience quality and all expenditures, including resource rental fees paid to edge servers and bandwidth fees paid to routers. The goal of each Metaverse user is to find an optimal strategy to maximize its long-term return:

[0096]

[0097] Where γ∈[0,1] represents the discount factor, which determines the importance of future returns.

[0098] The Metaverse system uses a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables to solve the partially observable Markov decision process problem at the user level of the Metaverse. The algorithm first constructs a compact, decodable, and semantically smooth latent representation space to map the original discrete-continuous mixed actions into this space, and then uses a classic multi-agent reinforcement learning algorithm in this space to learn the latent strategy, which is then decoded into actions in the original space to interact with the environment.

[0099] The following is a detailed description of the multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables in two parts: the construction of the latent representation space and the strategy learning in the latent space.

[0100] 1) Construction of latent representation space

[0101] The Metaverse system first creates an embedding table Each line of e ζ,v (v is the row index) is a d1-dimensional continuous vector used to represent the discrete action v. Then, the observation o of the metaverse user and the discrete action representation e ζ,v As a condition, we use a conditional variational autoencoder (CVAE) q φ (φ is a learnable parameter), q φ Map the continuous action C to the latent action in the representation space Specifically, CVAEq φ Will o,e ζ,v and C as input, and then output a Gaussian distribution The mean and standard deviation of , from which latent variables z are then sampled.

[0102] For any hidden action and They should all be decoded into actions in the original action space, so that they can interact with the environment. For continuous actions, a CVAE decoder p with learnable parameters ψ is used ψ Decode, decoder p ψ With o and e ζ,v As a condition, z is deterministically decoded into a continuous action C in the original action space. For discrete actions, a nearest neighbor search is performed in the embedding table to decode it into

[0103] The encoding and decoding process of CVAE can be expressed as the following formula:

[0104] Encoding:e ζ,v =E ζ (v),z~q φ (·|C,o,e ζ,v )

[0105]

[0106]

[0107] The Metaverse system learns the embedding table and CVAE by minimizing the following Loss function:

[0108]

[0109] Where, L HyAR represents the loss function of the hybrid action representation model; φ represents the encoder parameters of the conditional variational autoencoder; ψ represents the decoder parameters of the conditional variational autoencoder; ζ represents the parameters of the embedding table; is the expected value, which represents the average of all samples; α represents the weight hyperparameter, which is used to balance the reconstruction error term; Indicates continuous action C i Instead of the refactored version The L2 norm square between is the reconstruction error; λ represents the weight hyperparameter, which is used to balance the prediction error term; Indicates actual resource price changes; represents the predicted resource price change; η represents the weight hyperparameter used to balance the KL divergence term; D KL stands for Kullback-Leibler divergence, which measures the difference between two probability distributions; represents the encoder of the conditional variational autoencoder, which outputs the parameters of a Gaussian distribution to represent the latent action space;

[0110] Represents a labeled normal distribution, which serves as the reference distribution for the KL divergence.

[0111] That is, the first term represents the mean squared reconstruction error between the original continuous actions and the reconstructed continuous actions; the third term represents the KL divergence between the latent variable and the standard Gaussian distribution; and the second term represents the mean squared prediction error between the actual metaverse user observations (edge ​​server resource prices) and their predicted values.

[0112] 2) Strategy learning in latent space

[0113] The Metaverse system introduces a multi-agent dual-delay deep deterministic policy gradient algorithm for policy learning in latent space. It can not only ensure that Metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience by optimizing resource leasing and allocation strategies, but also comprehensively consider the multi-dimensional resource requirements and limitations such as computing, storage, network bandwidth, etc. required for the operation of the Metaverse to achieve efficient use of resources.

[0114] In the multi-agent dual-delayed deep deterministic policy gradient algorithm, each Metaverse user has a policy network and two value networks. In the Metaverse system scenario, all Metaverse users have the same observation space, action space, and benefit function form, so the Metaverse system uses a parameter sharing framework to train all networks more efficiently.

[0115] For the value network, the Metaverse system uses the following Loss function to learn:

[0116]

[0117] Where E is the expectation operator; is the lth value network's potential actions for the current state s and all agents (e 1:I ,z 1:I )’s value estimate; i is the timing difference target, which can be expressed as follows:

[0118]

[0119] In the formula, r i is the immediate reward obtained by the ith agent at time step t; γ is a discount factor used to determine the importance of future rewards relative to current rewards; is the value network's policy output for the next state s' and all agents estimated value.

[0120] For the policy network, the Metaverse system uses deterministic policy gradient to learn:

[0121]

[0122] Step 4: Each Metaverse user calculates a bid based on the current resource pricing of the optimal edge server it is connected to and the amount of resources it requests, and sends it to the corresponding edge server. After receiving all the bids, the edge server updates its resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

[0123] Specifically, the rendering resource auction mechanism of the edge server is as follows:

[0124] The decision of the Metaverse user includes the quantity of the three rendering resources requested, but because a large number of Metaverse users request the resources of the edge server at the same time, this will cause contention for resources among Metaverse users. The edge server needs to design a reasonable pricing-based resource leasing mechanism to allocate its limited resources. At the same time, the resource pricing method also needs to play a role in incentivizing Metaverse users to participate in long-term rendering cooperation.

[0125] The Metaverse system designs the resource leasing mechanism as an auction, where all edge servers are auctioneers and all Metaverse users are bidders. Each Metaverse user calculates a bid based on the current resource pricing of the edge server and the number of resources requested, and sends it to the corresponding edge server. After receiving all the bids, the edge server updates the resource price using the following resource pricing method, calculates the amount of resources that each Metaverse user can actually lease, and then leases the resources to the corresponding Metaverse user:

[0126]

[0127]

[0128] In the formula, represents the price set by edge server j for resource type h at time t+1; represents the operating cost borne by edge server j for using unit resource h; U j,t represents the set of all users accessing edge server j at time t; V represents the bid of Metaverse user i for resource h at time t; j h represents the total capacity of resource type h owned by edge server j.

[0129] It has been proven that this resource pricing method satisfies the following three conditions, and is therefore reasonable and can incentivize long-term cooperation among Metaverse users:

[0130] 1) Individual rationality condition. The profit of the edge server is not less than 0:

[0131]

[0132] Where U j,t represents the set of all users accessing edge server j at time t; represents the price set by edge server j for GPU resources at time t; represents the operating cost borne by edge server j for using unit GPU resources; g′ i，t represents the amount of GPU resources leased by Metaverse user i from edge server j at time t; f′ i，t represents the amount of CPU resources leased by Metaverse user i from edge server j at time t; represents the price set by edge server j for bandwidth resources at time t; represents the operating cost borne by edge server j for using unit bandwidth resources; w′ i，tIt represents the amount of bandwidth resources leased by Metaverse user i from edge server j at time t.

[0133] 2) Resource capacity limitation: For each edge server, the sum of the resources allocated in each time round cannot exceed the total amount of the resource it owns.

[0134] 3) Participation incentive conditions. The Metaverse market is still in its infancy. In order to attract more Metaverse users to participate in the social activities of the Metaverse, the resource price of the edge server should be as low as possible after satisfying individual rational conditions and resource capacity constraints.

[0135] The present invention uses edge server resources for background rendering while processing foreground interactions locally. The Metaverse system dynamically adjusts resource allocation through user behavior analysis and resource leasing modeling. It also introduces a user experience quality model based on the Weber-Fechner law and a multi-agent reinforcement learning algorithm to optimize resource request and allocation strategies, ensure that users obtain high-quality visual and interactive experiences, achieve efficient resource utilization and reasonable market pricing, thereby improving the performance of the entire Metaverse platform and Metaverse user satisfaction.

[0136] Exemplarily, when the method is applied to a metaverse system, the metaverse system divides the virtual reality scene into two parts: foreground interaction and background environment; the metaverse system performs mobility analysis based on user behavior, and the edge server only needs to pre-render the background environment on the user's mobile path; the metaverse system models the resource leasing problem as a two-layer decision problem, including a partially observable Markov decision process at the user layer and a resource pricing problem at the edge server layer; the metaverse system uses the Weber-Fechner law to characterize the user's subjective evaluation of picture quality, and multiplies it with the objective pre-rendering delay to establish a user experience quality model; the metaverse system uses a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables to solve the decision problem at the user layer and obtain the optimal edge server and resource quantity.

[0137] Exemplarily, when the method is applied to the Metaverse user layer in the Metaverse system, the user device calculates a bid based on the resource pricing of the optimal edge server to which it is connected and the number of resources it requests; the user device sends the bid to the corresponding edge server; the user device receives resource leasing feedback from the edge server and performs foreground interactive rendering.

[0138] Exemplarily, when the method is applied to the edge server layer in the metaverse system, the edge server updates its resource price after receiving all bids; the edge server calculates the amount of resources that each user can actually rent; the edge server rents the resources to the corresponding metaverse user; the edge server pre-renders the background environment on the user's moving path based on the user's mobility analysis results.

[0139] In another embodiment of the present invention, a device for collaborative rendering of a metaverse virtual reality scene based on edge computing is provided, which is used to implement the above-mentioned method for collaborative rendering of a metaverse virtual reality scene based on edge computing, and is applied to a metaverse system including an edge server layer and a metaverse user layer, wherein the edge server layer includes multiple edge servers, and the metaverse user layer includes multiple metaverse users. The rendering method includes:

[0140] The division module is used to divide the Metaverse virtual reality scene into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer by leasing the resources of the edge server, and the foreground interaction is rendered locally by the Metaverse user layer.

[0141] A building module is used to construct the dynamic resource leasing problem between multiple edge servers and multiple metaverse users into a two-level decision problem, wherein the two-level decision problem includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level.

[0142] The solution module is used to use a multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables and a pre-built Metaverse user experience quality model to solve the partially observable Markov decision process problem of the Metaverse user layer, and obtain the optimal edge server for Metaverse user access and the optimal GPU, CPU and bandwidth resource requirements.

[0143] The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected to it, as well as the Metaverse user's own optimal GPU, CPU, and bandwidth resource requirements, and send it to the corresponding edge server; after the edge server receives the bids from all Metaverse users, it updates its own resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

[0144] All relevant contents of each step involved in the embodiment of the aforementioned method for collaborative rendering of a metaverse virtual reality scene based on edge computing can be referred to the functional description of the functional module corresponding to the device for collaborative rendering of a metaverse virtual reality scene based on edge computing in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present invention can be integrated into a processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0145] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a metaverse virtual reality scene collaborative rendering method based on edge computing.

[0146] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the above-mentioned embodiment regarding a method for collaborative rendering of a metaverse virtual reality scene based on edge computing.

[0147] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0151] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0152] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for collaborative rendering of a metaverse virtual reality scene based on edge computing, characterized in that: Applied to a metaverse system including an edge server layer and a metaverse user layer, the edge server layer including a plurality of edge servers, the metaverse user layer including a plurality of metaverse users, the rendering method comprising: The Metaverse virtual reality scene is divided into foreground interaction and background environment. The background environment is remotely rendered by the Metaverse user layer by leasing resources from the edge server, and the foreground interaction is rendered locally by the Metaverse user layer. The dynamic resource leasing problem between multiple edge servers and multiple metaverse users is constructed as a two-level decision problem, which includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level. Using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, we solve the partially observable Markov decision process problem at the Metaverse user layer and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements. Each Metaverse user calculates a bid based on the current resource pricing of the optimal edge server to which it is connected, as well as the Metaverse user's own optimal GPU, CPU, and bandwidth resource requirements, and sends it to the corresponding edge server. After receiving the bids from all Metaverse users, the edge server updates its own resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

2. According to the edge computing-based metaverse virtual reality scene collaborative rendering method of claim 1, it is characterized in that: The Metaverse user experience quality model uses the Weber-Fechner law to characterize the Metaverse user's subjective evaluation of picture quality, and is constructed by multiplying it with the objective pre-rendering delay.

3. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 2, characterized in that: The Weber-Fechner law is used to characterize the subjective evaluation of the image quality by Metaverse users, specifically: In the formula, Represents the subjective evaluation results of the Metaverse users on the picture quality; I i,n represents the interest of Metaverse user i in object n; r i,n k represents the resolution density that Metaverse user i requires the edge server to render object n; i represents the compression ratio requested by Metaverse user i; r th and k th Represents the minimum acceptable resolution density and compression ratio; N i represents the total number of virtual objects that Metaverse user i is interested in in the background environment; κ represents the adjustment coefficient, which is used to balance the impact of visual quality and rendering delay on the quality of Metaverse user experience.

4. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 3 is characterized in that: The objective pre-rendering delay is specifically expressed as: In the formula, Indicates the objective pre-rendering delay; T unit Represents the time it takes for a Metaverse user to move between adjacent grid points; It represents the time required for Metaverse user i to request edge server j to pre-render the panorama.

5. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 4, characterized in that: The time required for the metaverse user i to request the edge server j to pre-render the panorama is specifically: In the formula, represents the request time that Metaverse user i needs to request edge server j to render the corresponding background environment; d ij represents the network distance between Metaverse user i and edge server j; β' represents the time it takes for a request to pass through a single router; represents the rendering time of Metaverse user i renting edge server j to render the background environment using its GPU resources; g i,j represents the number of GPU resources of edge server j rented by Metaverse user i; u i,n represents the pixel surface area that Metaverse user i requires the edge server to render object n; represents the encoding time of Metaverse user i renting edge server j to encode the background environment frame using its CPU resources; f i,j represents the amount of CPU resources of edge server j rented by Metaverse user i; represents the number of CPU cycles required to encode a single pixel in edge server j; represents the transmission time of edge server j transmitting the encoded background environment frame to metaverse user i; w i,j represents the amount of output bandwidth resources of edge server j rented by Metaverse user i; h represents the size of a single pixel.

6. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 5, characterized in that: The Metaverse User Experience Quality Model is specifically expressed as: Where, QoE i Represents the quality of Metaverse user experience.

7. The method for collaborative rendering of a metaverse virtual reality scene based on edge computing according to claim 1, characterized in that: The multi-agent reinforcement learning algorithm with discrete-continuous mixed decision variables includes: Construct a latent representation space, map the original discrete-continuous hybrid actions into the latent representation space, then use the classic multi-agent reinforcement learning algorithm to learn the latent strategy in the latent representation space, and then decode the latent strategy into actions in the original space to interact with the environment; The construction of the latent representation space is specifically as follows: In a multi-agent environment, the action space faced by each metaverse user includes both discrete choices of edge servers to connect to and continuous decisions on the rendering quality and amount of resources requested; The strategy learning in the latent space is: A multi-agent dual-delay deep deterministic policy gradient algorithm is introduced for policy learning in the latent space. By optimizing resource leasing and allocation strategies, it ensures that Metaverse users can obtain the required computing and rendering resources to achieve high-quality visual and interactive experience.

8. A metaverse virtual reality scene collaborative rendering device based on edge computing, characterized in that: Applied to a metaverse system including an edge server layer and a metaverse user layer, the edge server layer including a plurality of edge servers, the metaverse user layer including a plurality of metaverse users, the rendering method comprising: A partitioning module is used to divide the Metaverse virtual reality scene into a foreground interaction and a background environment. The background environment is remotely rendered by the Metaverse user layer by leasing resources from the edge server, and the foreground interaction is rendered locally by the Metaverse user layer. A construction module is used to construct a dynamic resource leasing problem between multiple edge servers and multiple metaverse users as a two-level decision problem, wherein the two-level decision problem includes a partially observable Markov decision process problem at the metaverse user level and a resource pricing problem at the edge server level; A solution module is used to solve the partially observable Markov decision process problem of the Metaverse user layer by using a multi-agent reinforcement learning algorithm with discrete-continuous hybrid decision variables and a pre-built Metaverse user experience quality model, and obtain the optimal edge server accessed by Metaverse users and the optimal GPU, CPU, and bandwidth resource requirements; The calculation module is used for each Metaverse user to calculate the bid based on the current resource pricing corresponding to the optimal edge server connected to it, as well as the Metaverse user's own optimal GPU, CPU, and bandwidth resource requirements, and send it to the corresponding edge server; after the edge server receives the bids from all Metaverse users, it updates its own resource price, calculates the amount of resources that each Metaverse user can actually rent, and rents the resources to the corresponding Metaverse user.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a method for collaborative rendering of a metaverse virtual reality scene based on edge computing as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for collaborative rendering of a metaverse virtual reality scene based on edge computing as described in any one of claims 1 to 7.

Images