A QoE-based approach to ensuring fair resource optimization using digital twins.

Abstract

This invention discloses a QoE-assured fair resource optimization method based on digital twins, comprising: initializing mode selection; obtaining the transmission power, GPU cycle frequency, and computation time of the HDM client in TR mode, VPR mode, and based on the mode selection decision; obtaining the HDM client mode selection decision strategy based on the transmission power, VPR mode, GPU cycle frequency, and computation time; determining the target value based on the transmission power, VPR mode, GPU cycle frequency, computation time, and mode selection decision strategy; and repeating the above steps according to preset thresholds and judgment conditions until the jointly optimized mode selection decision, transmission power in TR mode, transmission power in VPR mode, GPU cycle frequency, and computation time are obtained. This invention, through joint optimization, can maximize the QoE of head-mounted display clients and achieve fair resource allocation.

Description

Technical Field

[0001] This invention belongs to the field of wireless virtual reality technology, specifically relating to a QoE-based fair resource optimization method based on digital twins. Background Technology

[0002] With the improvement of electronic product performance in recent years, Virtual Reality (VR) has received widespread attention from industry and academia for its extraordinary immersive experience. The significant advancements in VR technology have led to the widespread development of wireless VR technologies in education, video games, culture, and healthcare. In wireless VR systems, head-mounted displays (HDMs) can serve as VR devices, allowing users to enjoy a fully immersive experience. However, due to the heterogeneity of HDMs in terms of computing power and network conditions, high-resolution viewport rendering performed solely by HDMs cannot meet the quality and latency requirements of VR videos. Furthermore, the computationally intensive tasks performed by HDMs can lead to increased computing costs, such as reduced battery life and soaring unit prices. To reduce the computational load on HDMs, edge computing is a promising technology that can be applied to viewport rendering.

[0003] Providing high-resolution, low-latency immersive VR services poses significant challenges to wireless edge networks. These challenges mainly manifest in three aspects: 1) High bandwidth consumption: During VR video viewing, users need to constantly move the HDM (High-Degree Display) to achieve the best viewing experience, and frequent real-time viewport requests increase bandwidth consumption; 2) Latency sensitivity: It is well known that the latency from VR motion to photons must be strictly less than 20ms to alleviate motion sickness, thus computation and transmission in the network may increase latency; 3) Heterogeneous computing capabilities: Heterogeneous HDM devices affect the rendering and display of VR videos. To address these challenges, VR video services in edge networks have been extensively studied in recent years. In edge computing-assisted wireless VR systems, the Quality of Experience (QoE) of VR video services is a crucial indicator for measuring user service perception, controlled by numerous factors such as rendering, transmission, and display. Existing work has designed QoE models for VR video transmission in wireless networks to maximize the average QoE for users. Most of these models focus on improving the performance of time-invariant VR video services in wireless networks. However, due to the dynamic and time-varying nature of network conditions, static resource management decisions in wireless VR systems may produce suboptimal results. Therefore, constructing a real-time and accurate VR video transmission model is worthy of attention. Digital Twin (DT) technology, as a potential solution, is effective in addressing this issue. DT is a digital description of a physical entity (PE). Through real-time interaction between DT and PE, a precise dynamic mapping of physical space can be achieved. In wireless VR video streaming systems, DT technology can establish a more accurate VR video transmission model through real-time data analysis, which can be used for resource allocation management and improve QoE.

[0004] However, while enjoying the surprises that DT brings to wireless VR systems, it is necessary to rethink the research challenges that focus primarily on communication and computing. First, demonstrating real-time interaction between DT and PE is crucial for accurately characterizing the performance of VR video transmission models. Second, the heterogeneous computing capabilities of HDM clients affect the execution efficiency of video rendering tasks, and a reasonable strategy is a prerequisite for meeting VR video latency requirements. Finally, from the user's perspective, they expect the services they receive in the system to be as equitable as possible; therefore, it is urgent to develop a reasonable resource allocation system to ensure fairness in usage. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, the present invention is achieved through the following technical solution:

[0006] This invention provides a QoE-based fair resource optimization method based on digital twins, the optimization method comprising:

[0007] Step 1: Initialize the mode selection for all HDM clients. make m≤m max ,in, N represents the total number of HDM clients, and n represents the number of HDM clients out of the N HDM clients. Let a represent the set of the HDM clients n. n This indicates the mode selection for the HDM client n, when a n When in TR mode, it indicates that the HDM client n is in slice re-encoding mode, when a n In VPR mode, it indicates that the HDM client n is in window rendering encoding mode, and m represents the iteration number. max Let m = 0, representing the threshold for the number of iterations in this round.

[0008] Step 2: Select based on the mode of all HDM clients. Obtain the transmission power of the HDM client in TR mode during the m-th iteration. and the transmission power of the HDM client in VPR mode in, This represents the transmission power of HDM client n in TR mode during the m-th iteration. This represents the transmission power of HDM client n in VPR mode during the m-th iteration;

[0009] Step 3: Select based on the mode of all HDM clients. and the transmission power of the HDM client in the TR mode Obtain the GPU cycle frequency of the HDM client in TR mode during the m-th iteration. in, This represents the GPU cycle frequency required for the HDM client n in TR mode to render video in the m-th iteration;

[0010] Step 4: Select based on the mode of all HDM clients. Obtain the computation time of the HDM client in VPR mode. in, This represents the rendering computation time of HDM client n in VPR mode during the m-th iteration;

[0011] Step 5: Based on the transmission power The transmission power The GPU cycle frequency and the calculation time Get the optimization mode selection for all HDM clients

[0012] Step 6: Based on the transmission power The transmission power The GPU cycle frequency The calculation time and the optimization mode selection Determine the target value for the m-th iteration.

[0013] Step 7: When m < 1, let m = m + 1, and repeat steps 2 to 7; when m ≥ 1, if Then the current iteration ends, and the optimal solution is: as well as like And m < m max Let m = m + 1, and repeat steps 2 to 7, where ε represents a preset threshold. This represents the optimal solution for mode selection by all HDM clients in this iteration. This represents the optimal transmission power solution for the HDM client in TR mode during this iteration. This represents the optimal transmission power solution for the HDM client in VPR mode during this iteration. This represents the optimal GPU cycle frequency required for an HDM client to render video in TR mode. This represents the optimal solution for rendering computation time for an HDM client in VPR mode.

[0014] In one embodiment of the present invention, step 2 includes:

[0015] When HDM client n is in TR mode, the selection is based on the modes of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method:

[0016]

[0017]

[0018]

[0019]

[0020] The transmission power of the HDM client in TR mode is obtained in the m-th iteration.

[0021]

[0022] in, and The input parameters for latency and energy efficiency of the QoE client digital twin are obtained based on the nonlinear regression method. ω represents the bitstream size transmitted to the HDM client n. n,TR This represents the bandwidth consumption of HDM client n. P represents the channel gain of the HDM client n. max This indicates the maximum transmit power of the base station. Indicates the transmission rate required by the HDM client n; μ, ν n and γ n Let V and V represent Lagrange multipliers, respectively, and μ ≥ 0. TR Indicates the video quality in TR mode;

[0023] When HDM client n is in VPR mode, the selection is based on the modes of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method:

[0024]

[0025]

[0026]

[0027] Obtain the transmission power of the HDM client in VPR mode during the m-th iteration.

[0028]

[0029]

[0030] in, ω represents the bitstream size transmitted to the HDM client n in VPR mode. n,VPR This represents the bandwidth consumption of HDM client n in VPR mode. V represents the channel gain of the HDM client n in VPR mode. VPR This indicates the video quality in VPR mode.

[0031] In one embodiment of the present invention, step 3 includes:

[0032] Based on the mode selection of all HDM clients. The transmission power Solving optimization problems using heuristic search algorithms:

[0033] and Obtain the GPU cycle frequency in TR mode

[0034]

[0035]

[0036]

[0037] in, It is an auxiliary variable. and The input parameters ω for the QoE client digital twin are obtained based on nonlinear regression, which are used to obtain the parameters for latency and energy efficiency. n,TR This represents the bandwidth consumption of HDM client n in TR mode, where G(*) denotes a function of *. This represents the GPU cycle frequency required for an HDM client n in TR mode to render video during the m-th iteration, where max represents the maximum frequency. n {*} represents the largest value in set *. This indicates the bitstream size transmitted to the HDM client n in TR mode. k represents the channel gain of the HDM client n in TR mode. h Indicates the effective switched capacitor of the HDM client, c n This indicates the processing density required for the edge server to render the video from the HDM client n. and V represents the minimum and maximum GPU cycle frequencies required by the HDM client n, respectively. n This indicates the video quality obtained by the HDM client n.

[0038] In one embodiment of the present invention, step 4 includes:

[0039] Based on the mode selection of all HDM clients. Solving the computation time in VPR mode Optimization issues:

[0040]

[0041]

[0042] and And by introducing an auxiliary variable ξ, the problem is transformed into:

[0043]

[0044]

[0045]

[0046]

[0047] The computation time under VPR mode is obtained by solving the above optimization problem using the barrier function method. in, This indicates that the input parameters for time delay of the QoE client digital twin are obtained based on the nonlinear regression method. k represents the sensitivity of the HDM client n to energy efficiency. e Indicates the effective switched capacitor of the edge server, c n I represents the processing density required for the edge server to render the video for the HDM client n. n This indicates the size of the rendered video segment for the HDM client n. This represents the bitstream size transmitted to HDM client n in VPR mode, and F represents the total computing power of the edge servers corresponding to the N HDM clients.

[0048] In one embodiment of the present invention, step 5 includes:

[0049] Step 5.1: Initialize the set Ω = {C1, C2, ... C} N} Select the HDM client set in VPR mode Select the set of HDM clients in TR mode. The set of all HDM clients is R = {1, 2, ..., N}, where, To obtain the input parameters of QoE client digital twin regarding video quality, latency, and energy efficiency based on nonlinear regression, This indicates the transmission time of the n-bit stream from the HDM client in TR mode. This indicates the transmission time of the HDM client's n-stream in VPR mode. η represents the computation time for client n to render the video. n TT η represents the transmission energy efficiency of HDM client n under TR. n TV This indicates the transmission energy efficiency of HDM client n in VPR mode. This represents the computational energy efficiency of HDM client n. This represents the computational energy efficiency of the edge server, where T represents the number of video segments. This represents the viewport width of the HDM client n. Indicates the height of the HDM client n;

[0050] Step 5.2: Initialize n = 1;

[0051] Step 5.3, based on the above The The and stated C was calculated n When C n <0, then a n =0、 When n = n + 1, repeat step 5.3 until n = N;

[0052] Step 5.4, when get It contains the maximum value C n If n, then a n =1、 Repeat step 5.4 when Perform step 5.5, where ω n,VPR ω represents the bandwidth consumption of HDM client n in TR mode. n,VPR B represents the bandwidth consumption of the HDM client n in VPR mode, and B represents the system bandwidth.

[0053] Step 5.5, when At that time, a n =0, in Repeat step 5.5 when When the execution ends;

[0054] Step 5.6: Obtain the optimization mode selection.

[0055] In one embodiment of the present invention, step 6 includes:

[0056] According to the transmission power The transmission power The GPU cycle frequency The calculation time and the optimization mode selection Determine the target value for the m-th iteration. HDM client n's QoE model Q n Represented as:

[0057] Q n =λ n,1 V n -λ n,2 D n -λn,3 η n By introducing a digital twin layer, determine

[0058] in, Let λ represent the QoE model of the HDM client n in the m-th iteration of the digital twin layer. n,1 , λ n,2 , λ n,3 These represent the sensitivity of the HDM client n to video quality, service latency, and energy efficiency, respectively. V is the QoE model evaluation parameter of the client-side digital twin, calculated using a nonlinear regression method. n D represents the video quality obtained by the HDM client n. n η represents the service latency of HDM client n. n This indicates the energy efficiency of the HDM client n.

[0059] In one embodiment of the present invention, when the HDM client n is in TR mode, the video clip transmitted to the HDM client n is rendered on the HDM client n; when the HDM client n is in VPR mode, the video clip transmitted to the HDM client n is rendered on the edge server corresponding to the N HDM clients.

[0060] In one embodiment of the present invention, when a n When a = 0, it indicates that the HDM client n is in VPR mode; when a n When = 1, it indicates that the HDM client n is in TR mode.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] This invention presents a QoE-guaranteed fair resource optimization method based on digital twins. To achieve effective resource allocation and ensure user QoE fairness as much as possible, a heuristic and efficient algorithm is developed, and a fair resource allocation framework for a wireless VR system supporting digital twins (DT) is proposed. To ensure user fairness in the evaluation system, a QoE model is constructed, which is a linearly weighted combination of multiple factors such as video quality, service latency, and energy efficiency. At the same time, the optimization variables are decoupled by jointly optimizing the coding mode selection, transmit power, edge server rendering computation time, and client rendering GPU cycle frequency, thereby achieving the goal of effective resource allocation and minimizing average latency. This algorithm has good convergence and reduces the computational complexity of the algorithm.

[0063] Furthermore, the joint optimization scheme proposed in this invention exhibits better performance in terms of average latency compared to existing technologies.

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0065] Figure 1 This is an application scenario diagram of a QoE-based fair resource optimization method based on digital twins provided in an embodiment of the present invention;

[0066] Figure 2 This is a flowchart illustrating a QoE-based fair resource optimization method based on digital twins provided in an embodiment of the present invention;

[0067] Figure 3 (a) in the figure is a schematic diagram showing the relationship between the total computing power F of the edge server provided in the embodiment of the present invention and the energy efficiency in the worst case. Figure 3 (b) is a schematic diagram showing the relationship between the total computing power F of the edge server provided in this embodiment of the invention and the QoE of the HDM client in the worst case.

[0068] Figure 4 This is a schematic diagram of the network and the target QoE value of each client under different schemes provided in the embodiments of the present invention;

[0069] Figure 5 This is a data illustration of the target QoE values ​​of the network, worst client, and best client under different schemes provided in the embodiments of the present invention;

[0070] Figure 6 This refers to the service latency and the maximum GPU cycle frequency f of the HDM client under different schemes provided in the embodiments of the present invention. max Relationship diagram;

[0071] Figure 7 This is a schematic diagram illustrating the relationship between video quality and bandwidth B under different schemes provided in the embodiments of the present invention;

[0072] Figure 8 This is a schematic diagram illustrating the relationship between QoE and the number of HDM clients N under different schemes provided in the embodiments of the present invention;

[0073] Figure 9 This is the worst-case energy efficiency and maximum transmit power P of the client provided in this embodiment of the invention. max A diagram illustrating the relationship between the two. Detailed Implementation

[0074] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the solution according to the present invention is provided in conjunction with the accompanying drawings and specific embodiments.

[0075] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0076] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0077] Please see Figure 1 , Figure 1 This is an application scenario diagram of a QoE-based fair resource optimization method based on digital twins provided by an embodiment of the present invention. The application scenario mainly includes: a cloud server and physical entities. The physical entities include a base station (BS) and N HDM clients communicating with the base station, where N is a positive integer greater than or equal to 1.

[0078] Specifically, the cloud server stores the encoded 360° video, each segment being broken down into T blocks, where T is a positive integer greater than or equal to one. Edge servers and network controllers are deployed in the base station, and the digital twin layer is also deployed on the edge server, which corresponds to each of the N HDM clients. The edge server provides available computing resources to these N HDM clients. The edge server retrieves these video segments, broken down into T blocks, by accessing the cloud server. Considering the computing power of the HDM clients, the edge server uses either Tile Rewriting (TR) encoding or Viewport Rendering (VPR) encoding. When the HDM client has strong computing power, TR mode is used to encode the video segments, meaning the video segments are rendered on the HDM client; when the HDM client has weak computing power, VPR mode is used to encode the video segments, meaning the video segments are rendered on the edge server corresponding to the N HDM clients. The processed video is encoded into an HEVC bitstream and transmitted to the HDM clients for display. The digital twin layer includes N client digital twins (CDTs) corresponding to the N HDM clients, allowing... n represents the number of HDM clients out of N HDM clients. This represents the set of HDM clients n. Each CDT can acquire data from the corresponding HDM client in real time to construct a virtual QoE model. The network controller can realize data exchange and QoE model updates between physical entities and the digital twin layer.

[0079] Please see Figure 2 , Figure 2 This is a flowchart illustrating a QoE-based fair resource optimization method based on digital twins provided in an embodiment of the present invention. The method includes the following steps:

[0080] Step 1: Initialize the mode selection for all HDM clients. make m≤m max ;

[0081] Specifically, N represents the total number of HDM clients, and n represents the number of HDM clients out of the N HDM clients. Let a represent the set of HDM clients n. n This indicates the mode selection for the HDM client n, when a n When in TR mode, it indicates that HDM client n is in TR mode. When HDM client n is in TR mode, the video segment transmitted to HDM client n is rendered on HDM client n; when an In VPR mode, it means that HDM client n is in VPR mode. When HDM client n is in VPR mode, the video clip transmitted to HDM client n is rendered on the edge server corresponding to N HDM clients. m represents the iteration number. max Let m = 0, which represents the threshold for the number of iterations in this round.

[0082] In this embodiment, when a n When a = 0, it indicates that HDM client n is in VPR mode, meaning that the video segment transmitted to HDM client n is rendered on the edge server corresponding to N HDM clients; when a n When = 1, it indicates that the HDM client n is in TR mode, meaning that the video segment transmitted to the HDM client n is rendered on the HDM client n. In other embodiments, other numbers, letters, or strings can be used to replace 'a' in the above-mentioned format. n =0 and a n =1, to represent VPR mode and TR mode respectively.

[0083] In this embodiment of the invention, the base station consists of a Mobile Edge Computing (MEC) server and 15 HDM clients, i.e., N=15, with a coverage area of ​​2×2km. 2 .

[0084] Step 2: Select the mode according to all HDM clients Obtain the transmission power of the HDM client in TR mode during the m-th iteration. and the transmission power of the HDM client in VPR mode

[0085] According to the formula and Obtain the transmission power of the HDM client in TR mode during the m-th iteration. and the transmission power of the HDM client in VPR mode Specifically, This represents the transmission power of HDM client n in TR mode during the m-th iteration. This represents the transmission power of HDM client n in VPR mode during the m-th iteration.

[0086] Obtain the transmission power of the HDM client in TR mode Specifically, this includes: when HDM client n is in TR mode, selecting based on the modes of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method:

[0087]

[0088]

[0089]

[0090]

[0091] The transmission power of the HDM client in TR mode is obtained in the m-th iteration.

[0092]

[0093] in, and The input parameters for latency and energy efficiency of the QoE client digital twin are obtained based on the nonlinear regression method. ω represents the bitstream size transmitted to the HDM client n. n,TR This represents the bandwidth consumption of HDM client n. P represents the channel gain of the HDM client n. max This indicates the maximum transmit power of the base station. Indicates the transmission rate required by the HDM client n; μ, ν n and γ n Let V and V represent Lagrange multipliers, respectively, and μ ≥ 0. TR Indicates the video quality in TR mode.

[0094] Obtain the transmission power of the HDM client in VPR mode. Specifically, this includes: when HDM client n is in VPR mode, selecting based on the mode of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method:

[0095]

[0096]

[0097]

[0098] Obtain the transmission power of the HDM client in VPR mode during the m-th iteration.

[0099]

[0100]

[0101] Furthermore, ω represents the bitstream size transmitted to the HDM client n in VPR mode. n,VPR This represents the bandwidth consumption of HDM client n in VPR mode. V represents the channel gain of the HDM client n in VPR mode. VPR This indicates the video quality in VPR mode.

[0102] Step 3: Select the mode according to all HDM clients Transmission power of HDM clients in TR mode Obtain the GPU cycle frequency of the HDM client in TR mode during the m-th iteration.

[0103] Specifically, This represents the GPU cycle frequency required for the HDM client n in TR mode to render video during the m-th iteration.

[0104] Furthermore, select the mode based on all HDM clients. Transmission power Solving optimization problems using heuristic search algorithms:

[0105] and Obtain the GPU cycle frequency in TR mode

[0106]

[0107]

[0108]

[0109] in, It is an auxiliary variable. and The input parameters ω for the QoE client digital twin are obtained based on nonlinear regression, which are used to obtain the parameters for latency and energy efficiency. n,TR This represents the bandwidth consumption of HDM client n in TR mode, where G(*) denotes a function of *. This represents the GPU cycle frequency required for an HDM client n in TR mode to render video during the m-th iteration, where max represents the maximum frequency. n {*} represents the largest value in set *. This indicates the bitstream size transmitted to the HDM client n in TR mode. k represents the channel gain of the HDM client n in TR mode. h Indicates the effective switched capacitor of the HDM client, c nThis indicates the processing density required for the edge server to render the video from the HDM client n. and V represents the minimum and maximum GPU cycle frequencies required by the HDM client n, respectively. n This indicates the video quality obtained by the HDM client n.

[0110] Step 4: Select the mode based on all HDM clients. Obtain the computation time of the HDM client in VPR mode.

[0111] Specifically, This represents the rendering computation time of the HDM client n in VPR mode during the m-th iteration.

[0112] Obtain the computation time of the HDM client in VPR mode. Specifically, it includes:

[0113] First, select based on the mode of all HDM clients. Solving the computation time in VPR mode Optimization issues:

[0114]

[0115]

[0116]

[0117] Subsequently, by introducing an auxiliary variable ξ, the problem is transformed into:

[0118]

[0119]

[0120]

[0121]

[0122] Solving the above optimization problem using the barrier function method yields the computation time under the VPR mode. ， in, This indicates that the input parameters λ for the QoE client digital twin are obtained based on a nonlinear regression method. n,3 k represents the sensitivity of the HDM client n to energy efficiency. e Indicates the effective switched capacitor of the edge server, c n I represents the processing density required for the edge server to render the video for the HDM client n. nThis indicates the size of the rendered video segment for the HDM client n. This represents the bitstream size transmitted to HDM client n in VPR mode, and F represents the total computing power of the edge servers corresponding to N HDM clients.

[0123] Step 5: Based on transmission power Transmission power GPU cycle frequency and calculation time Get the optimization mode selection for all HDM clients

[0124] Furthermore, obtain the optimization mode selection for all HDM clients. Specifically, it includes:

[0125] Step 5.1: Initialize the set Ω = {C1, C2, ... C} N} Select the HDM client set in VPR mode Select the set of HDM clients in TR mode. The set of all HDM clients R = {1,2,...N};

[0126] Specifically, To obtain the input parameters of QoE client digital twin regarding video quality, latency, and energy efficiency based on nonlinear regression, This indicates the transmission time of the n-bit stream from the HDM client in TR mode. This indicates the transmission time of the HDM client's n-stream in VPR mode. η represents the computation time for client n to render the video. n TT η represents the transmission energy efficiency of HDM client n under TR. n TV This indicates the transmission energy efficiency of HDM client n in VPR mode. This represents the computational energy efficiency of HDM client n. This represents the computational energy efficiency of the edge server, where T represents the number of video segments. This represents the viewport width of the HDM client n. This represents the height of the HDM client n.

[0127] Step 5.2: Initialize n = 1;

[0128] Step 5.3, based on and C was calculated n When C n <0, then an =0、 When n = n + 1, repeat step 5.3 until n = N;

[0129] Step 5.4, when get It contains the maximum value C n If n, then a n =1、 Repeat step 5.4 when Proceed to step 5.5;

[0130] Specifically, ω n,VPR ω represents the bandwidth consumption of HDM client n in TR mode. n,VPR B represents the bandwidth consumption of the HDM client n in VPR mode, and B represents the system bandwidth.

[0131] Step 5.5, when At that time, a n =0, in Repeat step 5.5 when When the execution ends;

[0132] Step 5.6: Obtain the optimization mode selection.

[0133] Step 6: Based on transmission power Transmission power GPU cycle frequency Calculation time and optimization mode selection Determine the target value for the m-th iteration.

[0134] Specifically, the QoE model of HDM client n is Q n It is represented as:

[0135] Q n =λ n,1 V n -λ n,2 D n -λ n,3 η n By introducing a digital twin layer, determine for:

[0136]

[0137] Furthermore, Let λ represent the QoE model of the HDM client n in the m-th iteration of the digital twin layer. n,1 , λ n,2 , λn,3 These represent the sensitivity of the HDM client n to video quality, service latency, and energy efficiency, respectively. To calculate the digital twin evaluation parameters of the CDT QoE model using the nonlinear regression method, V n D represents the video quality obtained by the HDM client n. n η represents the service latency of HDM client n. n This indicates the energy efficiency of the HDM client n.

[0138] Step 7: When m < 1, let m = m + 1, and repeat steps 2 to 7; when m ≥ 1, if Then the current iteration ends, and the optimal solution is: as well as

[0139] Specifically, ε represents a preset threshold. This represents the optimal solution for mode selection by all HDM clients in this iteration. This represents the optimal transmission power solution for the HDM client in TR mode during this iteration. This represents the optimal transmission power solution for the HDM client in VPR mode during this iteration. This represents the optimal GPU cycle frequency required for an HDM client to render video in TR mode. This represents the optimal solution for rendering computation time for an HDM client in VPR mode.

[0140] In addition, if And m < m max Let m = m + 1, and repeat steps 2 to 7.

[0141] This invention proposes a QoE-based fair resource optimization method based on digital twins. To achieve effective resource allocation and ensure user QoE fairness, a heuristic and efficient algorithm is developed, and a fair resource allocation framework supporting digital twins (DT) for wireless VR systems is proposed. To guarantee user fairness in the evaluation system, a QoE model is constructed using a linearly weighted combination of multiple factors such as video quality, service latency, and energy efficiency. Simultaneously, the optimization variables are decoupled by jointly optimizing encoding mode selection, transmit power, edge server rendering computation time, and client-side GPU cycle frequency, achieving effective resource allocation and minimizing average latency. This algorithm exhibits good convergence and reduces computational complexity. Furthermore, the joint optimization scheme proposed in this invention demonstrates better performance in terms of average latency compared to existing technologies.

[0142] Please see Figure 3 , Figure 3 (a) in the figure is a schematic diagram showing the relationship between the total computing power F of the edge server provided in the embodiment of the present invention and the energy efficiency in the worst case. Figure 3 (b) is a schematic diagram illustrating the relationship between the total computing power F of the edge server provided in this embodiment of the invention and the QoE of the HDM client in the worst case. To further demonstrate the system efficiency of the proposed solution, three baseline solutions are constructed and compared with the proposed solution in this embodiment of the invention. These three baseline solutions are: APG, APC, and NMS. Specifically, APG: This solution fixes the computation time of the server-side rendering task, and its joint optimization mode selection... Transmit power P and HDM client GPU frequency APC: This solution jointly optimizes computation time under VPR mode. Mode Selection Transmit power P (the computing power of the HDM client is fixed); NMS: Unoptimized mode selection.

[0143] exist Figure 3 In (a), energy efficiency increases with the server's computing power F. This shows that, compared to other solutions, the present invention achieves a higher target value with lower energy efficiency, demonstrating better performance. Figure 3 (b) It can be seen that as the total computing power F of the server increases, QoE shows a trend of first rising and then falling. When the number of HDM clients is fixed, the increase of F can shorten the time required for the server to perform rendering. When F < 3.5 GHz, QoE will increase with the increase of F. The solution proposed in this embodiment of the invention has the best performance. When F exceeds 3.5 GHz, QoE decreases slightly. The solution proposed in this embodiment of the invention has the best performance.

[0144] Please see Figure 4 , Figure 4 This diagram illustrates the target QoE of the network and each client under different schemes provided in this embodiment of the invention. Specifically, purple represents the statistical data of the APC scheme, yellow represents the statistical data of the NMS scheme, red represents the statistical data of the APG scheme, and blue represents the scheme proposed in this invention. It can be seen that although the QoE of the NMS scheme is higher than that of the scheme proposed in this invention for some clients, the QoE of each client under the NMS scheme is not balanced, and fair resource allocation for clients cannot be achieved. The scheme proposed in this invention can balance the QoE of each client, and compared with the APG and APC schemes, the QoE of the scheme proposed in this invention is higher.

[0145] Please see Figure 5 , Figure 5This is a data illustration of the target QoE values ​​of the network, worst-case client, and best-case client under different schemes provided in the embodiments of the present invention. It can be seen that the QoE of the best-case and worst-case clients differs greatly in NMS, while the QoE of each client in the algorithm of the present invention achieves a better balance, and the QoE loss of the network is reduced.

[0146] Please see Figure 6 , Figure 6 This refers to the service latency and the maximum GPU cycle frequency f of the HDM client under different schemes provided in the embodiments of the present invention. max The diagram illustrates the relationship. Specifically, given the minimum GPU cycle frequency f of the HDM client... min =0.5GHz. As can be seen from the graph, with the maximum GPU cycle frequency f of the HDM client... max As the total computing resources of the client increase, the service latency gradually decreases. This is because as the total computing resources of the client increase, more computing resources are allocated to execute rendering tasks, so the allocated GPU cycle frequency is inversely proportional to the computing time. Therefore, the NMS solution has the highest service latency and the worst performance. Furthermore, the APC solution maintains a constant service latency because the client's GPU cycle frequency is not affected by f. max The impact, although the APC scheme in f max <1.9*10 9 While the service latency is lower than other solutions at GHz, the performance of the APC solution gradually falls below that of the solution proposed in this invention as the total computing resources of the client increase. The APC solution cannot flexibly adapt to changes in total computing resources. Therefore, the solution proposed in this invention offers the best performance among all comparative solutions.

[0147] Please see Figure 7 , Figure 7 This diagram illustrates the relationship between video quality and bandwidth B under different schemes provided in this embodiment of the invention. The bandwidth setting in the system affects mode selection; therefore, examining the performance of different schemes under different bandwidths is worthwhile when evaluating the system's performance across different bandwidths. It can be observed that the video quality of all schemes increases with increasing bandwidth B. This is because the TR mode offers higher video quality. When the system bandwidth increases, the system tends to select the TR mode for the HDM client. The diagram shows that the proposed algorithm performs best, followed by the APC and APG schemes, while the NMS scheme performs the worst.

[0148] Please see Figure 8 , Figure 8This is a schematic diagram showing the relationship between QoE and the number of HDM clients N under different schemes provided in the embodiments of the present invention. Due to limited resources, namely limited bandwidth and computing resources, QoE decreases as the number of HDM clients increases. When the value of N increases to 20, the rate of QoE decrease slows down in all schemes, but the algorithm proposed in this invention has the best performance among all the comparison schemes.

[0149] Please see Figure 9 , Figure 9 This is the worst-case energy efficiency and maximum transmit power P of the client provided in this embodiment of the invention. max A schematic diagram illustrating the relationship between energy efficiency and P. max As P increases, it increases. max When it increases to a certain extent, that is, P max When the value is greater than 0.8, the rate of improvement in energy efficiency slows down. This is because the communication channel bandwidth is limited, power allocation is saturated, and ultimately computational energy efficiency takes precedence. Nevertheless, for a given P... max The solution provided by this invention has the lowest energy efficiency.

[0150] The simulation results show that the method provided by this invention achieves effective resource allocation and minimizes average latency. The algorithm has good convergence, and the joint optimization scheme in the algorithm shows better performance in terms of average latency than existing schemes.

[0151] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A QoE-based fair resource optimization method based on digital twins, characterized in that, include: Step 1: Initialize the mode selection for all HDM clients. make m≤m max ,in, N represents the total number of HDM clients, and n represents the number of HDM clients out of the N HDM clients. Let a represent the set of the HDM clients n. n This indicates the mode selection for the HDM client n, when a n When in TR mode, it indicates that the HDM client n is in slice re-encoding mode, when a n In VPR mode, it indicates that the HDM client n is in window rendering encoding mode, and m represents the iteration number. max Let m = 0, representing the threshold for the number of iterations in this round. Step 2: Select based on the mode of all HDM clients. Obtain the transmission power of the HDM client in TR mode during the m-th iteration. and the transmission power of the HDM client in VPR mode in, This represents the transmission power of HDM client n in TR mode during the m-th iteration. This represents the transmission power of HDM client n in VPR mode during the m-th iteration; Step 3: Select based on the mode of all HDM clients. and the transmission power of the HDM client in the TR mode Obtain the GPU cycle frequency of the HDM client in TR mode during the m-th iteration. in, This represents the GPU cycle frequency required for the HDM client n in TR mode to render video in the m-th iteration; Step 4: Select based on the mode of all HDM clients. Obtain the computation time of the HDM client in VPR mode. in, This represents the rendering computation time of HDM client n in VPR mode during the m-th iteration; Step 5: Based on the transmission power The transmission power The GPU cycle frequency and the calculation time Get the optimization mode selection for all HDM clients Step 6: Based on the transmission power The transmission power The GPU cycle frequency The calculation time and the optimization mode selection Determine the target value for the m-th iteration. Step 7: When m < 1, let m = m + 1, and repeat steps 2 to 7; when m ≥ 1, if Then the current iteration ends, and the optimal solution is: as well as like 1) |≥ε and m<m max Let m = m + 1, and repeat steps 2 to 7, where ε represents a preset threshold. This represents the optimal solution for mode selection by all HDM clients in this iteration. This represents the optimal transmission power solution for the HDM client in TR mode during this iteration. This represents the optimal transmission power solution for the HDM client in VPR mode during this iteration. This represents the optimal GPU cycle frequency required for an HDM client to render video in TR mode. This represents the optimal solution for rendering computation time for an HDM client in VPR mode.

2. The QoE-based fair resource optimization method based on digital twins according to claim 1, characterized in that, Step 2 includes: When HDM client n is in TR mode, the selection is based on the modes of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method: The transmission power of the HDM client in TR mode is obtained in the m-th iteration. in, and The input parameters for latency and energy efficiency of the QoE client digital twin are obtained based on the nonlinear regression method. ω represents the bitstream size transmitted to the HDM client n. n, This represents the bandwidth consumption of HDM client n. P represents the channel gain of the HDM client n. max This indicates the maximum transmit power of the base station. Indicates the transmission rate required by the HDM client n; μ, ν n and γ n Let V and V represent Lagrange multipliers, respectively, and μ ≥ 0. TR Indicates the video quality in TR mode; When HDM client n is in VPR mode, the selection is based on the modes of all HDM clients. Solving optimization problems based on the Lagrange dual decomposition method: Obtain the transmission power of the HDM client in VPR mode during the m-th iteration. in, ω represents the bitstream size transmitted to the HDM client n in VPR mode. n, This represents the bandwidth consumption of HDM client n in VPR mode. V represents the channel gain of the HDM client n in VPR mode. VPR This indicates the video quality in VPR mode.

3. The QoE-based fair resource optimization method based on digital twins according to claim 1, characterized in that, Step 3 includes: Based on the mode selection of all HDM clients. The transmission power Solving optimization problems using heuristic search algorithms: and Obtain the GPU cycle frequency in TR mode in, It is an auxiliary variable. and The input parameters ω for the QoE client digital twin are obtained based on nonlinear regression, which are used to obtain the parameters for latency and energy efficiency. n, This represents the bandwidth consumption of HDM client n in TR mode, where G(*) denotes a function of *. Max represents the GPU cycle frequency required for an HDM client n in TR mode to render video during the m-th iteration. n {*} represents the largest value in set *. This indicates the bitstream size transmitted to the HDM client n in TR mode. k represents the channel gain of the HDM client n in TR mode. h Indicates the effective switched capacitor of the HDM client, c n This indicates the processing density required for the edge server to render the video from the HDM client n. and V represents the minimum and maximum GPU cycle frequencies required by the HDM client n, respectively. n This indicates the video quality obtained by the HDM client n.

4. The QoE-based fair resource optimization method based on digital twins according to claim 1, characterized in that, Step 4 includes: Based on the mode selection of all HDM clients. Solving the computation time in VPR mode Optimization issues: and By introducing an auxiliary variable ξ, the problem is transformed into: The computation time under VPR mode is obtained by solving the above optimization problem using the barrier function method. in, This indicates that the input parameters λ for the QoE client digital twin are obtained based on a nonlinear regression method. n, k represents the sensitivity of the HDM client n to energy efficiency. e Indicates the effective switched capacitor of the edge server, c n I represents the processing density required for the edge server to render the video for the HDM client n. n This indicates the size of the rendered video segment for the HDM client n. This represents the bitstream size transmitted to HDM client n in VPR mode, and F represents the total computing power of the edge servers corresponding to the N HDM clients.

5. The QoE-assured fair resource optimization method based on digital twins according to claim 1, characterized in that, Step 5 includes: Step 5.1: Initialize the set Ω = {C1, C2, ... C} N } Select the HDM client set in VPR mode Select the set of HDM clients in TR mode. The set of all HDM clients is R = {1, 2, ..., N}, where, To obtain the input parameters of QoE client digital twin regarding video quality, latency, and energy efficiency based on nonlinear regression, This indicates the transmission time of the n-bit stream from the HDM client in TR mode. This indicates the transmission time of the HDM client's n-stream in VPR mode. η represents the computation time for client n to render the video. n TT η represents the transmission energy efficiency of HDM client n under TR. n TV This indicates the transmission energy efficiency of HDM client n in VPR mode. This represents the computational energy efficiency of HDM client n. This represents the computational energy efficiency of the edge server, where T represents the number of video segments. This represents the viewport width of the HDM client n. Indicates the height of the HDM client n; Step 5.2: Initialize n = 1; Step 5.3, based on the above The The and stated C was calculated n When C n <0, then a n =0、 When n = n + 1, repeat step 5.3 until n = N; Step 5.4, when get It contains the maximum value C n If n, then a n =1、 Repeat step 5.4 when Perform step 5.5, where ω n, ω represents the bandwidth consumption of HDM client n in TR mode. n, B represents the bandwidth consumption of the HDM client n in VPR mode, and B represents the system bandwidth. Step 5.5, when At that time, a n =0, in Repeat step 5.5 when When the execution ends; Step 5.6: Obtain the optimization mode selection.

6. The QoE-assured fair resource optimization method based on digital twins according to claim 1, characterized in that, Step 6 includes: According to the transmission power The transmission power The GPU cycle frequency The calculation time and the optimization mode selection Determine the target value for the m-th iteration. HDM client n's QoE model Q n Represented as: Q n =λ n, V n -λ n, D n -λ n, η n By introducing a digital twin layer, determine in, Let λ represent the QoE model of the HDM client n in the m-th iteration of the digital twin layer. n, , λ n, , λ n, These represent the sensitivity of the HDM client n to video quality, service latency, and energy efficiency, respectively. V is the QoE model evaluation parameter of the client-side digital twin, calculated using a nonlinear regression method. n D represents the video quality obtained by the HDM client n. n η represents the service latency of HDM client n. n This indicates the energy efficiency of the HDM client n.

7. The QoE-based fair resource optimization method based on digital twins according to claim 1, characterized in that, When the HDM client n is in TR mode, the video clips transmitted to the HDM client n are rendered on the HDM client n. When the HDM client n is in VPR mode, the video clips transmitted to the HDM client n are rendered on the edge server corresponding to the N HDM clients.

8. The QoE-assured fair resource optimization method based on digital twins according to any one of claims 1-7, characterized in that, when a n When a = 0, it indicates that the HDM client n is in VPR mode; when a n When = 1, it indicates that the HDM client n is in TR mode.

Images