360-degree virtual reality video multi-user transmission system and resource allocation method

Through transcoding and multicast technology, combining video encoding and active caching, predicting user vision, optimizing video code rate and resource allocation, the problem of cable network quality fluctuations in 360-degree virtual reality video transmission in multi-user scenarios is solved, achieving high Quality and low latency video transmission.

CN119011798APending Publication Date: 2024-11-22ZHEJIANG UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411020509.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art has failed to effectively consider the impact of the multi-hop wired network transmission quality fluctuations from cloud VR servers to MEC servers on the service quality of 360-degree mobile virtual reality video users in multi-user scenarios, resulting in unreliable transmission quality.

Method used

By adopting transcoding and multicast technology, through the collaborative work of cloud servers, MEC servers and users, video encoding, active caching and multicast technology, predict user vision, optimize video code rate, and perform group transmission and resource allocation of video blocks. Ensure quality of service for each user and allocate resources within the end-to-end delay threshold.

Benefits of technology

Multi-user transmission of 360-degree virtual reality videos under low-end to end delay threshold is realized, making use of wireless resources to the maximum extent, ensuring the service quality of each user, and improving the reliability and quality of video transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011798A_ABST
    Figure CN119011798A_ABST
Patent Text Reader

Abstract

The invention discloses a 360-degree virtual reality video multi-user transmission system using transcoding and multicast and a resource allocation method, which utilize the communication, calculation and storage functions of a cloud server, an MEC server and a user side in the system, and jointly consider the influence of video coding, active caching, transcoding and multicast on 360-degree virtual reality video users. Wireless resources of the system can be fully utilized on the premise that the time delay is lower than an end-to-end time delay threshold value, and the service quality of each 360-degree virtual reality video user is guaranteed. Besides, according to the resource allocation method based on multicast packet transmission, under the constraint of 360-degree virtual reality video end-to-end time delay, by taking maximization of the sum of video code rates of all users as a target, multicast user grouping, intra-group video block code rate selection and bandwidth and power allocation of each user group are jointly optimized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications, and in particular relates to a 360-degree virtual reality video multi-user transmission system and a resource allocation method using transcoding and multicasting. Background Art

[0002] Mobile virtual reality (VR) is expected to become one of the killer applications of future 5G and B5G networks. It refers to the virtual reality service of storing and rendering mobile virtual reality video and sound files from cloud servers to user terminal devices through multi-hop networks. With the help of cloud computing technology and stable gigabit fiber networks, this application has been proven to be feasible; 360-degree video, also known as three-degree-of-freedom spherical video, can provide users with an immersive experience. Since 360-degree mobile virtual reality video combines the high capacity of enhanced mobile broadband services in 5G and the strict latency and reliability requirements of ultra-reliable low-latency communication services, there are still many technical difficulties to support this application so far.

[0003] The Chinese patent application with publication number CN116389757A proposes a wireless VR video transmission method based on MEC assistance, which determines the computing resource allocation of the VR user end and the MEC server end by optimizing the VR video encoding parameters; the Chinese patent application with publication number CN112995636A proposes a 360-degree VR video transmission system based on edge computing and active caching, which jointly considers video encoding, active caching, computing offloading and data transmission. However, the above two patent technologies only design the transmission system for a single VR or 360-degree VR user.

[0004] For multi-user scenarios, the Chinese patent application with publication number CN114640870A combines video transcoding to propose an adaptive transmission optimization method with the goal of maximizing the sum of the visual experience of all users; the paper "Collaborative edge caching and transcoding for 360° video streaming based on deep reinforcement learning (author T. Yang et al., published in IEEE Internet of Things Journal, vol. 9, no. 24, pp. 25551-25564, 2022)" proposes using multiple MEC servers for collaborative edge caching and transcoding for 360-degree VR videos. In order to save communication resources and meet users' video quality requirements, multicast technology can also be combined with caching, transcoding and other technologies to be applied to VR video multi-user transmission systems. For example, the paper "CMU-VP: Cooperative multicast and unicast with viewport prediction for VR video streaming in 5G H-CRAN (author J. Yang et al., published in IEEE Access, vol. 7, pp. 134187-134197, 2019)" proposed a transmission scheme of multicasting base layer video to all users and unicasting enhancement layer video to each user; the paper "Optimal wireless streaming of multi-quality 360VR video by exploiting natural, relative smoothness-enabled, and transcoding-enabled multicast opportunities (author K. Long et al., published in IEEE Trans.Multimedia, vol.23, pp.3670-3683, 2021)" proposes a joint optimization transmission method for multi-360-degree VR user multicast grouping, bit rate selection, group transmission time and transmission power allocation with the goal of minimizing transmission energy or minimizing the weighted sum of transmission energy and transcoding energy. However, none of the above patents or papers consider the impact of fluctuations in wired network transmission quality from cloud VR servers to MEC servers on the quality assurance of 360-degree mobile virtual reality video user services in the transmission scheme.

[0005] In summary, although some valuable designs and optimization solutions for 360-degree mobile virtual reality video transmission have been proposed from the perspectives of edge caching, computing offloading, and communication integration, the multi-hop wired part from the cloud VR server to the MEC server is still a neglected link in the existing solutions. Considering the stringent transmission rate and latency requirements of 360-degree VR itself, as well as the inevitable fluctuations in transmission quality due to congestion, packet loss and other factors in the wired network, there is an urgent need for a comprehensive solution that considers the complete content provider cloud VR server to the operator MEC server to the user, including multi-hop wired and single-hop wireless networks, and is designed for caching, computing, and communication integration. Summary of the invention

[0006] In view of the above, the present invention provides a 360-degree virtual reality video multi-user transmission system and resource allocation method using transcoding and multicast, which utilizes the communication, computing, and storage functions of the cloud server, MEC server, and user terminal in the system, and jointly considers the impact of video encoding, active caching, transcoding, and multicast on 360-degree virtual reality video users. The wireless resources of the system can be fully utilized under the premise of being lower than the end-to-end delay threshold, thereby ensuring the service quality of each 360-degree virtual reality video user.

[0007] A 360-degree virtual reality video multi-user transmission system using transcoding and multicasting includes a cloud server, an MEC server, and multiple user terminals. If there is a user terminal that needs a 360-degree virtual reality video service in the system, the user terminal index is represented as in:

[0008] The cloud server stores 360-degree virtual reality videos and generates corresponding saliency maps through neural networks. When encoding the video, it is divided into N×M tiles by equidistant cylindrical projection, and the data of multiple video frames of each tile are compressed into a set of picture sequences, where N and M are both custom natural numbers. At the same time, the cloud server predicts FoV (Field of View) based on the saliency map and transmits the corresponding video data packet to the MEC server. In order to meet the video bit rate requirements of different user terminals, the cloud server uses advanced coding technology (such as HEVC or H.264, etc.) to pre-encode the video blocks into bit rate versions with different quality levels. Indicates the optional bit rate, where r1<r2<...<r max ; Assuming that tiles of the same quality level have the same bitrate, let represents the minimum video bit rate requirement of user terminal i, σ is the tolerance threshold of the user terminal to the bit rate change within the field of view, and the acceptable bit rate range of user terminal i is

[0009] The MEC server will make grouping decisions for the user end for multicast based on the channel conditions of the user end, the requested video content, and the cache hit results of the MEC server, and will cache and transcode the video data packets sent by the cloud server and transmit the video data packets to the corresponding user end;

[0010] In addition to receiving data sent by the MEC server, the user terminal also needs the cloud server to provide it with data packets that the MEC server missed. After decoding and rendering, the data obtained from the MEC server and the cloud server are combined into a displayable image to complete the conversion of the 2D video to a 360-degree spherical video and play it.

[0011] The system adopts a periodic working mode. Each working cycle consists of a caching stage and a calculation and transmission stage. The video data is divided in the time domain into multiple video blocks. The duration of each video block is D1, and it is stipulated that the index of the video block watched by each user terminal at the same time in the lth working cycle is l-1, where l is a natural number greater than 1; in the caching stage, the cloud server transmits the corresponding video data packet to the MEC server according to the field of view prediction result. The duration of the caching stage is the duration of a video block, i.e. D1; at the beginning of the calculation and transmission stage, according to the FoV of each user terminal and the cache of the MEC server As a result, the MEC server makes multicast grouping decisions for all user terminals and transcodes the data packets for unicast or multicast data transmission to each user group; at the same time, the MEC server immediately starts to request the cloud server to retransmit the data required by the user terminal but not hit in the MEC server cache. This part of the data is transmitted sequentially from the cloud server to the MEC server; after receiving the data retransmitted by the cloud server, the MEC server transmits this data to different user terminals in packets. After receiving the data from the MEC server and the cloud server, the user terminal decodes, renders and plays the video data packets.

[0012] Furthermore, for the cache phase, at the beginning of the lth working cycle, the cloud server predicts the user's field of view probability distribution P at the beginning of the l+1th working cycle based on the saliency map of the lth video block: F =[P F (n (l) ,m (l) )] N×M , where P F (n (l) ,m (l) ) indicates that at the beginning of the l+1th working cycle, the user's viewpoint falls into the coordinates (n (l) ,m (l) ) in the tile, and are the sets of horizontal and vertical coordinates of the tile respectively; define P R=[P R (n (l) ,m (l) )] N×M is the probability matrix of the user requesting a tile, where Indicates that at the beginning of the next cycle, the coordinates are (n (l) ,m (l) ) falls within the user's FoV area, indicating the function The coordinates are (n (l) ,m (l) ) falls into the tile with coordinates (n c ,m c ) is the FoV area centered on the tile, otherwise the indicator function is equal to 0. and They represent the set of horizontal and vertical coordinates of the tiles within the user's field of view, respectively. a1 is the number of tiles contained in the user's field of view in the horizontal and vertical directions.

[0013] Furthermore, after the cache phase, the calculation and transmission phase begins. At this time, the real FoV information of any user terminal i It has been determined that Indicates that user terminal i starts to determine the tile with coordinates (n, m) in the l+1 working cycle according to FoV calculation sub-phase, and vice versa Use collection It represents the set of all tile coordinates required for viewing by client i in the l+1th working cycle, and is expressed as Indicates that all users correspond to Composed of a collection, using a collection Records whether the MEC server caches the complete data of the tile in the lth cycle, where Indicates that the MEC server completely caches the tile with coordinates (n,m), otherwise make It represents the set of coordinates of tiles cached in the MEC server and requested by the user terminal i in the lth cycle; the calculation and transmission stage includes the MEC server transcoding sub-stage, the transmission sub-stage from the MEC server to the grouped user, the supplementary transmission sub-stage from the cloud server to the MEC server, the supplementary transmission sub-stage from the MEC server to the grouped user, and the calculation sub-stage of the user terminal.

[0014] Furthermore, in the MEC server transcoding sub-stage, when receiving new field of view requests from different users, the MEC server makes grouping decisions for user multicast based on the field of view requests of the users and the user channel conditions, and checks whether there are video data packets required by each user in the cache, and transcodes according to the video quality requirements of the users in the group; represents the bitrate set of the tile with coordinates (n, m) in the MEC server that meets the viewing needs of user terminal i in the l+1th working cycle, where Represents the set of bitrates of tiles with coordinates (n,m) in the MEC server;

[0015] In the transmission sub-phase from the MEC server to the grouped users, for different multicast or unicast groups, the MEC server will make decisions on the channel bandwidth, transmission power, and video bitrate for each user group based on the channel quality of the user group, the bandwidth and power of the system, and the video quality requirements of the user group;

[0016] In the supplementary transmission sub-phase from the cloud server to the MEC server, the cloud server sequentially supplements the video data packets that the MEC server does not have cache and that the user needs. represents the bitrate set of tiles with coordinates (n, m) in the cloud servers not cached by the MEC server that meet the viewing needs of user terminal i in the l+1th cycle, where represents the set of coordinates of tiles that are needed by client i to watch in the l+1th period but are not cached in the MEC server;

[0017] In the supplementary transmission sub-stage from the MEC server to the grouped user, the MEC server performs supplementary transmission of the data packet according to the grouping and resource decision, and in order to improve the resource utilization of the system, the supplementary transmission from the cloud server to the MEC server and the transcoding of the MEC server are carried out simultaneously; in addition, for the data hit by the MEC server transmission and the data supplemented by the cloud server, the wireless transmission of these two parts of data is realized in a serial manner;

[0018] In the user-side calculation sub-phase, the user side decodes and renders the received video data packets. When all user sides complete the calculation and decoding, the communication and decoding tasks of the lth working cycle are completed, and the user side plays the video frame of the lth video block at the beginning of the l+1th working cycle.

[0019] In order to maximize the use of bandwidth resources and ensure the service quality of users, the present invention also provides a resource allocation method for multicast packet transmission of the above system. When there are I user terminals, the system has a total of G=2 I -1 The number of multicast or unicast packets, the index of the user group is use Represents the set of users in user group g, using and They represent the bitrate set of the tile with coordinates (n, m) in the MEC server and the cloud server that meets the viewing needs of all users in the user group g in the l+1th cycle; the sufficient conditions for the activation of the user group g include:

[0020] ① At least one tile is required by all clients in the group.

[0021] ② For tiles that meet ①, their bitrate can meet the requirements of all users in the group: If the tile with coordinates (n, m) has been cached by the MEC server and is needed by all users in user group g, but If the tile with coordinates (n, m) is not cached by the MEC server but is needed by all users in user group g, but

[0022] Let G′ be the number of user groups remaining after removing the user groups that do not meet the above conditions, represents the set of user groups remaining after removing the user groups that do not meet the above conditions. Let the indicator variable Indicates whether user group g is activated, using B g and P g Denote the bandwidth and transmission power allocated to user group g respectively, and y = [y g ] 1×G′ Indicates the indication policy of the user group, using represents the bitrate strategy for transmitting tiles within each user group, where the indicator variable Indicates whether the tile with coordinates (n,m) is at rate r κ Transmit to all users of user group g, using B = [B g ] 1×G′ It represents the bandwidth allocation strategy of the base station connected to the MEC server to each unicast or multicast group, with P = [P g ] 1×G′ It represents the allocation strategy of the transmission power to each user group; then the user grouping, video tile bit rate selection and wireless resource allocation strategy are determined by the optimization problem OP1 of maximizing the sum of the received bit rates of all user terminals. The expression of the optimization problem OP1 is as follows:

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] Where: Formula (2) represents the delay violation probability that the end-to-end delay of any user terminal i in any user group g exceeds the delay threshold Both need to be less than the delay violation probability threshold P out,th , the probability is a function of the user grouping strategy y, the video tile bitrate selection strategy x, the bandwidth allocation strategy B and the power allocation strategy P; Formula (3) indicates that if the tile with coordinates (n, m) is required by user terminal i to watch in the l+1th cycle, the tile must be transmitted to user terminal i, and user terminal i only needs to receive the tile once; Formulas (4) and (5) are indicator variables and g The value range of ; Formulas (6) and (7) respectively indicate that the bandwidth allocated to each user group and the sum of the bandwidths should be greater than or equal to 0 and not exceed the total channel bandwidth B; Formulas (8) and (9) indicate that the transmit power allocated to each user group and the sum of the powers should be greater than or equal to 0 and not exceed the total transmit power P of the base stations connected to the MEC server; θ in Formulas (10) and (11) is a sufficiently large positive number, indicating that if user group g is not activated, the bandwidth and transmit power allocated to user group g are 0; Formula (12) indicates that if user group g is not activated, no tile and code rate will be allocated to user group g.

[0036] Furthermore, given the known allocation strategy π = (y, x, B, P), according to the known cache set in the lth working cycle Request Collection And the end-to-end delay under transmission strategy π is greater than D th The conditional probability density function of the delay violation probability in formula (2) is It is expressed as:

[0037]

[0038]

[0039] in: The random variable D represents the end-to-end delay from the time when user i in user group g sends a view request to the time when it can play the lth video block in the system in the lth working cycle. th is a given delay threshold, is the random variable of user i in user group g The conditional probability density function, t (l) is a random variable The corresponding value, represents the random variable of the MEC server transcoding phase latency in the lth working cycle, represents the random variable of the transmission sub-phase delay from the MEC server to the user end in user group g in the lth working cycle, represents the random variable of the delay of the supplementary transmission sub-phase from the cloud server to the MEC server in the lth working cycle, represents the random variable of the delay of the supplementary transmission sub-phase from the MEC server to the user end in user group g in the lth working cycle, A random variable representing the delay of the computation sub-phase of client i in the lth working cycle.

[0040] Furthermore, the conditional probability density function The calculation expression is as follows:

[0041]

[0042] in: Represents a random variable The conditional probability density function, u is the value corresponding to the random variable U, express The conditional probability density function of The value of .

[0043] Furthermore, the conditional probability density function as well as The expression is as follows:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] in: is the conditional cumulative distribution function of U; let t2 be a random variable The corresponding value, t3 is a random variable The corresponding value, t4 is a random variable The corresponding value is and Respectively The conditional cumulative distribution function and conditional probability density function of and Respectively The conditional cumulative distribution function and conditional probability density function of express The conditional probability density function of and They are The conditional cumulative distribution function and conditional probability density function of The value of and They are The conditional cumulative distribution function and conditional probability density function of The value of ; is the absorption boundary of the transmission sub-phase from the MEC server to the user end in user group g, is the absorption boundary of the supplementary transmission sub-stage from the cloud server to the MEC server, is the absorption boundary of the sub-stage of MEC server supplementing the transmission to the user end in user group g, α 2,g and β 2,g They represent the drift coefficient and diffusion coefficient of the transmission process from the MEC server to the user end in user group g, α1 and β1 represent the drift coefficient and diffusion coefficient of the transmission process from the cloud server to the MEC server, 1 / λ 2,g and V 2,gare the mean and variance of the interval variable of the arrival time of the data packet received by user group g from the MEC server, 1 / λ1 and V1 are the mean and variance of the interval variable of the arrival time of the data packet received by the MEC server from the cloud server, and the expression of the function Φ is represents the total amount of cache hit data that the MEC server needs to transmit to the users in user group g during the lth working cycle, It represents the total amount of data transmitted from the cloud server to the MEC server in the lth working cycle. represents the total amount of data transmitted by the MEC server to the user end in user group g during the lth working cycle, k is the size of each video data packet, sgn() is the sign function, R1(t) and R 2,g (t) represents the wired network transmission rate from the cloud server to the MEC server and the wireless network transmission rate from the MEC server to the user end in user group g at time t, respectively; E() represents the mean, and Var() represents the variance.

[0057] Furthermore, the optimization problem OP1 is analyzed and converted into a two-layer optimization problem for solution. The upper layer optimization problem is the user grouping problem, and the lower layer optimization problem is the code rate selection and communication resource allocation problem.

[0058] Furthermore, the upper layer optimization problem is the group decision problem OP2, whose objective function is to maximize the sum of the received bit rates of all users. The expression of the group decision problem OP2 is as follows:

[0059]

[0060]

[0061]

[0062] In order to maintain the flexibility of the grouping strategy and expand the range of feasible solutions, constraint (13) is introduced to ensure that all tiles required by users under the current user grouping strategy y are transmitted to the user end.

[0063] The upper optimization problem can be solved by using the ant colony algorithm. The grouping mode selection strategy in the ant colony algorithm can be expressed as:

[0064]

[0065]

[0066]

[0067]

[0068]

[0069] Where: g * is the index of the user group selected for each grouping decision, q is a random number uniformly distributed between [0,1], q0 is a parameter between [0,1], and τ g is the pheromone of user group g, η g is the heuristic information of user group g, β∈[0,1] is a parameter that determines the relative importance between pheromone and heuristic information, Y is an indicator variable that selects whether the user group is activated according to the activation probability through the roulette rule, and the activation probability p g Expressed as C delay,g With C rate,g They represent the delay weight and bit rate weight of user group g respectively, and They represent the delay and bit rate of user group g in the lth working cycle after normalization and mapping to the range of [0,1], It represents the delay mean estimate of user terminal i in user group g after multiple samplings under different channel conditions in the lth working cycle. The weighting parameter ω∈[0,1] in the heuristic information is the balance coefficient reflecting the end-to-end delay and bit rate of the user terminals in user group g.

[0070] Select a feasible user group g * When , the ant colony algorithm needs to perform local pheromone update, and the update formula is:

[0071] τ g* =(1-χ)τ g* +χτ0

[0072] Where: χ∈[0,1] is the attenuation coefficient of the pheromone, and τ0 is the initial value of the pheromone.

[0073] When a set of feasible strategies (y * ,x * ,B * ,P * ), the ant colony algorithm needs to perform global pheromone update, and the update formula is:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] Where: Δτ sum Represents strategy (y * ,x * ,B * ,P * ) under the total pheromone value added, and They represent strategies (y * ,x * ,B * ,P * ) under the delay weight and bit rate weight, and They represent strategies (y * ,x * ,B * ,P * ) In the lth working cycle, the user group g is mapped to the delay and bit rate in the range of [0,1] through the normalization method. Represents strategy (y * ,x * ,B * ,P * ) is the average delay estimation obtained by multiple sampling of user terminal i in user group g under different channel conditions in the lth working cycle, and They represent strategies (y * ,x * ,B * ,P * ) and the end-to-end delay of user terminal i in user group g.

[0080] The process of grouping decision-making using the above ant colony algorithm is as follows:

[0081] Step 1: Initialize the set of groups to be activated Initialize all user group indicator variables y g = 0; according to the bit rate set in the MEC server and the cloud server that meets the block requirements of group g users and Initialize the rate selection variable x0 with the minimum bit rate of the tile with coordinates (n, m) in the group; initialize the bandwidth allocation variable B0 and the power allocation variable P0 by evenly allocating the bandwidth and power of each group;

[0082] Step 2: Select a strategy based on the grouping pattern of the ant colony algorithm from Select the user group to be activated in * , and let y g* =1;

[0083] Step 3: Add the user group* From the set of groups to be activated Removed;

[0084] Step 4: According to τ g* =(1-χ)τ g* +χτ0 updates the local pheromone in the ant colony algorithm;

[0085] Step 5: Check whether the user grouping strategy y satisfies formula (13);

[0086] Step 6: Loop steps 2 to 5 until equation (13) is satisfied;

[0087] Step 7: Output user grouping strategy y.

[0088] Furthermore, after obtaining the upper-layer optimized user grouping strategy y, the lower-layer optimization problem needs to solve the code rate selection and inter-group wireless resource allocation. The expression of the optimization problem OP3 is as follows:

[0089]

[0090] st formula (3), formula (4), formula (6) to (12)

[0091] 0≤ξ≤1

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] Where: h is the identifier of the number of iterations in solving the underlying optimization problem, ξ is the penalty term of the objective function, and is the slack variable associated with user group g, is the code rate indicator variable at the hth iteration, x h represents the value of the code rate selection variable at the hth iteration, Indicates the bit rate selection x that the MEC server needs to transmit to the user end in user group g during the lth working cycle at the hth iteration h The corresponding total data volume, Indicates the bit rate selection x given by the MEC server to the user terminal in user group g in the lth working cycle at the hth iteration hThe corresponding amount of supplementary data, and They represent the values ​​of the slack variables related to user group g at the hth iteration, N0 is the noise power spectrum density, d i is the distance between user terminal i in user group g and the base station connected to MEC, β p is the path loss exponent.

[0099] The specific process of code rate selection and wireless resource allocation algorithm is as follows:

[0100] Step 1: Based on the bit rate set in the MEC server and cloud server that meets the block requirements of each group of g users and Initialize the rate selection variable x0 with the minimum rate of the tile with coordinates (n, m) in each group, and initialize the iteration count identifier h = 1;

[0101] Step 2: Calculate the hth iteration and The value of

[0102] Step 3: Solve the optimization problem OP3 using the branch and bound method;

[0103] Step 4: Iterate step 2 to step 3 until the objective function of the optimization problem OP3 converges;

[0104] Step 5: Calculate the total number of user i in all user groups g.

[0105] Step 6: If all users of all user groups If both satisfy the delay requirement in formula (2), the algorithm succeeds; otherwise, the algorithm fails.

[0106] Step 7: Output the result (x, B, P).

[0107] Furthermore, the upper and lower double-layer optimization problems are jointly solved. The specific process is as follows:

[0108] Step 1: Initialize the iteration indicator variable iter = 1; according to the bit rate set in the MEC server and cloud server that meets the required block of each group g users and Initialize the bitrate selection variable x0 with the minimum bitrate of the tile with coordinates (n, m) in the group; initialize the bandwidth allocation variable B0 and the power allocation variable P0 by evenly allocating bandwidth and power to each user group; initialize (y * ,x * ,B * ,P * ) is empty;

[0109] Step 2: When the number of iterations iter is less than the maximum number of iterations iter max When the ant colony algorithm is used to generate pop user grouping strategies {y1,y2,...,y pop};

[0110] Step 3: For each user grouping strategy, the code rate selection and wireless resource allocation algorithm is used to generate the code rate selection, bandwidth and transmit power allocation results (x1, B1, P1), (x2, B2, P2), …, (x pop ,B pop ,P pop );

[0111] Step 4: Select the strategy that maximizes the user end receiving bit rate from the pop user grouping strategies, that is,

[0112] Step 5: Update the policy by The global pheromone:

[0113] Step 6: If Update

[0114] Step 7: Repeat steps 2 to 6 until the maximum number of iterations iter is reached. max ;

[0115] Step 8: Output allocation strategy (y * ,x * ,B * ,P * ).

[0116] Based on the above technical solution, the present invention has the following beneficial technical effects:

[0117] 1. The present invention designs a 360-degree virtual reality video multi-user transmission system and resource allocation method based on transcoding and multicast from the perspective of comprehensive utilization of the three resources of transmission, storage and computing of cloud servers, edge computing servers and user terminals. In order to alleviate the bandwidth requirements of 360-degree virtual reality video transmission in multi-user scenarios and ensure the service quality of users, the present invention proposes to cache 360-degree virtual reality videos from cloud servers on edge computing servers according to the user's field of view prediction results, transcode the cached video blocks on edge computing servers according to the user's bit rate requirements, and use multicast at the base station to send the data required by multiple users of 360-degree virtual reality videos.

[0118] 2. The resource allocation method based on multicast packet transmission of the present invention jointly optimizes the multicast user grouping, the intra-group video block bit rate selection, and the bandwidth and power allocation of each user group under the constraint of the end-to-end delay of the 360-degree virtual reality video, with the goal of maximizing the sum of the video bit rates of all users. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 It is a structural schematic diagram of a 360-degree virtual reality video multi-user transmission system of the present invention.

[0120] Figure 2 The diagram is a schematic diagram of field of view prediction and video encoding of a 360-degree virtual reality video multi-user transmission system of the present invention.

[0121] Figure 3 The figure is a schematic diagram of the working process of the 360-degree virtual reality video multi-user transmission system of the present invention.

[0122] Figure 4 The figure is a flow chart of the algorithm for upper and lower double-layer joint optimization of grouping decision and resource allocation of the present invention.

[0123] Figure 5 It is a simulation diagram of the relationship between the number of iterations of the upper and lower double-layer joint optimization algorithm and the average user video total bit rate under different bandwidths and numbers of service users of the present invention.

[0124] Figure 6 It is a simulation diagram for comparing the success rates of the present invention and the existing algorithms.

[0125] Figure 7 The figure is a schematic diagram of simulation of the relationship between the number of users and the average total bit rate of user videos according to the present invention and the existing algorithm.

[0126] Figure 8 It is a schematic diagram of simulation of the end-to-end delay cumulative distribution function of the present invention and the existing algorithm when serving different numbers of users. DETAILED DESCRIPTION

[0127] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0128] Embodiment 1

[0129] like Figure 1 As shown, the 360-degree virtual reality video multi-user transmission system using transcoding and multicasting of the present invention includes a cloud server, an MEC server, a head-mounted display of one user, a multi-hop wired network from the cloud server to the MEC server, and a single-hop wireless network from the MEC server to each user. The user index is represented as

[0130] The cloud server includes:

[0131] Video storage module, used for storing 360-degree virtual reality videos;

[0132] Tile encoding and compression module, which is used for equirectangular projection of 360-degree VR video, divides it into N×M tiles, and compresses the data of multiple video frames of each tile into a set of pictures;

[0133] Tile selection module, used to transmit data packets to the MEC server for active caching based on the field of view prediction results obtained from the saliency map;

[0134] Saliency map generation module: used to generate a saliency map through a neural network based on pre-stored video data;

[0135] Field of view prediction module: used to predict the probability distribution of the user's field of view based on the saliency map.

[0136] The MEC server includes:

[0137] Data packet cache, used to cache data packets from cloud servers;

[0138] Decoder, used for decoding data packets;

[0139] Transcoder, used for transcoding data packets;

[0140] The encoder is used to encode and compress the data packets cached or transcoded in the MEC server and send them to the VR users;

[0141] The hit data analysis module is used to determine the data hit by the MEC server for each user based on the sensor data sent by each user, and obtain the index of the tile that the cloud server needs to retransmit to each user due to the MEC server's miss.

[0142] The user head mounted display includes:

[0143] Data packet cache, used to cache data packets from MEC servers and cloud servers;

[0144] Decoder, used for decoding data packets;

[0145] A graphics processing unit is used to render images, that is, to complete the conversion of 2D video to 360-degree spherical video and combine data packets into video frames;

[0146] Display screen for playing 360-degree virtual reality videos;

[0147] Motion sensors provide the MEC server with information about the user's real field of view.

[0148] The multi-hop wired network from the cloud server to the MEC server is used to transmit data from the cloud server to the MEC server, and its instantaneous transmission rate at time t is R1(t).

[0149] The single-hop wireless network from the MEC server to each user is used for data transmission from the MEC server to the user. The index of the user group is use represents the set of users in the gth user group; let the instantaneous transmission rate from the base station connected to the MEC server to the users in the user group g at time t be recorded as R 2,g (t), which can be modeled by Shannon's formula as Where N0 is the noise power spectral density, B g and P g are the bandwidth and transmission power allocated to group g users respectively. In order to ensure the fairness of transmission, The channel gain of the user with the worst channel condition is defined as the channel gain of group g Here represents the channel gain of user i at time t, It follows an exponential distribution with a mean of 1, which is used to characterize the impact of small-scale channel fading. i is the distance from the base station connected to the MEC server to user i, β p is the path loss exponent.

[0150] As the network changes dynamically, the data packets have different delays from the cloud server to the MEC server or from the MEC server to the user end. Without loss of generality, let the interval variables of the video data packet arrival time in the above two network conditions follow a given but arbitrary distribution, and their means are recorded as 1 / λ1 and 1 / λ respectively. 2,g , the variances are recorded as V1 and V 2,g , these means and variances are given by:

[0151]

[0152]

[0153] Where: k represents the size of each video data packet in bits, E(·) represents the mean, and Var(·) represents the variance.

[0154] The tile-based 360-degree VR video encoding and FoV prediction model on the cloud server side of the present invention is as follows: Figure 2As shown, the initial 360° video is projected onto a two-dimensional video plane with an angle range of 180°×360° through an equirectangular projection and encoded into N×M tiles of different bitrate versions. Since the encoding and compression in the cloud server utilize the dependencies of time and space, the user's head-mounted display or MEC server requires data packets of all video frames of a tile for correct decoding; different tiles are encoded and compressed independently, and even if the field of view only covers part of the tile, the complete tile needs to be transmitted.

[0155] At the beginning of the cache phase of working cycle l, the cloud server uses the saliency map to obtain the visual field prediction result of the user in the l+1th working cycle. For the saliency map, the cloud server can generate it by using the Convolution neural network (CNN) + Long short-term memory (LSTM) + Gaussian mixture model (GMM) saliency map network proposed in the paper "Recurrent mixture density network for spatiotemporal visual attention (author L. Bazzani, published in ICLR, 2017, pp. 1-17)". Its specific neural network structure can be expressed as: C64-P-C128-P-C256-C256-P-C512-C512-P-C512-P-FC4096-FC4096-softmax, where C, P, FC and softmax represent the three-dimensional convolution layer, pooling layer, fully connected layer and softmax function network layer, respectively. The numbers after the above letters represent the number of cores of the relevant layer (such as C64 means 64 cores), the video frame sliding window of the LSTM network is 16 video frames, and the number of Gaussian distributions is set to 20. Then, the cloud server can use the LSTM viewpoint prediction network proposed in the paper "Predicting head movement in panoramic video: A deep reinforcement learning approach (author M. Xu, published in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 11, pp. 2693-2708, Nov. 2019)" to generate the viewing probability matrix P of each tile of the future video frame using the saliency map of the past 16 video frames in the sliding window. F =[P F (n (l) ,m (l) )] N×M, where the element P F (n (l) ,m (l) ) indicates that the coordinates are (n (l) ,m (l) ) is viewed; because P F (n (l) ,m (l) ) is obtained only from the saliency map, so it is equal for each user and can be pre-computed offline.

[0156] Since the predicted field of view may intersect with the boundary of the video plane, a1 is used to represent the number of tiles in the horizontal or vertical direction of the FoV area; when the center of the field of view falls on the coordinates (n c ,m c ), the set of horizontal and vertical coordinates of the tile falling in the FoV area can be expressed as:

[0157]

[0158]

[0159] in: and Represents the set of all tile coordinates in the horizontal and vertical directions respectively.

[0160] To meet the video bit rate requirements of different users, the cloud server uses advanced encoding methods such as HEVC or H.264 to pre-encode video blocks into bit rate versions with different quality levels. Indicates the optional bit rate, where r1<r2<...<r max Assuming that tiles of the same quality level have the same bitrate, let represents the minimum video bit rate requirement of user i, σ is the user’s tolerance threshold for bit rate changes within the field of view, that is, the acceptable bit rate range for user i is

[0161] The user's request probability matrix is ​​defined as P R =[P R (n (l) ,m (l) )] N×M ,in Indicates that at the beginning of the next cycle, the coordinates are (n (l) ,m (l) ) falls within the user's FoV area, indicating the function The coordinates are (n (l) ,m (l) ) falls into the tile with coordinates (n c ,mc ) is the FoV area centered on the tile, otherwise the indicator function is equal to 0.

[0162] Divide N×M tiles into three parts: tiles in the predicted FoV, tiles in the predicted FoV extension window, and other tiles, and cache these three parts of tiles in sequence; when caching each part of tiles, each tile is cached according to P R (n (l) ,m (l) ) are cached from high to low. In order to improve the viewing quality of users, the MEC server selects an appropriate bitrate for each cached tile based on its relationship with the predicted FoV. For the tile with coordinates (n, m), the specific bitrate selection rule is designed as follows:

[0163] Predicted tile bitrate within FoV: Set a fixed maximum bitrate Select the number of tiles to cache for this bitrate.

[0164] Predicted tile bitrate in FoV extension area: The bitrate in the predicted FoV extension area can be lower than the bitrate in the predicted FoV area, set to Where Δ(≤σ) is an adjustable bit rate change value; assuming that the coefficient of the predicted FoV extension area is a2(≥a1), the set of horizontal and vertical coordinates of all tiles in the predicted FoV extension area can be expressed as:

[0165]

[0166]

[0167] Among them: The number of tiles that fall only within the predicted FoV expansion area is

[0168] Tile bitrate in non-predicted FoV and predicted FoV extended area: To save bandwidth, set the tile bitrate in non-predicted FoV and predicted FoV extended area to The total number of such tiles is N3 = MN-N1-N2.

[0169] The working process of the multi-user transmission system of the present invention is as follows: Figure 3As shown in the figure, the system adopts a periodic working mode, and each working cycle consists of a caching phase and a calculation and transmission phase. The video is divided into video blocks in the time domain, and the duration of each video block is D1. It is stipulated that in the lth working cycle, the index of the video watched by the user is l-1. In the caching phase, the cloud server transmits data packets to the MEC server according to the prediction results provided by the saliency map for active caching. The duration of the caching phase is the duration of each video block D1. At the beginning of the calculation and transmission phase, the user sends a field of view request to the MEC server. The MEC server will group the users according to factors such as the user's channel conditions, the requested video content, and the cache hit results of the MEC server. It will transcode the video bitrate requirements of different users and transmit the processed data packets to each user group g in a multicast manner, where the transcoding delay of the MEC server is recorded as The transmission sub-phase delay from the MEC server to group g is recorded as The cloud server transmits the data that the MEC server misses to each group of users via the MEC server. The delay of the sub-stage from the cloud server to the MEC server during the supplementary transmission is recorded as The delay of the supplementary transmission sub-phase from the MEC server to group g is recorded as After receiving the data from the cloud server and MEC server, user i decodes and renders it, and provides it to the user for playback. The computational sub-stage delay of user i in group g is recorded as

[0170] like Figure 3 As shown in the figure, the end-to-end delay of user i from the time it sends a field of view request to the time it can play the lth video block in working period l can be expressed as The max{·} operation is because the MEC server can request the cloud server to retransmit data in parallel with the MEC server transcoding and multicasting the hit data to the user. g ] 1×G′ represents the user grouping indication strategy, G′ is the total number of user groups that meet certain grouping requirements; Indicates the bitrate strategy for transmitting tiles within each group, where the indicator variable Indicates whether the tile with coordinates (n,m) is at rate r κ Transmit to all users in user group g; use B = [B g ] 1×G′ Indicates the bandwidth allocation strategy of the base station connected to the MEC server to each unicast or multicast group; P = [P g ] 1×G′ Indicates the strategy for allocating transmit power to each unicast or multicast group.

[0171] When the transmission strategy π = (y, x, B, P) is known, the specific process of end-to-end delay analysis of working cycle l is as follows:

[0172] (1) MEC server cache stage

[0173] According to the diffusion process approximation method, at time t1 when the working cycle l starts, the queue length of the MEC server receiving data packets from time zero can be represented by a continuous variable X1(t1). At the end of the cache phase (i.e., the end of the D1 phase), the number of data packets cached by the MEC server x1 satisfies the following probability density function:

[0174]

[0175] in: and are the drift coefficient and diffusion coefficient of the diffusion process respectively, where Δt1 represents the increment of time t1, The maximum number of packets of all N×M tiles that can be cached by the MEC server during the caching phase of duration D1 is called the absorption boundary. Here r0(n,m) represents the initial bitrate of tile (n,m) cached by the MEC server, which is set by the cloud server according to the aforementioned bitrate setting rules based on the positional relationship between tile (n,m) and the predicted FoV area.

[0176] Define the variable Z (l) It represents the number of complete tiles owned by the MEC server after the cache of period l is completed. Its probability mass function is P(Z (l) =z); let θ(z) represent the number of packets cached at time D1, which satisfies:

[0177]

[0178] According to the definition, P(Z (l) = z) is:

[0179]

[0180] (2) Calculation and transmission phase for user i in group g

[0181] The computing and transmission phase includes the MEC server transcoding sub-phase, the transmission sub-phase from the MEC server to the grouped users, the supplementary transmission sub-phase from the cloud server to the MEC server, the supplementary transmission sub-phase from the MEC server to the grouped users, and the computing sub-phase on the user side.

[0182] 2.1 MEC server transcoding sub-phase: When the user watches the video data received in the previous cycle at the end of cycle l, the MEC server also completes the active cache of this cycle; Indicates whether the MEC server caches the complete data of the tile with coordinates (n, m) in the lth period, where Indicates that the MEC server completely caches tile (n, m), otherwise it is 0. The MEC server receives the actual view request information of each user i at the beginning of the transcoding sub-phase make Indicates that user i starts to watch tile(n,m) in the l+1th working cycle according to FoV calculation sub-phase, and vice versa Use collection represents the set of all tile coordinates that user i needs to watch in the l+1th working cycle, and is expressed as Represents the set consisting of the sets of tile coordinates required for viewing by all users in the l+1th working cycle.

[0183] The MEC server can transcode the cached tiles according to the user's bit rate requirements. A feasible transcoding strategy is: represents the set of tiles cached in the MEC server in period l and requested by user i, according to the threshold Check each user and tile from high to low. If There exists tile(n,m) whose bit rate is not within the acceptable bit rate range for user i, that is, Then the code rate of tile(n,m) is converted from r0(n,m) to

[0184] Because of multiple transcoding, each tile may have multiple bitrate versions. The set of bitrates of tile (n, m) in the MEC server is recorded as The subscript j indicates that this is the jth transcoded version of the tile. Indicates whether tile(n,m) is from r κ The code rate is converted to r κ′ Bit rate, It means it is transcoded, otherwise it is equal to 0. κ >r κ′ ; The total transcoding delay in cycle l can be expressed as:

[0185]

[0186] Where: W M is the computing power of the MEC server.

[0187] It can be seen from formula [6] that the calculation time of the MEC server at this stage is consistent for all users, so the conditional probability mass function of the transcoding delay of the MEC server can be expressed as:

[0188]

[0189] 2.2 Transmission sub-phase from MEC server to user i in group g: We use Z 2,g (t2) represents the number of cache hit packets received by group g users from the base station connected to the MEC server at time t2. The reception of the packet is marked by the successful reception of the data of the worst channel user in the group. The diffusion process is used to model the packet transmission process, and the continuous process X is used to model the packet transmission process. 2,g (t2) Replace a discrete number of packets Z 2,g (t2). The initial queue length is 0, and the absorption boundary is the total number of data packets that the MEC server can transmit to group g users at this stage. in Indicates the total amount of data that the base station needs to transmit for group g users, satisfying:

[0190]

[0191] in: represents the bitrate set of tile(n,m) cached in the MEC server that meets the viewing requirements of all users in group g in the l+1th cycle, where It represents the bitrate set of tile (n, m) cached in the MEC server that meets the viewing needs of user i in the l+1th working cycle.

[0192] According to formula [3], for group g, the number of data packets that can be transmitted to users in group g at time t2 is X 2,g (t2) reaches The conditional probability density function of for:

[0193]

[0194] in: and are the drift coefficient and diffusion coefficient of the diffusion process respectively, and Δt2 represents the increment of time t2.

[0195] Given the initial and boundary conditions, we can find The conditional cumulative distribution function and conditional probability density function are satisfied:

[0196]

[0197]

[0198] in:

[0199] 2.3 The supplementary transmission sub-phase from the cloud server to the MEC server: the amount of data that the cloud server needs to transmit to the MEC server It can be expressed as:

[0200]

[0201] Among them: sgn(·) is a sign function, which returns 1 when its argument is greater than 0 and returns 0 when it is equal to 0; when the cloud server transmits the data to the MEC server, tiles (n, m) with the same bit rate but appearing in different groups only need to be transmitted once, and the sgn(·) function is used to count the number of times each tile is transmitted.

[0202] Similar to equations [9] to

[11] , the data transmission process from the cloud server to the MEC server is modeled using a diffusion process; the initial queue length of the MEC server cache is 0, and the absorption boundary is The drift coefficient and diffusion coefficient of the transmission process from the cloud server to the MEC server are α1 and β1 respectively. The conditional cumulative distribution function and conditional probability density function are:

[0203]

[0204]

[0205] 2.4 Supplementary transmission from MEC server to group g users: The MEC server needs to transmit the supplementary data received from the cloud server to each user group. The total amount of data that group g users need to receive is:

[0206]

[0207] in: represents the set of tiles that user i needs to watch in the l+1th period but is not cached by the MEC server; represents the bitrate set of the supplementary tile (n, m) in the cloud server that meets the viewing requirements of all users in group g in the l+1th cycle, where represents the bitrate set of tile(n,m) in the cloud server that meets the viewing requirement of user i in the l+1th period.

[0208] From formula

[15] , we can see that the number of data packets that the MEC server transmits to users in group g is According to formula

[11] , the transmission delay of group g users to receive all data in this stage is The conditional probability density function is:

[0209]

[0210] 2.5 User i’s calculation sub-stage: The user end needs to decode and render the received data and play it in the user terminal. The total amount of data that user i needs to process in this stage is for:

[0211]

[0212] The corresponding calculation time required is:

[0213]

[0214] Where: W U The computing power of the user-end device.

[0215] Therefore, the probability distribution of the delay of decoding and rendering at user i is:

[0216]

[0217] In summary, if Figure 3 The end-to-end delay from when user i in group g sends a real view request to when the user can re-watch the new video chunk in one cycle is given as The conditional probability density function can be expressed as:

[0218] Where: The second term in the integral term is given by equation

[16] , and the first term in the integral term is expressed as Conditional probability density function.

[0219] Combining equations

[11] and

[14] , the conditional probability density function of U can be expressed as:

[0220]

[0221] When the end-to-end delay from when user i sends a real view request to when he can re-watch the new video block is greater than the preset delay threshold D th When , the user will feel dizzy and the quality of experience will be significantly reduced. In summary, in period l, when the cache set of a given MEC server Request Collection and transmission strategy π, the delay violation probability of user i in group g It can be defined as the end-to-end delay from when user i in group g sends a real view request to when he can re-watch the new video chunk The probability of being greater than the threshold is as follows:

[0222]

[0223] like Figure 4As shown, we take the lth working cycle as an example to illustrate the user grouping in each working cycle, the code rate selection of each packet transmission tile, and the bandwidth and transmission power allocation method for each packet by the base station.

[0224] The present invention optimizes the use of system resources and improves the user service quality through the multicast method. In a system with I users, the user groups have a total of G=2 I -1 possibility, but not every group can be activated. The groups that can be activated need to meet certain rate constraints: ① At least one tile is required by all users in the group. ② For tiles that satisfy ①, their bitrate can meet the requirements of all users in the group. If tile (n, m) has been cached by the MEC server and is needed by all users in group g, but If tile (n, m) is not cached by the MEC server but is needed by all users in group g, but Let G′ be the number of groups remaining after removing the groups that do not satisfy the constraints, Represents the set of groups remaining after removing groups that do not satisfy the constraints.

[0225] The following is an optimization problem modeling with the transmission strategy π = (y, x, B, P) as the optimization variable and the goal of maximizing the sum of the received bit rates of all users in the lth cycle:

[0226]

[0227]

[0228]

[0229]

[0230]

[0231]

[0232]

[0233]

[0234]

[0235]

[0236]

[0237]

[0238] Formula

[24] indicates that the probability that the end-to-end delay of each user in group g exceeds the delay threshold needs to be less than the delay violation probability threshold P out,th , the probability is a function of x, y, B and P; Equation

[25] indicates that if tile (n, m) is required by user i, the tile must be transmitted to user i, and user i only needs to receive the tile once; Equations

[26] and

[27] are indicator variables and g The value range of ; Equations

[28] and

[29] respectively indicate that the bandwidth allocated to each user group and the sum of the bandwidths should be greater than or equal to 0 and not exceed the total bandwidth B of the channel; Equations

[30] and

[31] indicate that the transmit power allocated to each user group and the sum of the powers should be greater than or equal to 0 and not exceed the total transmit power P of the base stations connected to the MEC server; θ in Equations

[32] and

[33] is a sufficiently large positive number, indicating that if group g is not activated, the bandwidth and transmit power allocated to group g are 0; Equation

[34] indicates that if group g is not activated, no tile and code rate will be allocated to group g.

[0239] Problem OP1 is a mixed integer nonlinear programming problem. We transform it into a two-level optimization problem to design a solution algorithm. The upper-level optimization problem is the user grouping problem, and the lower-level optimization problem is the code rate selection and communication resource allocation problem.

[0240] The user grouping strategy can be modeled as:

[0241]

[0242] st type

[27]

[0243]

[0244] The introduction

[36] ensures that all tiles required by users under the current grouping strategy y are transmitted to users under the premise of flexible grouping strategies and a larger feasible solution range.

[0245] For problem OP2, the ant colony algorithm is used to make grouping decisions. The algorithm consists of three parts: grouping mode selection, decision feasibility evaluation and pheromone management, which are discussed as follows:

[0246] Grouping mode selection: In the ant colony algorithm, the goal of ants’ movement is to find the best path to the destination, where the distance traveled by ants is heuristic information, and ants can communicate with each other using pheromones; in the proposed algorithm, the location selection of ants is converted into the selection of grouping decision mode for the task, and the sum of the weighted delay and the inverse of the total bit rate of each group is used as heuristic information. It should be noted that if the mode with the highest pheromone concentration and heuristic information is selected every time, the algorithm will converge to the local optimal solution prematurely and fail to find the global optimal solution. Therefore, randomness is added to the allocation rules as a remedial strategy. Some choices need to follow the allocation strategy with the highest pheromone concentration and heuristic information, and some choices need to follow the random allocation strategy of probability distribution. How to choose from... Select the appropriate group * The strategy is given by:

[0247]

[0248]

[0249] Where: g * is the index of the group selected for each grouping decision, q is a random number uniformly distributed between [0,1], q0 is a parameter between [0,1], τ g is the pheromone of group g, η g is the heuristic information of group g, β∈[0,1] is the parameter that determines the relative importance between pheromone and heuristic information, and Y is obtained by the roulette rule according to the activation probability p g An indicator variable that selects whether the group is active.

[0250] The heuristic information of group g is expressed as:

[0251]

[0252] Where: The weighted parameter ω∈[0,1] is the balance coefficient reflecting the end-to-end delay and bit rate of users in group g, C delay,g and C rate,g They represent the delay weight and code rate weight of group g respectively, and their expressions are:

[0253]

[0254]

[0255] in: and are the maximum delay of users in group g and the total bit rate of users in group g mapped to the range of [0,1] using the min-max normalization method, respectively, and are expressed as:

[0256]

[0257]

[0258] in: It represents the delay mean estimation of user i in group g in period l after multiple samplings under different channel conditions.

[0259] Decision feasibility evaluation: To determine whether the generated grouping decision y is feasible, the set bit rate selection variable x and the obtained y can be substituted into formula

[36] to check whether it is valid.

[0260] Pheromone management: In the ant colony algorithm, pheromones are needed to generate reasonable grouping decision solutions. Pheromone management is divided into local pheromone update and global pheromone update: When the mode selection operation selects group g * After updating group g * local pheromones, so as to make better use of pheromone information; when a set of feasible decisions (y * ,x * ,B * ,P * ) is performed globally to update the pheromone with the goal of increasing the information value of promising solutions.

[0261] Local pheromone renewal: for group g * The formula for updating the local pheromone is:

[0262] τ g* =(1-χ)τ g* +χτ0

[44]

[0263] Where: χ∈[0,1] is the attenuation coefficient of the pheromone, τ0 is the initial value of the pheromone; if τ g <τ0, that is, when the pheromone concentration is less than the initial value, it is necessary to increase the pheromone concentration of group g to increase the possibility of selecting group g; otherwise, τ g >τ0, it is necessary to reduce the pheromone concentration of group g to reduce the probability of assigning group g under different conditions and achieve search diversification.

[0264] Global pheromone update: The global pheromone update rule can be expressed as:

[0265]

[0266] Where: Δτ sum Represents decision (y * ,x * ,B * ,P * ) under the total pheromone value-added, similar to formula

[39] , can be expressed as:

[0267]

[0268] in: and are the global delay weight and bit rate weight, respectively expressed as:

[0269]

[0270]

[0271] Using a min-max normalization method similar to local pheromone, in the decision (y * ,x * ,B * ,P * ) and then use and represents the code rate indicator variable and group activation variable value under this decision, then the total delay of group g after processing in cycle l is and total bit rate Respectively expressed as:

[0272]

[0273]

[0274] in: Represents decision (y * ,x * ,B * ,P * ) is the delay mean estimate obtained after multiple samplings under different channel conditions for user i in group g in the next cycle l.

[0275] In summary, Algorithm 1 for grouping decision-making using the above-mentioned ant colony algorithm is solved through the following steps:

[0276] Step 1: Initialize the set of groups to be activated Initialize all user group indicator variables y g = 0; according to the bit rate set in the MEC server and the cloud server that meets the block requirements of group g users and Initialize the rate selection variable x0 with the minimum bit rate of the tile with coordinates (n, m) in the group; initialize the bandwidth allocation variable B0 and the power allocation variable P0 by evenly allocating the bandwidth and power of each group;

[0277] Step 2: Select a strategy based on the grouping pattern of the ant colony algorithm from Select the group to be activated in * , and let y g* =1;

[0278] Step 3: Group g * From the set of groups to be activated Removed;

[0279] Step 4: According to τ g* =(1-χ)τ g* +χτ0 updates the local pheromone in the ant colony algorithm;

[0280] Step 5: Check whether the grouping strategy y satisfies equation

[36] ;

[0281] Step 6: Repeat steps 2 to 5 until equation

[36] is satisfied;

[0282] Step 7: Output the grouping decision y.

[0283] After obtaining the grouping decision y of the upper layer optimization, the lower layer optimization needs to solve the following optimization problems of code rate selection and inter-group wireless resource allocation:

[0284]

[0285] st formula

[24] ~

[26] , formula

[28] ~

[34]

[0286] OP3 is a mixed integer nonlinear programming problem. The non-convexity of this problem is given by the complex integral in

[24] . Therefore, we change the formula

[24] Perform the following conversion:

[0287] (1) Introducing the approximate duration of the calculation and transmission phases To transform It can be expressed as:

[0288]

[0289] Where: E(R1) and E(R 2,g ) represent the average transmission rates from the cloud server to the MEC server and from the base station connected to the MEC server to the users in group g, respectively.

[0290] It should be noted that although we do not know the value of E(R1), we can scale E(R1) to simplify the calculation. From Jensen’s inequality and the arrival rate of data packets from the cloud server to the MEC server, we know that Substituting this inequality into equation

[52] , equation

[24] can be rewritten as:

[0291]

[0292] Where: ξ is the optimization parameter between [0,1], because Therefore, when equation

[53] holds, It must also be true.

[0293] (2) Yes Although

[53] avoids the probability constraint, this constraint is still difficult to handle directly. We introduce a relaxation variable related to group g For formula

[53] The function performs relaxation:

[0294]

[0295] Formula

[54] can be further expanded into multiple simplified constraints:

[0296]

[0297]

[0298] Formula

[56] is the final convex form of the upper limit constraint of the delay in the supplementary transmission phase from the cloud server to the MEC. For formula

[55] , due to The existence of is still a complex non-convex constraint, so we further introduce auxiliary variables Let equation

[55] become a tighter constraint, as shown in the following inequality:

[0299]

[0300] This inequality can be broken down into the following two expressions:

[0301]

[0302]

[0303] Add here The purpose is to ensure that the denominator is positive, and equation

[59] is a convex constraint.

[0304] (3) The upper limit constraint of the average transmission rate from the base station to the users in group g is processed; when the small-scale fading of the channels between all users and the base stations connected to the MEC server obeys independent and identically distributed Rayleigh fading and the path loss exponent is β p The transmission rate of users in group g is defined by the transmission rate of the user with the worst channel condition in group g. The average transmission rate from the base station connected to the MEC server to the users in group g can be calculated. in d i is the distance between user i and the base station in group g, For non-convex exponential integrals Using the inequality constraints in the literature "Some elementary inequalities relating to the Gamma and incomplete Gamma function (author W. Gautschi, published in Journal of Mathematics and Physics, vol. 38, pp. 77-81, April 1959)" We can get E(R 2,g ) has a convex lower bound of:

[0305]

[0306] (4) Use the Successive Convex Approximation (SCA) algorithm to process the non-convex constraint formula

[58] ; for the other non-convex component in formula

[58] The SCA algorithm is used to decompose it into the Difference-of-Two-Concave-Functions (DC) and then gradually approximate it in a convex form. can be rewritten as:

[0307]

[0308] Solved by DC method The basic process is as follows: Assume that the iteration number identifier of the DC algorithm is h, record The value after the hth iteration is Let x h is the value of variable x after the hth iteration, is a function related only to the variable x, and its value after the hth iteration is expressed as First, in the h-1th iteration, we get and and update the remaining parameters; then, use the first-order Taylor expansion to calculate Perform a linear approximation; finally, by solving the convex problem obtained after approximation, we get and

[0309] For convex functions exist Taylor expansion at gives the following lower bound:

[0310]

[0311] Combining equation

[61] , we can obtain the following inequality:

[0312]

[0313] Combined with

[60] , the lower bound

[58] of the transmission rate from the MEC server to group g at each iteration can be transformed into the following constraint:

[0314]

[0315] The same method is used to approximate the transmission delay of the supplementary transmission sub-phase from the MEC server to the user in group g. Introducing slack variables and Simplifying, we can get the following constraints on the slack variables:

[0316]

[0317] The lower bound of the transmission rate of the supplementary transmission from the MEC server to the users in group g at the hth iteration can be converted into:

[0318]

[0319] in: for The value after the hth iteration.

[0320] (5) Use and Respectively replace the and The total end-to-end approximate duration satisfies the following constraints:

[0321]

[0322] In order to ensure that the end-to-end delay is as small as possible under the constraint setting, ξ is introduced into the objective function as a penalty term. Then, at each iteration h, OP3 is transformed into the following optimization problem:

[0323]

[0324] st0≤ξ≤1

[69]

[0325] st formula

[25] , formula

[26] , formula

[28] to

[34] , formula

[56] , formula

[59] , formula

[64] to

[67]

[0326] OP4 is a mixed integer regular convex programming problem, which can be solved by the branch and bound algorithm.

[0327] The steps of solving OP3 in resource allocation algorithm 2 based on alternating optimization and difference of convex functions are as follows:

[0328] Step 1: Based on the bit rate set in the MEC server and cloud server that meets the block requirements of each group of g users and Initialize the rate selection variable x0 with the minimum rate of the tile with coordinates (n, m) in each group, and initialize the iteration count identifier h = 1;

[0329] Step 2: Calculate the hth iteration time according to equations [8] and

[15] and The value of

[0330] Step 3: Solve OP4 using the branch and bound method;

[0331] Step 4: Iterate step 2 to step 3 until the objective function of OP4 converges;

[0332] Step 5: Calculate the

[0333] Step 6: If all groups and all users If both satisfy the delay requirements in formula (24), then the optimization algorithm 2 succeeds, otherwise, algorithm 2 fails;

[0334] Step 7: Output the optimization result (x, B, P).

[0335] In summary, the framework of the upper and lower double-layer joint optimization algorithm 3 designed in the present invention for solving user grouping, intra-group code rate selection and wireless resource allocation is as follows:

[0336] Step 1: Initialize the iteration indicator variable iter = 1; according to the bit rate set in the MEC server and cloud server that meets the required block of each group g users and Initialize the bitrate selection variable x0 with the minimum bitrate of the tile with coordinates (n, m) in the group; initialize the bandwidth allocation variable B0 and the power allocation variable P0 by evenly allocating bandwidth and power to each user group; initialize (y * ,x * ,B * ,P * ) is empty;

[0337] Step 2: When the number of iterations iter is less than the maximum number of iterations iter max When pop user grouping decisions {y1,y2,...,y pop};

[0338] Step 3: For each user grouping decision, generate the code rate selection, bandwidth and transmit power allocation results (x1, B1, P1), (x2, B2, P2), …, (x pop,B pop ,P pop );

[0339] Step 4: Select the strategy that maximizes the bit rate received by the user end from the pop allocation strategies:

[0340] Step 5: Update via equation

[45] Global pheromones under;

[0341] Step 6: If Update

[0342] Step 7: Repeat steps 2 to 6 until the maximum number of iterations iter is reached. max ;

[0343] Step 8: Output (y * ,x * ,B * ,P * ).

[0344] In order to verify the effectiveness of the present invention, we use the 360° VR video “BlueWorld” to test the performance of the solution proposed in the present invention.

[0345] We use the same quantization parameters as in the paper "Optimal wireless streaming of multi-quality 360VR video by exploiting natural, relative smoothness-enabled, and transcoding-enabled multicast opportunities (author K. Long et al., published in IEEE Trans. Multimedia, vol. 23, pp. 3670-3683, 2021)" to encode the video, and the bitrate of each tile at different quality levels of the video is expressed as Minimum bitrate requirement for each user from The algorithm was tested for 50 cycles under 5 kinds of random positions of users, and the test was repeated 25 times for each cycle l. In the ant colony algorithm, the population size and the maximum number of iterations were set to pop = 10 and iter max=50, weighting parameter ω = 0.5, and the other parameter settings of the ant colony algorithm are similar to those in the literature "Abilevel optimization approach for joint offloading decision and resource allocation in cooperative mobile edge computing (author P.-Q. Huang et al., published in IEEE Trans. Cybernetics, vol. 50, no. 10, pp. 4228-4241, 2020)": β = 2, χ = 0.1, q0 = 0.9, τ0 = (IΔτ sum (y0,x0,B0,P0)) -1 , where y0, x0, B0 and P0 are initial values, and other default parameter settings in the simulation are shown in Table 1:

[0346] Table 1

[0347]

[0348]

[0349] The performance of the proposed algorithm was studied using indicators such as average user video bitrate, algorithm success rate, and cumulative distribution function of end-to-end delay. The average user video bitrate is the average of the sum of all user video bitrates within a period, the algorithm success rate refers to the proportion of resource allocation algorithms with solutions under delay constraints, and the cumulative distribution function of end-to-end delay is obtained by solving the algorithm each time to obtain the worst user's end-to-end delay statistics; at the same time, the algorithm of the present invention is compared with the following three schemes:

[0350] Unicast algorithm: All users use unicast services alone, and the other constraints and optimization variables remain unchanged. The same optimization method as Algorithm 2 is used to solve the user's bit rate selection, bandwidth and power allocation.

[0351] Average bandwidth allocation algorithm: The user grouping algorithm, code rate selection, and power resource allocation solution are the same as Algorithms 1 to 3, but the bandwidth of all multicast and unicast groups is evenly allocated.

[0352] Exhaustive search algorithm: The upper-layer grouping decision algorithm is replaced by exhaustive search, and the lower-layer algorithm uses the same optimization method as Algorithm 2 to solve the bit rate selection, bandwidth, and power allocation of users in the group.

[0353] Figure 5The relationship between the number of algorithm cycles (statisticed by the number of iterations of Algorithm 3) and the average user video bit rate under different bandwidths and numbers of users is shown. It can be seen from the figure that the algorithm of the present invention can obtain high-quality solutions in the initial stage and then converge quickly. When the number of users is small (such as I = 3), the increase in bandwidth from 100MHz to 200MHz has almost no effect on the convergence of the algorithm, which means that even with a bandwidth of 100MHz, the communication resources are still sufficient and there is no limit on the user's bit rate. However, as the number of users increases (such as I = 9), if all users are to be served at a bandwidth of 100MHz, the system can only reduce the user's receiving bit rate; when the bandwidth is increased to 200MHz, the user's bit rate is significantly improved.

[0354] Figure 6 The influence of different numbers of users on the success rate of the proposed algorithm is shown. It can be seen from the figure that when the number of users is small (such as I≤3), due to sufficient resources, the success rates of the algorithm of the present invention and the comparison algorithm are very high; as the number of users increases, the success rates of the unicast algorithm and the average bandwidth allocation algorithm drop significantly, while the algorithm of the present invention uses multicast and effective communication resource allocation and bit rate selection, and has a success rate similar to that of the exhaustive search algorithm. For example, when the number of service users I=9, the success rate of the algorithm of the present invention is about 91.91%, which is only 0.34% lower than the exhaustive search algorithm, 18.64% higher than the average bandwidth allocation algorithm, and 41.91% higher than the unicast algorithm.

[0355] Figure 7 The relationship between the number of users and the total average bit rate under different algorithms is shown. It can be seen from the figure that as the number of users increases, the average user video total bit rate of the algorithm of the present invention is higher than that of the unicast algorithm and the average bandwidth allocation algorithm; for example, when I=7, its total average bit rate is 4.87% and 18.07% higher than that of the average bandwidth allocation algorithm and the unicast algorithm, respectively. Compared with the exhaustive search algorithm, the total average bit rate of the algorithm of the present invention is only 0.36% different, but the execution time is significantly reduced compared with the exhaustive search algorithm.

[0356] Figure 8 The end-to-end delay performance of each algorithm under different numbers of users is compared. It can be seen from the figure that as the number of access users increases, the end-to-end delay of the algorithm increases; under the same number of access users, the end-to-end delay of the algorithm of the present invention is significantly smaller than that of other comparison algorithms, and is close to the end-to-end delay of the exhaustive search algorithm.

[0357] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made to the present invention by those skilled in the art based on the disclosure of the present invention should be within the scope of protection of the present invention.

Claims

1. A 360-degree virtual reality video multi-user transmission system using transcoding and multicasting, including a cloud server, an MEC server, and multiple client terminals. If there is a client terminal that needs a 360-degree virtual reality video service in the system, the client terminal index is represented as Features: The cloud server stores 360-degree virtual reality videos and generates corresponding saliency maps through neural networks. When encoding the video, it divides it into N×M tiles using equidistant cylindrical projection, and compresses the data of multiple video frames of each tile into a set of picture sequences, where N and M are both custom natural numbers. At the same time, the cloud server predicts FoV based on the saliency map and transmits the corresponding video data packet to the MEC server. In order to meet the video bit rate requirements of different user terminals, the cloud server uses advanced coding technology to pre-encode the video blocks into bit rate versions with different quality levels. Indicates the optional bit rate, where r1<r2<...<r max ;make represents the minimum video bit rate requirement of user terminal i, σ is the tolerance threshold of the user terminal to the bit rate change within the field of view, and the acceptable bit rate range of user terminal i is The MEC server will make grouping decisions for the user end for multicast based on the channel conditions of the user end, the requested video content, and the cache hit results of the MEC server, and will cache and transcode the video data packets sent by the cloud server and transmit the video data packets to the corresponding user end; In addition to receiving data sent by the MEC server, the user terminal also needs the cloud server to provide it with data packets that the MEC server missed. After decoding and rendering, the data obtained from the MEC server and the cloud server are combined into a displayable image to complete the conversion of the 2D video to a 360-degree spherical video and play it. The system adopts a periodic working mode. Each working cycle consists of a caching phase and a calculation and transmission phase. The video data is divided in the time domain into multiple video blocks. The duration of each video block is D1. It is stipulated that the index of the video block simultaneously watched by each user terminal in the lth working cycle is l-1, where l is a natural number greater than 1. In the caching stage, the cloud server transmits the corresponding video data packets to the MEC server according to the field of view prediction results. The duration of the caching stage is the duration of a video block, i.e., D1. At the beginning of the calculation and transmission stage, according to the FoV of each user terminal and the cache results of the MEC server, the MEC server makes multicast grouping decisions for all user terminals and transcodes the data packets for unicast or multicast data transmission to each user group. At the same time, the MEC server immediately starts to request the cloud server to retransmit the data required by the user terminal but not hit in the MEC server cache. This part of the data is transmitted sequentially from the cloud server to the MEC server. After receiving the data retransmitted by the cloud server, the MEC server transmits this data to different user terminals in groups. After receiving the data from the MEC server and the cloud server, the user terminal decodes, renders and plays the video data packets.

2. The 360-degree virtual reality video multi-user transmission system according to claim 1, characterized in that: For the cache phase, at the beginning of the lth working cycle, the cloud server predicts the user's field of view probability distribution P at the beginning of the l+1th working cycle based on the saliency map of the lth video block. F =[P F (n (l) ,m (l) )] N×M , where P F (n (l) ,m (l) ) indicates that at the beginning of the l+1th working cycle, the user's viewpoint falls into the coordinates (n (l) ,m (l) ) in the tile, and are the sets of horizontal and vertical coordinates of the tile respectively; define P R =[P R (n (l) ,m (l) )] N×M is the probability matrix of the user requesting a tile, where Indicates that at the beginning of the next cycle, the coordinates are (n (l) ,m (l) ) falls within the user's FoV area, indicating the function The coordinates are (n (l) ,m (l) ) falls into the tile with coordinates (n c ,m c ) is the FoV area centered on the tile, otherwise the indicator function is equal to 0. and They represent the set of horizontal and vertical coordinates of the tiles within the user's field of view, respectively. a1 is the number of tiles contained in the user's field of view in the horizontal and vertical directions.

3. The 360-degree virtual reality video multi-user transmission system according to claim 2, characterized in that: After the caching phase, the calculation and transmission phase begins. At this time, the real FoV information of any user terminal i It has been determined that Indicates that user terminal i starts to determine the tile with coordinates (n, m) in the l+1 working cycle according to FoV calculation sub-phase, and vice versa Use collection It represents the set of all tile coordinates required for viewing by client i in the l+1th working cycle, and is expressed as Indicates that all users correspond to Composed of a collection, using a collection Records whether the MEC server caches the complete data of the tile in the lth cycle, where Indicates that the MEC server completely caches the tile with coordinates (n,m), otherwise make It represents the set of coordinates of tiles cached in the MEC server and requested by the user terminal i in the lth cycle; the calculation and transmission stage includes the MEC server transcoding sub-stage, the transmission sub-stage from the MEC server to the grouped user, the supplementary transmission sub-stage from the cloud server to the MEC server, the supplementary transmission sub-stage from the MEC server to the grouped user, and the calculation sub-stage of the user terminal.

4. The 360-degree virtual reality video multi-user transmission system according to claim 3, characterized in that: In the MEC server transcoding sub-phase, when receiving new field of view requests from different users, the MEC server makes grouping decisions for user multicast based on the field of view requests and user channel conditions of the users. At the same time, it checks whether there are video data packets required by each user in the cache, and transcodes according to the video quality requirements of the users in the group. make represents the bitrate set of the tile with coordinates (n, m) in the MEC server that meets the viewing needs of user terminal i in the l+1th working cycle, where Represents the set of bitrates of tiles with coordinates (n,m) in the MEC server; In the transmission sub-phase from the MEC server to the grouped users, for different multicast or unicast groups, the MEC server will make decisions on the channel bandwidth, transmission power, and video bitrate for each user group based on the channel quality of the user group, the bandwidth and power of the system, and the video quality requirements of the user group; In the supplementary transmission sub-phase from the cloud server to the MEC server, the cloud server sequentially supplements the video data packets that the MEC server does not have cache and that the user needs. represents the bitrate set of tiles with coordinates (n, m) in the cloud servers not cached by the MEC server that meet the viewing needs of user terminal i in the l+1th cycle, where represents the set of coordinates of tiles that are needed by client i to watch in the l+1th period but are not cached in the MEC server; In the supplementary transmission sub-stage from the MEC server to the grouped user, the MEC server performs supplementary transmission of the data packet according to the grouping and resource decision, and in order to improve the resource utilization of the system, the supplementary transmission from the cloud server to the MEC server and the transcoding of the MEC server are carried out simultaneously; in addition, for the data hit by the MEC server transmission and the data supplemented by the cloud server, the wireless transmission of these two parts of data is realized in a serial manner; In the user-side calculation sub-phase, the user side decodes and renders the received video data packets. When all user sides complete the calculation and decoding, the communication and decoding tasks of the lth working cycle are completed, and the user side plays the video frame of the lth video block at the beginning of the l+1th working cycle.

5. A resource allocation method for multicast packet transmission of a 360-degree virtual reality video multi-user transmission system according to any one of claims 1 to 4, characterized in that: When there are I users, the system has a total of G = 2 I -1 The number of multicast or unicast packets, the index of the user group is use Represents the set of users in user group g, using and They represent the bitrate set of the tile with coordinates (n, m) in the MEC server and the cloud server that meets the viewing needs of all users in the user group g in the l+1th cycle; the sufficient conditions for the activation of the user group g include: ① At least one tile is required by all clients in the group. ② For tiles that meet ①, their bitrate can meet the requirements of all users in the group: If the tile with coordinates (n, m) has been cached by the MEC server and is needed by all users in user group g, but If the tile with coordinates (n, m) is not cached by the MEC server but is needed by all users in user group g, but Let G′ be the number of user groups remaining after removing the user groups that do not meet the above conditions, represents the set of user groups remaining after removing the user groups that do not meet the above conditions. Let the indicator variable Indicates whether user group g is activated, using B g and P g They represent the bandwidth and transmission power allocated to user group g, respectively, and y = [y g ] 1×G′ Indicates the indication policy of the user group, using represents the bitrate strategy for transmitting tiles within each user group, where the indicator variable Indicates whether the tile with coordinates (n,m) is at rate r κ Transmit to all users of user group g, using B = [B g ] 1×G′ It represents the bandwidth allocation strategy of the base station connected to the MEC server to each unicast or multicast group, with P = [P g ] 1×G′ It represents the allocation strategy of the transmission power to each user group; then the user grouping, video tile bit rate selection and wireless resource allocation strategy are determined by the optimization problem OP1 of maximizing the sum of the received bit rates of all user terminals. The expression of the optimization problem OP1 is as follows: Where: Formula (2) represents the delay violation probability that the end-to-end delay of any user terminal i in any user group g exceeds the delay threshold Both need to be less than the delay violation probability threshold P out,th , the probability is a function of the user grouping strategy y, the video tile bitrate selection strategy x, the bandwidth allocation strategy B and the power allocation strategy P; Formula (3) indicates that if the tile with coordinates (n, m) is required by user terminal i to watch in the l+1th cycle, the tile must be transmitted to user terminal i, and user terminal i only needs to receive the tile once; Formulas (4) and (5) are indicator variables and g The value range of ; Formulas (6) and (7) respectively indicate that the bandwidth allocated to each user group and the sum of the bandwidths should be greater than or equal to 0 and not exceed the total channel bandwidth B; Formulas (8) and (9) indicate that the transmit power allocated to each user group and the sum of the powers should be greater than or equal to 0 and not exceed the total transmit power P of the base stations connected to the MEC server; θ in Formulas (10) and (11) is a sufficiently large positive number, indicating that if user group g is not activated, the bandwidth and transmit power allocated to user group g are 0; Formula (12) indicates that if user group g is not activated, no tile and code rate will be allocated to user group g.

6. The resource allocation method according to claim 5, characterized in that: Given a known allocation strategy π = (y, x, B, P), according to the known cache set in the lth working cycle Request Collection And the end-to-end delay under transmission strategy π is greater than D th The conditional probability density function of the delay violation probability in formula (2) is It is expressed as: in: The random variable D represents the end-to-end delay from the time when user i in user group g sends a view request to the time when it can play the lth video block in the system in the lth working cycle. th is a given delay threshold, is the random variable of user i in user group g The conditional probability density function, t (l) is a random variable The corresponding value, represents the random variable of the MEC server transcoding phase latency in the lth working cycle, represents the random variable of the transmission sub-phase delay from the MEC server to the user end in user group g in the lth working cycle, represents the random variable of the delay of the supplementary transmission sub-phase from the cloud server to the MEC server in the lth working cycle, represents the random variable of the delay of the supplementary transmission sub-phase from the MEC server to the user end in user group g in the lth working cycle, A random variable representing the delay of the computation sub-phase of client i in the lth working cycle.

7. The resource allocation method according to claim 6, characterized in that: The conditional probability density function The calculation expression is as follows: in: Represents a random variable The conditional probability density function, u is the value corresponding to the random variable U, express The conditional probability density function of The value of .

8. The resource allocation method according to claim 7, characterized in that: The conditional probability density function as well as The expression is as follows: in: is the conditional cumulative distribution function of U; let t2 be a random variable The corresponding value, t3 is a random variable The corresponding value, t4 is a random variable The corresponding value is and Respectively The conditional cumulative distribution function and conditional probability density function of and Respectively The conditional cumulative distribution function and conditional probability density function of express The conditional probability density function of and They are The conditional cumulative distribution function and conditional probability density function of The value of and They are The conditional cumulative distribution function and conditional probability density function of The value of ; is the absorption boundary of the transmission sub-phase from the MEC server to the user end in user group g, is the absorption boundary of the supplementary transmission sub-stage from the cloud server to the MEC server, is the absorption boundary of the sub-stage of MEC server supplementing the transmission to the user end in user group g, α 2,g and β 2,g They represent the drift coefficient and diffusion coefficient of the transmission process from the MEC server to the user end in user group g, α1 and β1 represent the drift coefficient and diffusion coefficient of the transmission process from the cloud server to the MEC server, 1 / λ 2,g and V 2,g are the mean and variance of the interval variable of the arrival time of the data packet received by user group g from the MEC server, 1 / λ1 and V1 are the mean and variance of the interval variable of the arrival time of the data packet received by the MEC server from the cloud server, and the expression of the function Φ is represents the total amount of cache hit data that the MEC server needs to transmit to the users in user group g during the lth working cycle, It represents the total amount of data transmitted from the cloud server to the MEC server in the lth working cycle. represents the total amount of data transmitted by the MEC server to the user end in user group g during the lth working cycle, k is the size of each video data packet, sgn() is the sign function, R1(t) and R 2,g (t) represents the wired network transmission rate from the cloud server to the MEC server and the wireless network transmission rate from the MEC server to the user end in user group g at time t, respectively; E() represents the mean, and Var() represents the variance.

Citation Information

Patent Citations

  • Video data distribution method and system method and system for distributing video data

    CN101800885A

  • Method and device for transmitting data files to client side

    CN102833219A

  • User experience-driven multi-party video conference transcoding and multicast routing method and device

    CN111866438A

  • 360-degree virtual reality video transmission system based on edge calculation and active caching and parameter optimization method

    CN112995636A

  • QoE-driven wireless VR video adaptive transmission optimization method and system

    CN114640870A