Panoramic video transmission method for resisting field-of-view prediction error

Abstract

The invention relates to a panoramic video transmission method for resisting view field prediction errors, which belongs to the technical field of communication and network, and comprises the following steps: calculating a rate weighting ratio according to view field prediction errors of different users; according to the obtained rate weighting ratio, sub-carrier resources in the system are allocated to users; and allocating power resources in the system to users according to the obtained rate weighting ratio and the subcarrier resources. According to the method and the device, required network resources can be adaptively allocated to each user according to different view field prediction errors of each user, the user is supported to acquire panoramic video images with different view field areas, adverse effects caused by the view field prediction errors are effectively resisted, and the experience of the user when the user watches a 360-degree video is improved.

Description

Technical Field

[0001] The present invention relates to a panoramic video transmission method for resisting field of view prediction errors, belonging to the technical field of communication and network. Background Art

[0002] Panoramic video provides users with a 360-degree omnidirectional viewing angle, bringing a highly immersive visual experience. However, this immersion comes at the cost of an extremely high video bitrate, which poses a huge challenge to the transmission of panoramic video. Currently, the most advanced 360-degree video transmission technology adopts a transmission scheme based on the field of view (FoV). This scheme divides the 360-degree video into multiple independently transmissible picture units (called tiles), and by only transmitting the tiles within the user's field of view, it avoids the transmission of redundant pictures, thus significantly reducing the bandwidth required for panoramic video transmission.

[0003] Although the field-of-view-based transmission scheme can relieve the transmission pressure of panoramic video, this scheme requires accurate prediction of the user's future field-of-view coordinates. However, the prediction results obtained by existing field-of-view prediction methods all have errors, resulting in a mismatch between the actually transmitted picture and the picture that the user wants to view, seriously affecting the user's video viewing experience. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a panoramic video transmission method for resisting field of view prediction errors. By utilizing the characteristic that in the multi-user panoramic video transmission scenario, the field of view prediction errors of different users are different, it adaptively allocates more network resources to users with larger prediction errors, enabling these users to download more tiles to cover a larger field of view area, thereby avoiding the problem that the predicted field of view deviates from the user's true field of view, resulting in a mismatch between the actually transmitted picture and the picture that the user wants to view, and effectively improving the user experience when viewing panoramic video.

[0005] The present invention adopts the following technical solutions:

[0006] A panoramic video transmission method for resisting field of view prediction errors, comprising the following steps:

[0007] (1) Calculate the rate weighting ratio according to the field of view prediction errors of different users;

[0008] (2) Allocate the subcarrier resources in the system to users according to the rate weighting ratio obtained in step (1);

[0009] (3) Allocate the power resources in the system to users according to the rate weighting ratio calculated in step (1) and the subcarrier resources obtained in step (2).

[0010] Preferably, the implementation process of step (1) is:

[0011] Let the total number of users in the system be denoted as M, and the user set be denoted as The user index is denoted as m; the field of view prediction error of user m is ρ m , then the rate weighting ratio β m of this user is calculated as follows:

[0012]

[0013] where β min = 0, β max = 1, ρ max = max(ρ 1 ,..., ρ m ,..., ρ M ).

[0014] Preferably, in step (2), the subcarrier resource allocation steps are as follows:

[0015] (2.1) Let the total number of subcarriers in the system be denoted as N, and the subcarrier set be denoted as The subcarrier index is denoted as n; the total number of panoramic video segmentation tiles is denoted as S, the total number of tiles in the user's field of view is denoted as F, and the coding rate of each tile is denoted as r; the total system bandwidth is denoted as B, and the total transmission power of the base station is denoted as P tot ; calculate the channel noise ratio {H 1，1 ,..., H m，n ,..., H M，N}, and the calculation formula is:

[0016]

[0017] where H m,n represents the channel noise ratio of user m on subcarrier n; g m，n represents the channel gain between user m and the base station on subcarrier n; N 0 represents the noise power spectral density;

[0018] (2.2) Initialize the transmission rate R m of user m to 0, the subcarrier set allocated to user m, the remaining unallocated subcarrier set the remaining set of users participating in subcarrier allocation where represents the empty set;

[0019] (2.3) For each user m in the set Λ, perform the following steps:

[0020] ① In the remaining unallocated subcarrier set Γ, find the subcarrier n with the maximum channel noise ratio for user m, Hm，n ≥H m，n′ ,n′ ∈ Γ;

[0021] ② Assign the found sub - carrier n to user m, Ω m = Ω m ∪{n}, and update the set of remaining unassigned sub - carriers at the same time, Γ = Γ - {n};

[0022] ③ Calculate the transmission rate of user m according to the following formula:

[0023]

[0024] where p m，n = P tot / N represents the transmission power on sub - carrier n, N 0 is the noise power spectral density;

[0025] (2.4) Find the user m with the lowest weighted transmission rate in the set Λ, β m R m ≤β m′ R m′ , m′ ∈ Λ. If the transmission rate of this user is less than the coding rate of the entire panoramic video, R m <Sr (Sr represents the total number of tiles S multiplied by the coding rate r of each tile), then perform the following steps:

[0026] ① In the set of remaining unassigned sub - carriers Γ, find the sub - carrier n with the largest channel - to - noise ratio for user m, H m，n ≥H m，n′ , n′ ∈ Γ;

[0027] ② Assign the found sub - carrier n to user m, Ω m = Ω m ∪{n}, and update the set of remaining unassigned sub - carriers at the same time, Γ = Γ - {n};

[0028] ③ Calculate the transmission rate of user m according to the Shannon formula:

[0029]

[0030] where p m，n = P tot / N;

[0031] If the transmission rate of this user is greater than the coding rate of the entire panoramic video, R m >Sr, then remove user m from the set Λ of remaining users participating in sub - carrier allocation, Λ = Λ - {m};

[0032] (2.5) Repeat step (2.4) until there are no remaining unassigned sub - carriers,

[0033] Preferably, in step (3), the power resource allocation steps are as follows:

[0034] (3.1) First, obtain the power allocation scheme among users That is, allocate the total power P of the base station tot to M users;

[0035] (3.2) Further allocate the power allocated to each user to the subcarriers belonging to that user.

[0036] Preferably, step (3.1) is specifically:

[0037] ① Solve the following non - linear equations to obtain the preliminary power allocation scheme

[0038]

[0039] where N m represents the number of subcarriers allocated to user m, N 1 represents the number of subcarriers allocated to user 1, N m represents the number of subcarriers allocated to user m; Ξ m and Ψ m are auxiliary variables, expressed as:

[0040]

[0041] ② Use the bisection method to calculate the maximum allocable power value of user m;

[0042] ③ Arrange the users in descending order according to the magnitude of their transmission rates, such that R 1 ≥R 2 ≥...≥R M ;

[0043] ④ According to the calculated maximum allocable power, correct the preliminary user power allocation scheme : If the initial allocated power of user m is less than the maximum allocable power, that is then let If the initial allocated power of user m is greater than or equal to the maximum allocable power, that is then let Meanwhile, allocate the part of the power exceeding to the next user

[0044] ⑤ Repeat step ④ to complete the correction of the preliminary allocated power for all users and obtain the final power allocation strategy among users

[0045] Preferably, in step ②, the bisection method is used to calculate the maximum allocable power value of user m, and the process is as follows:

[0046] a. Initialize the upper bound of the Lagrange multiplier to any value greater than For example, let Initialize the lower bound of the Lagrange multiplier to 0, Determine the value of the iteration stop threshold ξ. For example, let ξ = 0.0001;

[0047] b. Use the following formula to calculate the value of the Lagrange multiplier in the current iteration round:

[0048]

[0049] c. Use the following formula to calculate the power value allocated to user m on the subcarrier:

[0050]

[0051] At the same time, use the Shannon formula to calculate the transmission rate of user m:

[0052]

[0053] d. If R m ≤ Sr, then let If R m > Sr, then let

[0054] e. Repeat steps b to d until the condition

[0055] f. Use the following formula to calculate the maximum allocable power value on the subcarrier of user m:

[0056]

[0057] g. Use the following formula to calculate the maximum allocable power of user m:

[0058]

[0059] Preferably, the process of step (3.2) is as follows:

[0060] ① Solve the following equation to obtain the Lagrange multiplier corresponding to the power P tot，m allocated to user m:

[0061]

[0062] ② Use the water-filling method to calculate the power allocated to each subcarrier:

[0063]

[0064] For the parts not elaborated in the present invention, the prior art can be adopted.

[0065] The beneficial effects of the present invention are as follows:

[0066] The present invention proposes a panoramic video transmission method for resisting field of view prediction errors. By allocating more network resources to users with large field of view prediction errors, this part of users is allowed to download more video tiles to cover a larger field of view area. The proposed scheme can improve the robustness of the panoramic video transmission scheme based on field of view perception. Even if there is a large deviation between the real field of view and the predicted field of view of the user, it can still ensure that the user can watch high-quality video pictures. The present invention can be used in video service systems in scenarios such as virtual reality, VR / AR live broadcast, etc., and can effectively improve the user's video viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a block diagram of a panoramic video transmission system for resisting field of view prediction errors of the present invention;

[0068] Figure 2 It is a schematic flowchart of a panoramic video transmission method for resisting field of view prediction errors of the present invention;

[0069] Figure 3 It is a comparison diagram of the number of video tiles that can be viewed by the method proposed in the present invention and the existing method under different total powers of the user. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments, but not limited to this. For those not elaborated in the present invention, the conventional techniques in the art are adopted.

[0071] Embodiment 1

[0072] A panoramic video transmission method for resisting field of view prediction errors, as Figure 1-2 shown, includes the following steps:

[0073] (1) Calculate the rate weighting ratio according to the field of view prediction errors of different users;

[0074] (2) Allocate the subcarrier resources in the system to users according to the rate weighting ratio obtained in step (1);

[0075] (3) Allocate the power resources in the system to users according to the rate weighting ratio calculated in step (1) and the subcarrier resources obtained in step (2).

[0076] Embodiment 2

[0077] A panoramic video transmission method for resisting field of view prediction error, as described in Embodiment 1, except that the implementation process of step (1) is as follows:

[0078] Let the total number of users in the system be denoted as M, and the user set be denoted as The user index is denoted as m; the field of view prediction error of user m is ρ m , then the rate weighting ratio β m of this user is calculated as follows:

[0079]

[0080] where β min = 0, β max = 1, ρ max = max(ρ 1 ,..., ρ m ,..., ρ M ).

[0081] Embodiment 3

[0082] A panoramic video transmission method for resisting field of view prediction error, as described in Embodiment 2, except that in step (2), the subcarrier resource allocation steps are as follows:

[0083] (2.1) Let the total number of subcarriers in the system be denoted as N, the subcarrier set be denoted as The subcarrier index is denoted as n; the total number of tiles of the panoramic video segmentation is denoted as S, the total number of tiles in the user's field of view is denoted as F, and the coding rate of each tile is denoted as r; the total system bandwidth is denoted as B, and the total transmission power of the base station is denoted as P tot ; calculate the channel noise ratio {H 1，1 ,..., H m，n ,..., H M，N}, and the calculation formula is:

[0084]

[0085] where H m,n represents the channel noise ratio of user m on subcarrier n; g m，n represents the channel gain between user m and the base station on subcarrier n; N 0 represents the noise power spectral density;

[0086] (2.2) Initialize the transmission rate R m of user m to 0, the subcarrier set assigned to user m, the remaining unassigned subcarrier set the remaining user set participating in subcarrier allocation where represents an empty set;

[0087] (2.3) For each user m in the set Λ, perform the following steps:

[0088] ① Find the subcarrier n with the maximum channel noise ratio for user m in the remaining unallocated subcarrier set Γ, such that H m，n ≥H m，n′ , where n′ ∈ Γ;

[0089] ② Allocate the found subcarrier n to user m, such that Ω m = Ω m ∪{n}, and at the same time update the remaining unallocated subcarrier set, Γ = Γ - {n};

[0090] ③ Calculate the transmission rate of user m according to the following formula:

[0091]

[0092] where p m，n = P tot / N represents the transmit power on subcarrier n, and N 0 is the noise power spectral density;

[0093] (2.4) Find the user m with the lowest weighted transmission rate in the set Λ, such that β m R m ≤β m′ R m′ , where m′ ∈ Λ. If the transmission rate of this user is less than the coding rate of the entire panoramic video, R m <Sr, then perform the following steps:

[0094] ① Find the subcarrier n with the maximum channel noise ratio for user m in the remaining unallocated subcarrier set Γ, such that H m，n ≥H m，n′ , where n′ ∈ Γ;

[0095] ② Allocate the found subcarrier n to user m, such that Ω m = Ω m ∪{n}, and at the same time update the remaining unallocated subcarrier set, Γ = Γ - {n};

[0096] ③ Calculate the transmission rate of user m according to the Shannon formula:

[0097]

[0098] where p m，n = P tot / N;

[0099] If the transmission rate of this user is greater than the coding rate of the entire panoramic video, R mIf \(S_m > Sr\), then user \(m\) is removed from the set \(\Lambda\) of remaining users participating in subcarrier allocation, \(\Lambda=\Lambda - \{m\}\);

[0100] (2.5) Repeat step (2.4) until there are no remaining unallocated subcarriers.

[0101] Embodiment 4

[0102] A panoramic video transmission method for resisting field of view prediction errors, as described in Embodiment 3. The difference is that in step (3), the power resource allocation steps are as follows:

[0103] (3.1) First, obtain the power allocation scheme among users That is, allocate the total power \(P\) of the base station tot to \(M\) users. Specifically:

[0104] ① Solve the following non - linear equations to obtain the preliminary power allocation scheme

[0105]

[0106] where \(N\) m represents the number of subcarriers allocated to user \(m\), \(N\) 1 represents the number of subcarriers allocated to user 1, \(N\) m represents the number of subcarriers allocated to user \(m\); \(\Xi\) m and \(\Psi\) m are auxiliary variables, expressed as:

[0107]

[0108] ② Use the bisection method to calculate the maximum allocable power value of user \(m\). The process is as follows:

[0109] a. Initialize the upper bound of the Lagrange multiplier to any value greater than For example, let Initialize the lower bound of the Lagrange multiplier to 0. Determine the value of the iteration stop threshold \(\xi\). For example, let \(\xi = 0.0001\);

[0110] b. Use the following formula to calculate the value of the Lagrange multiplier in the current iteration round:

[0111]

[0112] c. Use the following formula to calculate the power value allocated on the subcarriers of user \(m\):

[0113]

[0114] At the same time, use the Shannon formula to calculate the transmission rate of user \(m\):

[0115]

[0116] d. If R m ≤ Sr, then let If R m > Sr, then let

[0117] e. Repeat steps b to d until the condition

[0118] f. Calculate the maximum power value allocated to user m on the subcarrier using the following formula:

[0119]

[0120] g. Calculate the maximum allocable power of user m using the following formula:

[0121]

[0122] ③ Arrange the users in descending order of their transmission rates such that R 1 ≥ R 2 ≥... ≥ R M ;

[0123] ④ According to the calculated maximum allocable power, correct the preliminary user power allocation scheme : If the initially allocated power of user m is less than the maximum allocable power, i.e., then let If the initially allocated power of user m is greater than or equal to the maximum allocable power, i.e., then let Meanwhile, allocate the excess power beyond to the next user

[0124] ⑤ Repeat step ④ to complete the correction of the initially allocated power for all users and obtain the final inter - user power allocation strategy

[0125] (3.2) Further allocate the power allocated to each user to the subcarriers belonging to that user. The process is as follows:

[0126] ① Solve the following equation to obtain the Lagrange multiplier corresponding to the power P tot，m allocated to user m:

[0127]

[0128] ② Use the water - filling method to calculate the power allocated to each subcarrier:

[0129]

[0130] Determine the set of video tiles to be transmitted to the user according to the network resources allocated in step (3).

[0131] As Figure 3 shown in, comparison method 1 is the equal rate requirement method, that is, the influence of the user's field of view prediction error on the user's rate requirement is not considered; comparison method 2 is the method of maximizing the total system rate, that is, resource allocation is carried out with the goal of maximizing the total transmission rate of the system; comparison method 3 is the random allocation method, in which subcarriers are randomly allocated to all users, and the total power is evenly allocated to each subcarrier.

[0132] From Figure 3 it can be found that the method proposed by the present invention can significantly increase the number of video tiles within the user's field of view (i.e., the number of tiles visible to the user), and is close to the theoretical limit (i.e., Figure 3 as shown by the red dotted line in); and when the total system power is relatively low, the algorithm of the present invention has a greater performance improvement than comparison methods 1-3, which makes the method of the present invention more applicable to networks with limited resources.

[0133] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A panoramic video transmission method that resists field of view prediction errors, characterized in that: The steps include: (1) Calculate the rate weighted ratio according to the field of view prediction error of different users; (2) allocating subcarrier resources in the system to users according to the rate weighted ratio obtained in step (1); (3) Allocating power resources in the system to users according to the rate weighted ratio calculated in step (1) and the subcarrier resources obtained in step (2).

2. The panoramic video transmission method against field of view prediction error according to claim 1, characterized in that: The implementation process of step (1) is as follows: Assume that the total number of users in the system is denoted as M, and the user set is denoted as The user index is represented as m; the field of view prediction error of user m is ρ m , then the user's rate weighted ratio β m The calculation formula is as follows: among themb min =0,β max =1,ρ max =max(ρ1,…,ρ m ,…,r M )。 3. The panoramic video transmission method against field of view prediction error according to claim 2, characterized in that: In step (2), the subcarrier resource allocation steps are as follows: (2.1) The total number of subcarriers in the system is denoted as N, and the subcarrier set is denoted as The subcarrier index is denoted as n; the total number of tiles for panoramic video segmentation is denoted as S, the total number of tiles in the user's field of view is denoted as f, and the coding rate of each tile is denoted as r; the total system bandwidth is denoted as B, and the total transmission power of the base station is denoted as P tot ; Calculate the channel noise ratio {H 1，1 , ..., H m，n , ..., H M，N }, the calculation formula is: Among them, H m,n represents the channel noise ratio of user m on subcarrier n; g m，n represents the channel gain between user m and the base station on subcarrier n; N0 represents the noise power spectral density; (2.2) Initialize the transmission rate R of user m m =0, the subcarrier set allocated to user m The remaining unassigned subcarrier set The remaining set of users participating in subcarrier allocation in represents the empty set; (2.3) For each user m in the set Λ, perform the following steps: ① Find the subcarrier n with the maximum channel-to-noise ratio for user m in the remaining unassigned subcarrier set Γ, H m，n ≥H m，n′ , n′∈Γ; ② Allocate the found subcarrier n to user m, Ω m =Ω m ∪{n}, and update the remaining unassigned subcarrier set, Γ=Γ-{n}; ③ Calculate the transmission rate of user m according to the following formula: where p m，n =P tot / N represents the transmit power on subcarrier n, and N0 is the noise power spectral density; (2.4) Find the user m with the lowest weighted transmission rate in the set Λ, β m R m ≤β m′ R m′ , m′∈Λ; if the user's transmission rate is lower than the encoding rate of the entire panoramic video, R m <Sr, then perform the following steps: ① Find the subcarrier n with the maximum channel-to-noise ratio for user m in the remaining unassigned subcarrier set Γ, H m，n ≥H m，n′ , n′∈Γ; ② Allocate the found subcarrier n to user m, Ω m =Ω m ∪{n}, and update the remaining unassigned subcarrier set, Γ=Γ-{n}; ③ Calculate the transmission rate of user m according to Shannon's formula: where p m，n =P tot / N; If the user's transmission rate is greater than the encoding rate of the entire panoramic video, R m >Sr, then user m is removed from the remaining user set Λ participating in subcarrier allocation, Λ=Λ-{m}; (2.5) Repeat step (2.4) until there are no unassigned subcarriers left.

4. The panoramic video transmission method against field of view prediction error according to claim 3, characterized in that: In step (3), the power resource allocation steps are as follows: (3.1) First, obtain the power allocation plan between users The total power of the base station P tot Assigned to M users; (3.2) The power allocated to each user is further allocated to the subcarriers belonging to the user.

5. The panoramic video transmission method for resisting field of view prediction error according to claim 4, characterized in that: Step (3.1) is specifically: ① Solve the following nonlinear equations to obtain the preliminary power allocation scheme: Where N m represents the number of subcarriers allocated to user m, N1 represents the number of subcarriers allocated to user 1, and N m represents the number of subcarriers allocated to user m; m With m is an auxiliary variable, expressed as: ② Use the binary search method to calculate the maximum allocatable power value of user m; ③ Arrange the users according to their transmission rates from large to small, so that R1≥R2≥...≥R M ; ④ Based on the calculated maximum allocatable power, make a preliminary user power allocation plan Correction: If the initial allocated power of user m is less than the maximum allocable power, that is, Then let If the initial allocated power of user m is greater than or equal to the maximum allocatable power, that is, Then let At the same time, it will exceed Part of the power is allocated to the next user ⑤ Repeat step ④ to complete the correction of the initial power allocation for all users and obtain the final power allocation strategy between users 6. The panoramic video transmission method against field of view prediction error according to claim 5, characterized in that: In step ②, the maximum allocatable power value of user m is calculated using the binary search method. The process is as follows: a. Initialize the upper bound of the Lagrange multiplier to be any value greater than The value of initializes the lower bound of the Langrange multiplier to 0. Determine the value of the iteration stop threshold ξ; b. Use the following formula to calculate the value of the Lagrange multiplier for the current iteration: c. Use the following formula to calculate the power value allocated to user m subcarrier: At the same time, the Shannon formula is used to calculate the transmission rate of user m: d. If R m ≤Sr, then let If R m >Sr, then let e. Repeat steps b to d until the conditions are met f. Use the following formula to calculate the maximum power value allocated to user m subcarrier: g. Use the following formula to calculate the maximum allocatable power for user m:

7. The panoramic video transmission method for resisting field of view prediction error according to claim 6, characterized in that: The process of step (3.2) is: ① Solve the following equation to obtain the power P allocated to user m: tot，m The corresponding Lagrange multipliers are: ② Use the water injection method to calculate the power allocated to each subcarrier:

Images