A holographic video cooperative transmission method based on visible light communication

Abstract

The present application relates to a kind of holographic video cooperation transmission methods based on visible light communication, belong to optical communication and holographic video technical field.The method includes: S1: constructing holographic video cooperation transmission system based on visible light communication;S2: under the constraint condition of satisfying code rate selection, beam forming and channel capacity, construct optimization problem of jointly optimizing code rate selection variable and beam forming variable, maximize the user experience quality of holographic video cooperation transmission service;S3: integer variable, i.e. code rate selection variable is relaxed as continuous variable, so that the problem is converted into the optimization problem after relaxation;S4: for the optimization problem after relaxation, low-complexity continuous solution is obtained using alternating optimization iterative algorithm;S5: based on the continuous solution obtained in step S4, using rounding operation, obtain joint code rate selection and beam forming method for holographic video cooperation transmission system.The present application can improve the user experience quality performance of holographic video cooperation transmission system.

Description

Technical Field

[0001] This invention belongs to the field of optical communication and holographic video technology, and relates to a holographic video collaborative transmission method based on visible light communication. Background Technology

[0002] With the development of wireless communication, augmented reality, and virtual reality technologies, network and content providers now offer users access to three degrees of freedom (3-DoF) video content, often referred to as panoramic video. Using smart devices such as head-mounted displays and smartphones, users can immerse themselves in a virtual environment and interact with the content by changing their viewing orientation (e.g., up, down, left, right). However, panoramic video has a limitation: users cannot move freely within the virtual environment, reducing the overall immersion. In contrast, holographic video offers a more immersive six-DoF viewing experience, allowing users to physically change their position and perspective. However, the high degree of freedom in holographic video generates a large amount of data that needs to be transmitted. To address this issue, one approach is to increase channel capacity, and another is to reduce data rate requirements.

[0003] To improve channel capacity, several advanced communication technologies can be combined, such as visible light communication, cooperative multipoint (CoMP), and beamforming. Visible light communication, due to its abundant and license-free spectral resources, is widely recognized as a complementary technology to radio frequency networks and is suitable for high data rate applications such as video streaming. CoMP and beamforming improve channel capacity through different methods: the former coordinates data transmission from multiple visible light communication base stations, while the latter directs signals to users and effectively utilizes communication resources. To reduce the data rate requirements of video streaming, some studies have proposed segmenting the video into blocks, transmitting only the blocks of interest to the user, and optimizing the bitrate allocation of each block or effectively utilizing communication resources.

[0004] Existing research includes integrating CoMP and beamforming to improve channel capacity, studying beamforming to enhance the user experience quality of traditional video streams, researching joint rate selection and resource allocation to improve the user experience quality of traditional video streams, and studying resource allocation and joint rate selection and resource allocation for panoramic video streams. However, the results obtained from these studies are unrelated to any video feature (such as saliency). Previous work has further investigated the impact of panoramic video saliency on user experience quality, and some studies have extended this to holographic video scenarios, proposing saliency-based rate selection schemes to maximize user experience quality. However, most existing work is limited to resource-constrained radio frequency-based wireless networks, which have significant room for improvement in user experience quality performance. Therefore, there is an urgent need for a holographic video cooperative transmission system and method based on visible light communication to improve user experience quality performance. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a holographic video collaborative transmission method based on visible light communication, which maximizes the user experience quality of holographic video under constraints of bit rate selection, beamforming and channel capacity.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A holographic video collaborative transmission method based on visible light communication specifically includes the following steps:

[0008] S1: Construct a holographic video collaborative transmission system based on visible light communication, including a holographic video storage server, multiple visible light communication base stations and multiple mobile users, and all visible light communication base stations can use collaborative multipoint transmission technology and beamforming technology to provide video transmission services to all users simultaneously.

[0009] S2: Under the constraints of bit rate selection, beamforming and channel capacity, construct an optimization problem P1 that jointly optimizes the bit rate selection variable and the beamforming variable. The goal of this optimization problem is to maximize the user experience quality of the holographic video collaborative transmission service.

[0010] S3: Relax the integer variable (i.e. the bitrate selection variable) into a continuous variable, thereby transforming the optimization problem P1 into a relaxed optimization problem P2;

[0011] S4: For the relaxed optimization problem P2, an iterative algorithm based on alternating optimization technique is used to transform the optimization problem P2 into two sub-problems P3 and P4 to optimize the code rate selection variable and beamforming variable, thereby obtaining a low-complexity continuous solution;

[0012] S5: Based on the continuous solution obtained in step S4, the floor operation is used to obtain the joint bit rate selection and beamforming method for holographic video cooperative transmission system.

[0013] Furthermore, step S1 specifically includes: constructing a holographic video collaborative transmission system based on visible light communication. This system includes a holographic video storage server, N visible light communication base stations, and K mobile users, which respectively use... and This means that all visible light communication base stations can use cooperative multipoint transmission technology and beamforming technology to provide video transmission services to all users simultaneously; holographic video storage servers depict holographic videos of 3D objects in the room and connect to the visible light communication base stations via high-speed, low-latency backhaul links (such as fiber optics); users wear wireless head-mounted display devices and transmit holographic videos from the storage server through the visible light communication base stations.

[0014] Furthermore, in step S2, the mathematical model for the optimization problem P1, which involves jointly optimizing the code rate selection variable and the beamforming variable, is as follows:

[0015]

[0016]

[0017]

[0018]

[0019]

[0020] Wherein, constraint C1 indicates the bitrate selection for 3D blocks transmitted within the user's field of view, constraint C2 indicates that 3D blocks outside the user's field of view are not transmitted, C3 represents the power constraint during beamforming, and C4 represents the channel capacity constraint; Q(b) represents the user experience quality utility function of holographic video, b represents the bitrate selection variable, and w represents the beamforming variable; b t,k This represents the bitrate selection for the t-th 3D block that user k is interested in. Represents the set of video bitrates, μ l (Unit: bps) represents the bitrate of a 3D block of quality level l, where μ1 ≤ μ2 ≤ … ≤ μ L ,gather Indicates the quality level; This represents a set of 3D blocks, where T indicates that the holographic video is divided into T cubes, i.e., 3D blocks; during transmission, only the 3D blocks within each user's field of view are transmitted. Represents the set of 3D blocks within user k's field of view; P represents the transmission beamformer for the k-th user at the n-th visible light communication base station. n C represents the maximum transmission power (in Watts) of the nth visible light communication base station; k (w) represents the channel capacity of the k-th user.

[0021] Furthermore, in step S2, the expression for the user experience quality utility function Q(b) of the holographic video is:

[0022]

[0023] Where, δ t,k ∈[0,1] is the saliency score of the t-th 3D block, and it holds true for all users. D t,k Let α represent the distance between the k-th user and the t-th 3D block, where α, β, and γ are positive constants.

[0024] Furthermore, in step S3, the mathematical model for the relaxed optimization problem P2 is:

[0025]

[0026] Furthermore, in step S4, the iterative algorithm based on alternating optimization technology is as follows: the rate selection variable b and the beamforming variable w are alternately optimized in each iteration, with i representing the iteration index, i = 0, 1, 2, ... In the i-th iteration, w and b are optimized by solving two subproblems P3 and P4. The mathematical models of subproblems P3 and P4 are as follows:

[0027]

[0028] stC3

[0029]

[0030]

[0031] stC2,C5

[0032]

[0033] Use (w) (i+1) ,b (i+1) Let w represent the solution of the i-th iteration. Since P3 and P4 are both convex feasibility problems, the interior point method can be used to obtain the optimal solutions to these two subproblems. Using a low-complexity iterative algorithm to compute problem P2 can yield a low-complexity continuous solution to the relaxation problem, since w... (i+1) and b (i+1) The optimality of the sequence in problem P2 It will converge to a rest point, using (w) * ,b * ) is used to represent this.

[0034] Furthermore, in step S5, for the stationary point (w) * ,b * A suitable rounding operation is adopted, specifically: for each user, a feasible bitrate value is set according to whether the 3D block is in the field of view, and then the bitrate sequence is sorted in ascending order to form an ordered sequence. Finally, the feasible bitrate is updated according to the distance between the sorted bitrate and the candidate bitrate in the bitrate selection set, and finally a joint bitrate selection and beamforming method for holographic video collaborative transmission system is obtained.

[0035] The beneficial effects of this invention are as follows: Considering the 3D blocks within the user's field of view, the maximum power of beamforming, and the saliency of the video, this invention can better improve channel capacity and video transmission quality compared to existing research. It performs joint optimization design of bit rate selection parameters and beamforming parameters to maximize the user experience quality of holographic video. In addition, the proposed solution of this invention, namely the holographic video cooperative transmission system based on visible light communication and the low-complexity joint bit rate selection and beamforming method for this system, is superior to other comparative solutions under the given conditions.

[0036] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0038] Figure 1 This is a schematic diagram of the holographic video collaborative transmission system based on visible light communication according to the present invention.

[0039] Figure 2 This is a flowchart of the holographic video collaborative transmission method based on visible light communication according to the present invention;

[0040] Figure 3 This is a diagram showing the relationship between user experience quality and bandwidth in holographic video in this invention;

[0041] Figure 4 This is a graph showing the relationship between the user experience quality and transmission power of holographic video in this invention;

[0042] Figure 5 This is a graph showing the relationship between the user experience quality of holographic video and the number of visible light communication base stations in this invention.

[0043] Figure 6 This is a graph showing the relationship between the user experience quality of holographic video and the number of 3D blocks in this invention. Detailed Implementation

[0044] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0045] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0046] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0047] Please see Figures 1-6 This invention provides a holographic video collaborative transmission method based on visible light communication, specifically including the following steps:

[0048] S1: Establish a holographic video collaborative transmission system based on visible light communication. This system includes one holographic video storage server, N visible light communication base stations, and K mobile users, each using... and This means that all base stations can simultaneously provide video transmission services to all users using cooperative multipoint transmission technology and beamforming technology. A holographic video storage server depicts holographic videos of 3D objects in a room and connects to a visible light communication base station via a high-speed, low-latency backhaul link (such as fiber optic). Users wear wireless head-mounted display devices, transmitting holographic video from the storage server through the visible light communication base station.

[0049] S2: To improve the user experience quality of holographic video transmission services, we establish an optimization problem P1 that jointly optimizes the bitrate selection variable and the beamforming variable. The goal of this optimization problem is to maximize the user experience quality of holographic video collaborative transmission services. The optimization variables are the bitrate selection variable and the beamforming variable, and the optimization constraints are the bitrate selection constraint, the beamforming constraint, and the channel capacity constraint.

[0050] The specific steps for establishing the optimization problem P1, which involves jointly optimizing the code rate selection variable and the beamforming variable, are as follows:

[0051] S21: Select the video bitrate. The specific process is as follows:

[0052] The holographic video is divided into T cubes, each of which is called a 3D block, and a set is used. To represent; during transmission, only 3D blocks within each user's field of view are transmitted, using Represents the set of 3D blocks within user k's field of view;

[0053] The following constraints must be met when selecting video bitrate:

[0054]

[0055]

[0056] Among them, b t,k This represents the bitrate selection for the t-th 3D block that user k is interested in. Represents the set of video bitrates, μ l (Unit: bps) represents the bitrate of a 3D block of quality level l, where μ1 ≤ μ2 ≤ … ≤ μ L ,gather Indicates the quality level.

[0057] S22: To improve channel capacity, this invention proposes integrating cooperative multipoint technology and beamforming technology into the visible light communication system, allowing all visible light communication base stations to collaboratively transmit holographic video to all users. Beamforming needs to meet the following constraints:

[0058]

[0059] in, P represents the transmission beamformer for the k-th user at the n-th visible light communication base station. n This represents the maximum transmission power (in Watts) of the nth visible light communication base station.

[0060] S23: The signal-to-interference-plus-noise ratio (SINR) at the k-th user is expressed as follows:

[0061]

[0062] Among them, h n,k N0 represents the DC gain of the visible light communication channel from the nth visible light communication base station to the kth user, N0 (in dBm / Hz) is the noise power spectral density, and B represents the bandwidth (in MHz).

[0063] Based on the SINR of the k-th user, the channel capacity of the k-th user can be obtained as follows:

[0064]

[0065] To ensure that user k can successfully receive the 3D blocks within its field of view, we have:

[0066]

[0067] S24: Considering the logarithmic relationship between user experience quality and video bitrate, and the inverse relationship between user and spatial distance between the user and the holographic video, as well as the reason for video saliency, the user experience quality utility function of holographic video is defined as:

[0068]

[0069] Where, δ t,k ∈[0,1] is the saliency score of the t-th 3D block, and it holds true for all users. D t,k Let α represent the distance between the k-th user and the t-th 3D block, where α, β, and γ are positive constants.

[0070] S25: Under the constraints of bit rate selection, beamforming, and channel capacity, establish a mathematical model for the joint optimization of bit rate selection and beamforming, with the goal of maximizing the user experience quality of holographic video transmission services. P1:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] Among them, constraint C1 indicates the code rate selection of 3D blocks transmitted within the user's field of view, constraint C2 indicates that 3D blocks not within the user's field of view are not transmitted, C3 indicates the power constraint during beamforming, and C4 indicates the channel capacity constraint.

[0077] S3: Relax the integer variable (i.e., the bitrate selection variable b) into a continuous variable to obtain the relaxed user experience quality maximization problem P2.

[0078] In step S3, the relaxed integer variable (i.e., the bitrate selection variable b) is a continuous variable, so we have:

[0079]

[0080] This leads to the relaxed user experience quality maximization optimization problem, P2, whose mathematical model is:

[0081]

[0082] stC2~C4,

[0083]

[0084] S4: For the relaxed maximization problem P2, construct an iterative algorithm based on alternating optimization techniques to obtain a low-complexity continuous solution to the relaxed problem; specifically including the following steps:

[0085] Problem P2 can be solved by constructing an iterative algorithm based on alternating optimization techniques. The rate selection variable b and the beamforming variable w will be optimized alternately in each iteration, with i representing the iteration index (i = 0, 1, 2, ...). In the i-th iteration, w and b are optimized by solving two subproblems P3 and P4, as follows:

[0086]

[0087] stC3

[0088]

[0089]

[0090] stC2,C5

[0091]

[0092] Use (w) (i+1) ,b (i+1) Let w represent the solution of the i-th iteration. Since P3 and P4 are both convex feasibility problems, the interior-point method can be used to obtain the optimal solutions to these two subproblems. Using a low-complexity iterative algorithm to compute problem P2 can yield a low-complexity continuous solution to the relaxation problem, since w... (i+1) and b (i+1) The optimality of the sequence in problem P2 It will converge to a rest point, using (w) *,b * ) is used to represent this.

[0093] S5: Based on the continuous solution obtained in step S4, a suitable rounding operation is constructed to obtain a joint rate selection and beamforming method for holographic video cooperative transmission systems; specifically, it includes the following steps:

[0094] The iterative algorithm in problem P1:S4, proposed in this invention, can obtain the stationary point (w) of problem P2. * ,b * ), b * The continuity of the 3D block may not be a feasible solution to problem P1. Therefore, it is necessary to construct an appropriate rounding operation for it: for each user, the feasible bitrate value is set according to whether the 3D block is in the field of view. Then, the bitrate sequence is sorted in ascending order to form an ordered sequence. Finally, the feasible bitrate is updated according to the distance between the sorted bitrate and the candidate bitrate in the bitrate selection set. Finally, a joint bitrate selection and beamforming method for holographic video cooperative transmission system is obtained.

[0095] Comparative experiment:

[0096] like Figures 3-6 As shown, this experiment demonstrates the relationship between the user experience quality of holographic video and bandwidth, transmission power, the number of visible light communication base stations, and the number of 3D blocks. Furthermore, each figure presents the results of the proposed solution and three comparative solutions. These three comparative solutions are specifically: for each user k, The quality level of each 3D block is increased one level cyclically. In comparison scheme 1, it starts from any 3D block; in comparison scheme 2, it starts from the 3D block closest to user k; and in comparison scheme 3, it starts from the most prominent 3D block, until the channel capacity constraint is no longer satisfied, at which point the loop stops. As can be seen from the four figures, the performance of the proposed scheme is close to optimal and superior to the proposed comparison schemes, demonstrating their advantage in providing a high user experience quality. This is because the algorithm can collectively select an appropriate video bitrate and effectively utilize limited communication resources. The user experience quality of holographic video increases with increasing bandwidth, transmission power, and the number of visible light communication base stations, because the available communication resources increase with these increases. Furthermore, it can be seen that the user experience quality of holographic video decreases with increasing number of 3D blocks. This is because when the holographic video is divided into more 3D blocks, but the number of 3D blocks within the user's field of vision remains unchanged, it becomes more difficult for the user to view the holographic video, leading to a decrease in the user experience quality.

[0097] In summary, the holographic video cooperative transmission method based on visible light communication proposed in this invention, considering the 3D blocks within the user's field of view, the maximum power of beamforming, and the saliency of the video, can better improve channel capacity and video transmission quality compared to existing research. The method also achieves maximum user experience quality for holographic video through joint optimization design of bitrate selection parameters and beamforming parameters. Furthermore, the proposed scheme, namely the holographic video cooperative transmission system based on visible light communication and the low-complexity joint bitrate selection and beamforming method for this system, outperforms other considered comparative schemes under the given conditions.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A holographic video collaborative transmission method based on visible light communication, characterized in that, The method specifically includes the following steps: S1: Construct a holographic video collaborative transmission system based on visible light communication, including a holographic video storage server, N One visible light communication base station and K Each mobile user, using and This means that all visible light communication base stations use cooperative multipoint transmission technology and beamforming technology to provide video transmission services to all users simultaneously; S2: Under the constraints of bit rate selection, beamforming, and channel capacity, construct an optimization problem P1 that jointly optimizes the bit rate selection and beamforming variables. The goal of this optimization problem is to maximize the user experience quality of the holographic video collaborative transmission service. The mathematical model of the constructed optimization problem P1 that jointly optimizes the bit rate selection and beamforming variables is as follows: Among them, constraint C1 indicates the bit rate selection of 3D blocks transmitted within the user's field of view, constraint C2 indicates that 3D blocks not within the user's field of view are not transmitted, C3 indicates the power constraint during beamforming, and C4 indicates the channel capacity constraint. The utility function representing the user experience quality of holographic videos. This indicates the bitrate selection variable. Indicates beamforming variables; Indicates user The first thing I care about Bitrate selection for each 3D block Represents the set of video bitrates. Indicates quality grade as The bitrate of the 3D blocks, and ,gather Indicates the quality level; Represents a set of 3D blocks. T This indicates that the holographic video is segmented into Each cube is a 3D block; during transmission, only the 3D blocks within each user's field of view are transmitted. Indicates user A collection of 3D blocks within the field of view; Indicates the first The user in the first Transmission beamformer for a visible light communication base station Indicates the first The maximum transmission power of a visible light communication base station; Indicates the first Individual user channel capacity; User experience quality utility function of holographic video The expression is: in, It is the first The saliency score of each 3D block, and it is available for all users. , Indicates the first The user and the The distance between 3D blocks , and It is a positive number; S3: Relax the integer variable, i.e., the bitrate selection variable, into a continuous variable, thereby transforming the optimization problem P1 into a relaxed optimization problem P2. The mathematical model is as follows: S4: For the relaxed optimization problem P2, an iterative algorithm based on alternating optimization technique is used to transform the optimization problem P2 into two sub-problems P3 and P4 to optimize the code rate selection variable and beamforming variable, thereby obtaining a low-complexity continuous solution; The iterative algorithm based on alternating optimization techniques specifically involves: selecting the bitrate variable... and beamforming variables Alternate optimization in each iteration, using To represent the iterative index, In the In this iteration, optimization is achieved by solving two subproblems, P3 and P4. and The mathematical models for subproblems P3 and P4 are as follows: use To indicate the first In the next iteration, since both P3 and P4 are convex feasibility problems, the interior-point method is used to obtain the optimal solutions to these two subproblems; a low-complexity iterative algorithm is used to compute problem P2 to obtain a low-complexity continuous solution to the relaxation problem. and The optimality of the sequence in problem P2 It will converge to a stationary point, using To indicate; S5: Based on the continuous solution obtained in step S4, the floor operation is used to obtain the joint bit rate selection and beamforming method for holographic video cooperative transmission system.

2. The holographic video collaborative transmission method according to claim 1, characterized in that, In step S1, the holographic video storage server depicts holographic videos of 3D objects in the room and connects to a visible light communication base station via a high-speed, low-latency backhaul link; the user wears a wireless head-mounted display device and transmits the holographic video from the storage server through the visible light communication base station.

3. The holographic video collaborative transmission method according to claim 1, characterized in that, In step S5, for the stationary point The method employs a rounding operation, specifically: for each user, a feasible bitrate value is set based on whether the 3D block is within the field of view. Then, the bitrate sequence is sorted in ascending order to form an ordered sequence. Finally, the feasible bitrate is updated based on the distance between the sorted bitrate and the candidate bitrates in the bitrate selection set, ultimately resulting in a joint bitrate selection and beamforming method for holographic video collaborative transmission systems.

Images