Multi-user VR video streaming media rate adaptive adjustment method and system

Abstract

The invention provides a multi-user VR video streaming media rate-based adaptive adjustment method and system, and relates to the technical field of VR, comprising the following steps: constructing a double-layer architecture of an edge layer and a user layer, carrying out video space-time segmentation and multi-quality coding on the edge layer, and realizing video adaptive transmission based on user motion information; a forward and reverse weighted user experience evaluation model is adopted to carry out optimal allocation on system resources; a quality balanced user subset is constructed in combination with time dimension equilibrium constraint, and resource dynamic optimization is carried out; the multi-dimensional resource verification model is used to realize video quality grade optimization distribution; and generating a panoramic frame through integrity verification, time sequence synchronization and spherical projection reconstruction. According to the invention, the service quality balance of multi-user VR video transmission is improved, the system resource overhead is reduced, and the user watching experience is enhanced.

Description

Technical Field

[0001] The present invention relates to VR technology, and in particular to a method and system for adaptively adjusting the rate of multi-user VR video streaming media. Background Art

[0002] With the rapid development of virtual reality technology, multi-user virtual reality video streaming applications have gradually become a research hotspot. Existing VR video streaming technologies mainly focus on video encoding, transmission and decoding, aiming to improve the user's viewing experience. However, with the increase in the number of users and the complexity of the network environment, traditional video streaming technologies face many challenges.

[0003] Existing video streaming technologies cannot effectively adapt to changes in user perspectives in multi-user scenarios, resulting in low efficiency of video transmission and affected user experience.

[0004] The existing technology lacks comprehensiveness in user experience evaluation and fails to fully consider the user's multi-dimensional playback status information and bandwidth resource costs, resulting in unreasonable resource allocation and affecting the overall performance of the system.

[0005] The existing resource allocation strategy fails to effectively handle the video quality differences between user terminals and lacks a dynamic optimization mechanism, resulting in uneven service quality in a multi-user environment and failure to meet the personalized needs of users. Summary of the invention

[0006] The embodiments of the present invention provide a method and system for adaptively adjusting the rate of multi-user VR video streaming media, which can solve the problems in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a method for adaptively adjusting a rate of a multi-user VR video streaming media, including: Construct a two-layer architecture of edge layer and user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the edge layer to obtain video slice data, collect translation and rotation information of user terminals in the user layer, and selectively transmit video slice data based on translation and rotation information to achieve adaptive video transmission based on user perspective; Collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; Based on the user experience evaluation results, the video quality differences between user terminals are evaluated. The video quality differences are constrained by combining the time dimension balance constraints and the minimum quality level. The user terminals that meet the constraints are constructed as quality-balanced user subsets. Under the conditions of meeting multiple constraints, the resource allocation of the quality-balanced user subsets is dynamically optimized to achieve service quality balance in multi-user scenarios. Obtain the proposed quality level of each user terminal in the quality-balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize the video quality level of the user terminal based on the verification result; The video slice data transmitted after dynamic optimization allocation is subjected to integrity check and timing synchronization processing, and the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources. The decoded video slice data is input into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

[0008] Construct a two-layer architecture of edge layer and user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the edge layer to obtain video slice data, collect translation and rotation information of user terminals in the user layer, and selectively transmit video slice data based on translation and rotation information to achieve adaptive video transmission based on user perspective, including: The edge server is connected to the wireless network access point via an Ethernet cable. The edge server and the wireless network access point form an edge layer. The edge layer is connected to the user layer via wireless communication. The user layer includes multiple user terminals. The edge server performs time segmentation processing on the stored video, and divides the stored video into multiple video segments according to a preset duration; the edge server performs space segmentation processing on each frame of the video segment, and divides each frame into multiple rectangular tiles to obtain a first tile, a second tile, a third tile, and a fourth tile; The edge server uses multiple preset quality levels to encode the first tile, the second tile, the third tile, and the fourth tile respectively, and the multiple preset quality levels correspond to multiple different CRF values ​​of constant rate factors, and the CRF values ​​are set in ascending order from low to high; Each of the multiple user terminals collects motion information of the user, the motion information including translation information and rotation information, wherein the translation information includes the translation position along the X-axis, the translation position along the Y-axis and the translation position along the Z-axis, and the rotation information includes the pitch angle, the yaw angle and the roll angle; the user terminal sends the motion information to the edge layer according to a preset sampling period; The edge server determines the current viewing angle of the user terminal according to the received motion information, selects a target tile of a corresponding quality level from tiles encoded at multiple preset quality levels according to the current viewing angle, and sends the target tile to the corresponding user terminal through a wireless network access point; The user terminal plays the video according to the received target tile, and continuously collects and sends motion information to the edge layer during the video playback process. The edge server updates the selection of the target tile in real time according to the continuously received motion information, realizing adaptive video transmission based on the motion information from the user's perspective.

[0009] Collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize the system resource utilization efficiency based on the user experience evaluation result, including: Collecting multi-dimensional video playback status information of a user terminal, the multi-dimensional video playback status information including video playback indication information and bandwidth resource cost information, and dividing the video playback indication information into a playback quality parameter and a playback stability parameter; Normalizing the playback quality parameter and the playback stability parameter to obtain a normalized quality parameter and a normalized stability parameter, respectively, calculating a positive eigenvalue based on the normalized quality parameter and the normalized stability parameter, and calculating a negative eigenvalue based on bandwidth resource cost information; Multiply the positive eigenvalue by the first weighting coefficient to obtain a positive experience evaluation value, and multiply the negative eigenvalue by the second weighting coefficient to obtain a negative experience evaluation value, wherein the sum of the first weighting coefficient and the second weighting coefficient is 1, and input the positive experience evaluation value and the negative experience evaluation value into the positive and negative weighted user experience evaluation model to obtain a comprehensive user experience evaluation value; A resource allocation optimization objective function is constructed based on the comprehensive evaluation value of user experience to calculate the current resource utilization efficiency of the system. When the resource utilization efficiency is less than the preset efficiency threshold, a resource optimization allocation plan is generated using the resource allocation optimization objective function. The resource optimization allocation plan includes bandwidth allocation parameters and cache allocation parameters. The system resource utilization efficiency of the user terminal is dynamically optimized and allocated according to the bandwidth allocation parameters and the cache allocation parameters.

[0010] Based on the user experience evaluation results, the video quality differences between user terminals are evaluated. The video quality differences are constrained by combining the time dimension balance constraints and the minimum quality level. The user terminals that meet the constraints are constructed as quality-balanced user subsets. Under the conditions of meeting multiple constraints, the resource allocation of the quality-balanced user subsets is dynamically optimized to achieve service quality balance in multi-user scenarios, including: Calculating a video quality difference of multiple user terminals, where the video quality difference is determined by a deviation between a current video quality level of each user terminal and an average video quality level of all user terminals; Establishing a quality balancing constraint condition in the time dimension, the quality balancing constraint condition includes that the difference in video quality levels between any two user terminals within a preset video duration does not exceed a preset quality difference threshold; Obtaining a minimum video quality level among multiple user terminals, and constructing a quality-balanced user subset according to the minimum video quality level, wherein a difference between a user terminal in the quality-balanced user subset and the minimum video quality level does not exceed a preset maximum quality difference value; Obtaining bandwidth resource utilization efficiency of each user terminal in the quality-balanced user subset, and optimizing resource allocation for the user terminals in the quality-balanced user subset according to the bandwidth resource utilization efficiency; Determine whether the resource allocation optimization satisfies bandwidth resource constraints, quality balancing constraints, and minimum quality assurance constraints, where the bandwidth resource constraint is that the sum of bandwidth resources of all user terminals does not exceed the upper limit of system bandwidth resources, the quality balancing constraint is that the difference in video quality levels of any two user terminals in the quality balancing user subset does not exceed a preset maximum quality difference value, and the minimum quality assurance constraint is that the video quality level of all user terminals is not lower than the minimum video quality level; When bandwidth resource constraints, quality balance constraints, and minimum quality assurance constraints are met, the video quality level and resource allocation status of the user terminal are updated; The video quality difference is recalculated according to the updated video quality level and resource allocation status, and the resource allocation strategy is adjusted based on the video quality difference to achieve dynamic balance of service quality in multi-user scenarios.

[0011] Obtain the proposed quality level of each user terminal in the quality-balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification, and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize the video quality level of the user terminal based on the verification result, including: Obtaining a proposed quality level of the user terminal, where the proposed quality level is determined based on the current video playback quality level of the user terminal, and determining whether the proposed quality level exceeds a preset quality threshold. When the proposed quality level exceeds the preset quality threshold, the user terminal is removed from the user set; The bandwidth resource demand of the user terminal is calculated based on the proposed quality level. The bandwidth resource demand is calculated based on the bandwidth demand of the user terminal, the transmission delay requirement and the proposed quality level. It is determined whether the bandwidth resource demand exceeds the individual throughput upper limit of the user terminal. When the bandwidth resource demand exceeds the individual throughput upper limit, the user terminal is removed from the user set. Calculate the total resource requirements of all user terminals in the user set at the proposed quality level, determine whether the total resource requirements exceed the total available throughput of the edge server, and remove the user terminal from the user set when the total resource requirements exceed the total available throughput; Construct a multi-dimensional resource verification model, which includes quality threshold verification, individual resource verification, and system resource verification. The resource availability of the user terminal is determined based on the multi-dimensional resource verification model. The video quality level of the user terminal is updated according to the resource availability. When the resource availability meets the preset conditions, the video quality level of the user terminal is increased by a preset level based on the current video quality level. The acquisition of the proposed quality level, the calculation of the bandwidth resource requirement, the calculation of the overall resource requirement, the determination of resource availability, and the update of the video quality level are executed cyclically until all user terminals in the user set complete resource allocation.

[0012] The video slice data transmitted after dynamic optimization allocation is subjected to integrity check and timing synchronization processing, the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources, and the decoded video slice data is input into the spherical projection model to realize the stereoscopic reconstruction of the panoramic frame, including: Receiving panoramic video slice data, performing integrity check on the panoramic video slice data, obtaining a check result by calculating a hash value of the panoramic video slice data, and determining the validity of the panoramic video slice data based on the check result; Performing time synchronization processing on the panoramic video slice data, determining the alignment time of the panoramic video slice data according to the reference timestamp and the time offset of the panoramic video slice data, and sorting the panoramic video slice data according to the alignment time; Obtain processing capability parameters of the decoding device, the processing capability parameters include the processing capability of the central processing unit and the processing capability of the graphics processing unit, determine the resource allocation weight of the decoding task according to the processing capability parameters, and call the hardware decoder to decode the panoramic video slice data based on the resource allocation weight; A spherical projection model is constructed to map the decoded panoramic video fragment data from plane coordinates to spherical coordinates, the pixel fusion weights are calculated based on the spherical coordinates, and the panoramic video fragment data are reconstructed according to the pixel fusion weights to obtain a panoramic frame.

[0013] A second aspect of an embodiment of the present invention provides a system for adaptively adjusting a rate of a multi-user VR video streaming media, including: The first unit is used to construct a two-layer architecture of the edge layer and the user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the video at the edge layer to obtain video slice data, collect translational motion information and rotational motion information of the user terminal in the user layer, and selectively transmit the video slice data based on the translational motion information and the rotational motion information to achieve adaptive video transmission based on the user perspective; The second unit is used to collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; The third unit is used to evaluate the video quality difference between user terminals based on the user experience evaluation result, constrain the video quality difference by combining the balance constraint in the time dimension and the minimum quality level, construct the user terminals that meet the constraint conditions into a quality-balanced user subset, dynamically optimize the resource allocation of the quality-balanced user subset under the condition of meeting multiple constraints, and realize service quality balance in a multi-user scenario; The fourth unit is used to obtain the proposed quality level of each user terminal in the quality balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize and allocate the video quality level of the user terminal based on the verification result; The fifth unit is used to perform integrity check and timing synchronization processing on the video slice data transmitted after dynamic optimization allocation, decode the video slice data in combination with the intelligent scheduling mechanism of hardware decoding resources, and input the decoded video slice data into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

[0014] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] The beneficial effects of this application are as follows: By constructing a double-layer architecture of edge layer and user layer, the spatiotemporal segmentation and multi-quality encoding processing of the video are realized, and selective transmission can be performed based on the user's translation and rotation motion information, thereby improving the adaptability of video transmission and user experience.

[0017] Establishing a positive and negative weighted user experience evaluation model can dynamically optimize the allocation of system resources, improve resource utilization efficiency, and ensure that users can get a good viewing experience under different network conditions.

[0018] By evaluating and constraining the differences in video quality between user terminals, a quality-balanced user subset is formed, which achieves service quality balance in multi-user scenarios and ensures that all users can enjoy a relatively consistent quality experience when watching videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is an architecture diagram of a VR 360° video streaming system based on edge computing according to an embodiment of the present invention; Figure 2 The present invention is a structural diagram of a system for adaptively adjusting the rate of multi-user VR video streaming media based on an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0022] like Figure 1-Figure 2 As shown, the method of the embodiment of the present invention includes: Step 1: Figure 1 The present invention shows a VR 360° video streaming system architecture diagram based on edge computing. In the embodiment of the present application, the architecture consists of an edge layer and a user layer, which are seamlessly connected through wireless communication. The edge layer includes a wireless network access point and an edge server, and the user layer covers 6 users. ={1,2,3,4,5,6}, where 3 users use smartphones and the other 3 users use head-mounted displays.

[0023] Step 1.1: The wireless network access point is connected to the edge server via an Ethernet cable to ensure high-speed and stable wired communication.

[0024] Step 1.2: All videos cached in the edge server are segmented into frames, each of which has a duration of 5 minutes. Figure 2 As shown in the "Video Processing" function section in the video, each frame is further divided into four rectangular tiles, namely, tile 1, tile 2, tile 3 and tile 4. These tiles are encoded using FFmpeg with 10 quality levels. : ; Among them, 10 tiles of different quality levels are encoded, and the CRF values ​​used are {15, 17, 19, 21, 23, 27, 32, 37, 41, 45}.

[0025] Step 1.3: Each user’s device periodically sends its 6-DOF motion trajectory data to the edge layer, including translation (movement along the X, Y, and Z axes) and rotation (pitch, yaw, and roll).

[0026] Step 2: In the embodiment of the present application, a greedy algorithm is used to establish a QoE user feedback mechanism and calculate the user resource utilization efficiency.

[0027] Step 2.1: Collect users The video quality of the current time slot , Transmission time , decoding time , quality variance Real-time feedback from the user is used to calculate the QoE. The calculation method is as follows: ; in, Range .

[0028] Step 2.2: Calculate the QoE benefits of all users U as follows: ; in, , Used to indicate the current user Whether to play the video with quality level q normally.

[0029] Step 2.3: Calculate the efficiency of resource utilization as follows: ; in, Current playback quality level The network bandwidth resource cost.

[0030] Step 3: In the embodiment of the present application, in order to reduce the quality difference between users, Figure 2 As shown in the "Fairness" module in , a fairness metric is introduced. When allocating additional bandwidth, priority is given to users with lower quality levels to ensure that they will not be consumed by high-quality users before they reach an acceptable quality level.

[0031] Step 3.1: Prioritize more users to reach 5, rather than letting a few users enjoy the highest quality 10, to avoid bandwidth bottlenecks caused by upgrading too many users at the same time. Define fairness constraints: ; in, Indicates user In time slot The quality level, Indicates the length of the video, which is 5 minutes.

[0032] Step 3.2: Determine the minimum quality level among all current users: ; Step 3.3: Build a user subset , which contains users whose quality levels differ by no more than 5: ; Step 3.4: Ensure that while promoting low-quality users, the most efficient nodes are also selected to maintain the overall efficiency and fairness of the system, so from the subset Select a user , the user can maximize efficiency : ; Step 4: For the embodiment of the present application, use the bandwidth evaluation and sharing mechanism to perform quality verification to ensure that the service quality of each user does not exceed the carrying capacity of the network, and select the corresponding quality level To transmit, such as Figure 2 As shown in the Bandwidth Assessment module in .

[0033] Step 4.1: Check the proposed quality level Whether the maximum permissible mass is 10. If the condition is met , it means that the user has reached the highest quality threshold and cannot be further improved. From user collection Removed.

[0034] Step 4.2: Evaluate individual resource requirements. Check the quality level of the proposal. + 1, resource function Whether it exceeds the individual throughput of the user If the condition is met , it means that the user's request quality cannot be supported due to insufficient resources. Also from the user collection Removed.

[0035] Step 4.3: Evaluate total resource requirements. Further check that all users have the same quality level as proposed. The cumulative resource function under Whether the total available throughput of the edge server is exceeded This constraint ensures that the overall resource allocation does not exceed the capacity of the server. If the condition , it means that the overall resources are insufficient and the user Also from the user collection Removed.

[0036] Step 4.4: Update the user set. If the proposed quality level can be accommodated, the user Quality level Increase by 1.

[0037] Step 4.5: Return the target quality level that satisfies all conditions.

[0038] Step 5: Figure 2 As shown in the "User Layer" module in FIG, after receiving the target video segment, the user device uses Media Codec to decode it and reassemble it into a panoramic frame, which is presented to the user through the device's display screen.

[0039] Construct a two-layer architecture of edge layer and user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the edge layer to obtain video slice data, collect translation and rotation information of user terminals in the user layer, and selectively transmit video slice data based on translation and rotation information to achieve adaptive video transmission based on user perspective; Collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; Based on the user experience evaluation results, the video quality differences between user terminals are evaluated. The video quality differences are constrained by combining the time dimension balance constraints and the minimum quality level. The user terminals that meet the constraints are constructed as quality-balanced user subsets. Under the conditions of meeting multiple constraints, the resource allocation of the quality-balanced user subsets is dynamically optimized to achieve service quality balance in multi-user scenarios. Obtain the proposed quality level of each user terminal in the quality-balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize the video quality level of the user terminal based on the verification result; The video slice data transmitted after dynamic optimization allocation is subjected to integrity check and timing synchronization processing, and the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources. The decoded video slice data is input into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

[0040] In an optional implementation, a two-layer architecture of an edge layer and a user layer is constructed, and video is temporally and spatially segmented and multi-quality encoded at the edge layer to obtain video slice data, and translational motion information and rotational motion information of a user terminal in the user layer are collected, and the video slice data is selectively transmitted based on the translational motion information and the rotational motion information, so as to realize adaptive video transmission based on user perspective, including: The edge server is connected to the wireless network access point via an Ethernet cable. The edge server and the wireless network access point form an edge layer. The edge layer is connected to the user layer via wireless communication. The user layer includes multiple user terminals. The edge server performs time segmentation processing on the stored video, and divides the stored video into multiple video segments according to a preset duration; the edge server performs space segmentation processing on each frame of the video segment, and divides each frame into multiple rectangular tiles to obtain a first tile, a second tile, a third tile, and a fourth tile; The edge server uses multiple preset quality levels to encode the first tile, the second tile, the third tile, and the fourth tile respectively, and the multiple preset quality levels correspond to multiple different CRF values ​​of constant rate factors, and the CRF values ​​are set in ascending order from low to high; Each of the multiple user terminals collects motion information of the user, the motion information including translation information and rotation information, wherein the translation information includes the translation position along the X-axis, the translation position along the Y-axis and the translation position along the Z-axis, and the rotation information includes the pitch angle, the yaw angle and the roll angle; the user terminal sends the motion information to the edge layer according to a preset sampling period; The edge server determines the current viewing angle of the user terminal according to the received motion information, selects a target tile of a corresponding quality level from tiles encoded at multiple preset quality levels according to the current viewing angle, and sends the target tile to the corresponding user terminal through a wireless network access point; The user terminal plays the video according to the received target tile, and continuously collects and sends motion information to the edge layer during the video playback process. The edge server updates the selection of the target tile in real time according to the continuously received motion information, realizing adaptive video transmission based on the motion information from the user's perspective.

[0041] The edge server is connected to the wireless network access point via an Ethernet cable to form the edge layer. The edge server performs time segmentation processing on the stored video and divides the video into multiple video segments according to the preset duration (for example, each segment is 10 seconds). The processing of each video segment includes spatial segmentation of each frame and dividing it into multiple rectangular tiles. Assume that each frame is divided into four tiles, named first tile, second tile, third tile and fourth tile respectively.

[0042] The edge server encodes each tile using multiple preset quality levels. Each quality level corresponds to a constant rate factor (CRF), for example, the CRF values ​​from low to high are 18, 23, 28, and 33. The edge server encodes each tile into versions of different quality levels for subsequent selective transmission based on user needs.

[0043] Multiple user terminals are responsible for collecting user motion information, including translation information and rotation information. Translation information includes the change in the user's position in three-dimensional space, specifically the translation position along the X-axis, Y-axis, and Z-axis. Rotation information includes the user's pitch angle, yaw angle, and roll angle. The user terminal sends the motion information to the edge layer according to a preset sampling period (for example, 10 samples per second).

[0044] After receiving the motion information sent by the user terminal, the edge server first determines the user's current viewing angle. Based on the user's translation and rotation information, the edge server calculates the user's viewing direction. Subsequently, the edge server selects a target tile corresponding to the user's current viewing angle from tiles encoded at multiple preset quality levels. For example, if the user's viewing angle corresponds to a high-quality version of the first tile, the edge server will select that tile for transmission.

[0045] The edge server sends the target tile to the corresponding user terminal through the wireless network access point. The user terminal plays the video after receiving the target tile. During the playback process, the user terminal continuously collects and sends motion information to the edge layer. The edge server updates the selection of the target tile in real time based on the continuously received motion information, ensuring that the user always receives the best quality tile that matches his or her perspective.

[0046] The solution of this application can: Improve user experience: By dynamically selecting video tiles based on the user's real-time motion information, ensure that users get the best picture quality and smoothness when watching videos, and improve the overall viewing experience. Optimize bandwidth utilization: By selectively transmitting tiles of different quality levels, the edge server can effectively utilize network bandwidth, reduce unnecessary data transmission, and reduce network burden. Achieve adaptive transmission: This technical solution can adjust the video transmission strategy in real time according to the user's viewing behavior and environmental changes, realize true adaptive video transmission, and meet the needs of different users.

[0047] In an optional implementation, multi-dimensional video playback status information after the user terminal receives the video segment data is collected, a forward and reverse weighted user experience evaluation model is established, the multi-dimensional video playback status information and bandwidth resource cost information are input into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and the system resource utilization efficiency is dynamically optimized and allocated based on the user experience evaluation result, including: Collecting multi-dimensional video playback status information of a user terminal, the multi-dimensional video playback status information including video playback indication information and bandwidth resource cost information, and dividing the video playback indication information into a playback quality parameter and a playback stability parameter; Normalizing the playback quality parameter and the playback stability parameter to obtain a normalized quality parameter and a normalized stability parameter, respectively, calculating a positive eigenvalue based on the normalized quality parameter and the normalized stability parameter, and calculating a negative eigenvalue based on bandwidth resource cost information; Multiply the positive eigenvalue by the first weighting coefficient to obtain a positive experience evaluation value, and multiply the negative eigenvalue by the second weighting coefficient to obtain a negative experience evaluation value, wherein the sum of the first weighting coefficient and the second weighting coefficient is 1, and input the positive experience evaluation value and the negative experience evaluation value into the positive and negative weighted user experience evaluation model to obtain a comprehensive user experience evaluation value; A resource allocation optimization objective function is constructed based on the comprehensive evaluation value of user experience to calculate the current resource utilization efficiency of the system. When the resource utilization efficiency is less than the preset efficiency threshold, a resource optimization allocation plan is generated using the resource allocation optimization objective function. The resource optimization allocation plan includes bandwidth allocation parameters and cache allocation parameters. The system resource utilization efficiency of the user terminal is dynamically optimized and allocated according to the bandwidth allocation parameters and the cache allocation parameters.

[0048] The user terminal needs to collect multi-dimensional video playback status information through specific monitoring tools. This information includes video playback indication information and bandwidth resource cost information. Video playback indication information can be further subdivided into playback quality parameters and playback stability parameters. Playback quality parameters may include resolution, frame rate, bit rate, etc., while playback stability parameters may include buffering times, playback interruption times, etc. Bandwidth resource cost information involves the current network bandwidth usage and available bandwidth.

[0049] Based on the collected playback quality parameters and playback stability parameters, normalization is performed. The purpose of normalization is to convert parameters of different dimensions into a unified standard for subsequent comparison and calculation. Normalization can be achieved by subtracting the minimum value of each parameter and dividing it by its range. The results after processing are normalized quality parameters and normalized stability parameters.

[0050] Based on the normalized quality parameter and the normalized stability parameter, the positive eigenvalue and the negative eigenvalue are calculated respectively. The positive eigenvalue reflects the positive aspects of the user experience, which are usually related to the playback quality and stability. The negative eigenvalue reflects the negative aspects of the user experience, which are mainly related to the bandwidth resource cost information. By analyzing these eigenvalues, we can better understand the user's viewing experience.

[0051] The positive eigenvalue is multiplied by the first weighting coefficient to obtain the positive experience evaluation value; the negative eigenvalue is multiplied by the second weighting coefficient to obtain the negative experience evaluation value. The sum of the first weighting coefficient and the second weighting coefficient is 1 to ensure the rationality of the evaluation value. The positive experience evaluation value and the negative experience evaluation value are input into the positive and negative weighted user experience evaluation model to finally obtain the comprehensive user experience evaluation value.

[0052] Based on the comprehensive evaluation value of user experience, a resource allocation optimization objective function is constructed. This objective function is designed to maximize user experience while minimizing resource waste. By calculating the current system resource utilization efficiency, it is determined whether it is lower than the preset efficiency threshold.

[0053] When the system resource utilization efficiency is lower than the preset efficiency threshold, the resource allocation optimization objective function is used to generate a resource optimization allocation scheme. The scheme includes bandwidth allocation parameters and cache allocation parameters. The bandwidth allocation parameters determine the bandwidth allocation of each user terminal, while the cache allocation parameters affect the cache strategy of the video data.

[0054] Based on the generated bandwidth allocation parameters and cache allocation parameters, the system resources of the user terminal are dynamically optimized and allocated. By adjusting the bandwidth and cache strategy in real time, users can get the best experience when watching videos.

[0055] The solution of this application can: Improve the quality of user experience when watching videos, reduce playback interruptions and buffering, and enhance user satisfaction. Achieve efficient use of system resources, reduce waste of bandwidth and cache resources, and improve overall system performance. By dynamically optimizing allocation plans, adapt to changes in the needs of different users and improve system flexibility and responsiveness.

[0056] In an optional implementation, based on the user experience evaluation result, the video quality difference between user terminals is evaluated, and the video quality difference is constrained in combination with the balance constraint and the minimum quality level in the time dimension, and the user terminals that meet the constraint conditions are constructed as a quality balanced user subset. Under the condition of meeting multiple constraints, the resource allocation of the quality balanced user subset is dynamically optimized to achieve service quality balance in a multi-user scenario, including: Calculating a video quality difference of multiple user terminals, where the video quality difference is determined by a deviation between a current video quality level of each user terminal and an average video quality level of all user terminals; Establishing a quality balancing constraint condition in the time dimension, the quality balancing constraint condition includes that the difference in video quality levels between any two user terminals within a preset video duration does not exceed a preset quality difference threshold; Obtaining a minimum video quality level among multiple user terminals, and constructing a quality-balanced user subset according to the minimum video quality level, wherein a difference between a user terminal in the quality-balanced user subset and the minimum video quality level does not exceed a preset maximum quality difference value; Obtaining bandwidth resource utilization efficiency of each user terminal in the quality-balanced user subset, and optimizing resource allocation for the user terminals in the quality-balanced user subset according to the bandwidth resource utilization efficiency; Determine whether the resource allocation optimization satisfies bandwidth resource constraints, quality balancing constraints, and minimum quality assurance constraints, where the bandwidth resource constraint is that the sum of bandwidth resources of all user terminals does not exceed the upper limit of system bandwidth resources, the quality balancing constraint is that the difference in video quality levels of any two user terminals in the quality balancing user subset does not exceed a preset maximum quality difference value, and the minimum quality assurance constraint is that the video quality level of all user terminals is not lower than the minimum video quality level; When bandwidth resource constraints, quality balance constraints, and minimum quality assurance constraints are met, the video quality level and resource allocation status of the user terminal are updated; The video quality difference is recalculated according to the updated video quality level and resource allocation status, and the resource allocation strategy is adjusted based on the video quality difference to achieve dynamic balance of service quality in multi-user scenarios.

[0057] Calculate the video quality differences of multiple user terminals. The video quality difference is determined by the deviation between the current video quality level of each user terminal and the average video quality level of all user terminals. Specifically, first collect the current video quality levels of all user terminals, and then calculate the average of these levels. Then, calculate the difference between the video quality level and the average for each user terminal to form a difference list.

[0058] Establish a quality balance constraint in the time dimension. This constraint requires that within the preset video duration, the difference in video quality between any two user terminals does not exceed the preset quality difference threshold. To this end, the system needs to monitor the quality changes of each user terminal during video playback and check the quality difference between all user terminals at each time point to ensure that it meets the set threshold.

[0059] Get the minimum video quality level among multiple user terminals. By analyzing the quality levels of all user terminals, determine the minimum value, and construct a quality balanced user subset based on the minimum video quality level. The difference between the user terminals in this subset and the minimum video quality level does not exceed the preset maximum quality difference value. This step ensures that the quality level of all user terminals is within an acceptable range.

[0060] Obtain the bandwidth resource utilization efficiency of each user terminal in the quality-balanced user subset. Bandwidth resource utilization efficiency refers to the ratio of the bandwidth actually used by the user terminal to its available bandwidth under the current network conditions. By monitoring the bandwidth usage of the user terminal, the utilization efficiency of each terminal is calculated, and based on this data, resource allocation optimization is performed for the user terminals in the quality-balanced user subset.

[0061] Determine whether the resource allocation optimization meets the bandwidth resource constraint, quality balance constraint, and minimum quality assurance constraint. The bandwidth resource constraint requires that the sum of the bandwidth resources of all user terminals does not exceed the system's bandwidth resource upper limit. The quality balance constraint requires that the difference in video quality levels between any two user terminals in the quality balance user subset does not exceed the preset maximum quality difference value. The minimum quality assurance constraint requires that the video quality level of all user terminals is not lower than the minimum video quality level.

[0062] When all constraints are met, the video quality level and resource allocation status of the user terminal are updated. After the update, the system recalculates the video quality difference according to the new video quality level and resource allocation status, and adjusts the resource allocation strategy based on this to achieve dynamic balance of service quality in multi-user scenarios.

[0063] The solution of this application can: Improve user experience: By dynamically optimizing resource allocation, ensure that the video quality of all user terminals is within an acceptable range, thereby improving the overall user experience. Optimize resource utilization: Through effective management of bandwidth resources, avoid resource waste and improve the utilization efficiency of network resources. Enhance system stability: Through real-time monitoring and adjustment, ensure the stability of the system in multi-user scenarios and reduce user loss due to quality differences.

[0064] In an optional implementation, the proposed quality level of each user terminal in the quality-balanced user subset is obtained, a multi-dimensional resource verification model is constructed by combining quality threshold verification, individual resource verification, and system resource verification, the proposed quality level is input into the multi-dimensional resource verification model for verification, and the video quality level of the user terminal is dynamically optimized and allocated based on the verification result, including: Obtaining a proposed quality level of the user terminal, where the proposed quality level is determined based on the current video playback quality level of the user terminal, and determining whether the proposed quality level exceeds a preset quality threshold. When the proposed quality level exceeds the preset quality threshold, the user terminal is removed from the user set; The bandwidth resource demand of the user terminal is calculated based on the proposed quality level. The bandwidth resource demand is calculated based on the bandwidth demand of the user terminal, the transmission delay requirement and the proposed quality level. It is determined whether the bandwidth resource demand exceeds the individual throughput upper limit of the user terminal. When the bandwidth resource demand exceeds the individual throughput upper limit, the user terminal is removed from the user set. Calculate the total resource requirements of all user terminals in the user set at the proposed quality level, determine whether the total resource requirements exceed the total available throughput of the edge server, and remove the user terminal from the user set when the total resource requirements exceed the total available throughput; Construct a multi-dimensional resource verification model, which includes quality threshold verification, individual resource verification, and system resource verification. The resource availability of the user terminal is determined based on the multi-dimensional resource verification model. The video quality level of the user terminal is updated according to the resource availability. When the resource availability meets the preset conditions, the video quality level of the user terminal is increased by a preset level based on the current video quality level. The acquisition of the proposed quality level, the calculation of the bandwidth resource requirement, the calculation of the overall resource requirement, the determination of resource availability, and the update of the video quality level are executed cyclically until all user terminals in the user set complete resource allocation.

[0065] Get the proposed quality level of the user terminal. The proposed quality level is determined based on the current video playback quality level of the user terminal. The system monitors the real-time video playback of each user terminal and generates a proposed quality level based on the playback quality. If the proposed quality level exceeds the preset quality threshold, the user terminal is removed from the user set to ensure that only user terminals that meet the quality standards participate in subsequent resource allocation.

[0066] The bandwidth resource requirements of the user terminal are calculated based on the proposed quality level. The calculation of the bandwidth resource requirements takes into account the bandwidth requirements of the user terminal, the transmission delay requirements, and the proposed quality level. The system evaluates the bandwidth required by each user terminal under the current network conditions and compares it with the individual throughput limit. If the bandwidth resource requirement exceeds the individual throughput limit, the user terminal is removed from the user set to ensure the reasonable allocation of network resources.

[0067] Calculate the total resource requirements of all user terminals in the user set at the proposed quality level. The system summarizes the bandwidth requirements of all user terminals and determines whether the total resource requirements exceed the total available throughput of the edge server. If the total resource requirements exceed the total available throughput, the corresponding user terminal is removed from the user set to avoid network congestion and resource waste.

[0068] Construct a multi-dimensional resource verification model. The model includes quality threshold verification, individual resource verification, and system resource verification. Through the multi-dimensional resource verification model, the system can comprehensively judge the resource availability of user terminals and ensure that each user terminal can meet the preset conditions when allocating resources.

[0069] Update the video quality level of the user terminal based on resource availability. When the resource availability meets the preset conditions, the system will improve the video quality level of the user terminal based on the current video quality level to optimize the user experience.

[0070] The acquisition of the proposed quality level, the calculation of bandwidth resource requirements, the calculation of the overall resource requirements, the determination of resource availability, and the update of the video quality level are performed cyclically until all user terminals in the user set complete resource allocation. This cyclic process ensures the dynamic and real-time nature of resource allocation, allowing the system to flexibly adjust according to network conditions and user needs.

[0071] The solution of this application can: Improves the video playback quality of user terminals, ensuring that users get a better experience when watching videos. Optimizes the allocation of network resources, avoiding the degradation of user terminal quality due to insufficient resources. Enhances the system's dynamic adjustment capability, allowing user terminals to flexibly adapt to different network conditions and maintain good service quality.

[0072] In an optional implementation, integrity check and timing synchronization processing are performed on the video slice data transmitted after dynamic optimization allocation, the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources, and the decoded video slice data is input into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame, including: Receiving panoramic video slice data, performing integrity check on the panoramic video slice data, obtaining a check result by calculating a hash value of the panoramic video slice data, and determining the validity of the panoramic video slice data based on the check result; Performing time synchronization processing on the panoramic video slice data, determining the alignment time of the panoramic video slice data according to the reference timestamp and the time offset of the panoramic video slice data, and sorting the panoramic video slice data according to the alignment time; Obtain processing capability parameters of the decoding device, the processing capability parameters include the processing capability of the central processing unit and the processing capability of the graphics processing unit, determine the resource allocation weight of the decoding task according to the processing capability parameters, and call the hardware decoder to decode the panoramic video slice data based on the resource allocation weight; A spherical projection model is constructed to map the decoded panoramic video fragment data from plane coordinates to spherical coordinates, the pixel fusion weights are calculated based on the spherical coordinates, and the panoramic video fragment data are reconstructed according to the pixel fusion weights to obtain a panoramic frame.

[0073] After receiving the panoramic video segment data, the integrity check is first performed. By calculating the hash value of the video segment data, a unique check code is generated. This check code is compared with the pre-stored check code to determine the validity of the data. If the check code matches, the data is valid; otherwise, the data needs to be requested again. This step ensures that the received data has not been tampered with or damaged during transmission.

[0074] After confirming the data integrity, the timing synchronization process is performed. The alignment time of each video segment is calculated based on the reference timestamp and the time offset of the video segment data. Alignment time means adjusting the video segments with different timestamps to the same time reference. Then, the video segments are sorted according to the alignment time to ensure the timing consistency of the data in subsequent processing.

[0075] Obtain the processing power parameters of the decoding device, including the processing power of the CPU and GPU. Based on these parameters, determine the resource allocation weights of the decoding task. Resource allocation weights refer to how to reasonably allocate computing resources to improve decoding efficiency during the decoding process. Based on these weights, call the hardware decoder to decode the panoramic video slice data to ensure that the decoding process is efficient and stable.

[0076] After decoding is completed, a spherical projection model is constructed. The decoded panoramic video slice data is mapped from plane coordinates to spherical coordinates. This process involves converting the plane coordinates of each pixel into spherical coordinates for panoramic reconstruction. The pixel fusion weights are calculated based on the spherical coordinates, and the fusion weights are used to determine the degree of influence of different video slices in the reconstruction process. Finally, the panoramic video slice data is reconstructed according to the fusion weights to generate the final panoramic frame.

[0077] The solution of this application can: Improve the reliability of data transmission: Through integrity verification, ensure that the received video segment data has not been tampered with or damaged during transmission, thereby improving data reliability. Enhance the smoothness of video playback: Timing synchronization processing ensures that video segments are played in the correct time sequence, avoiding playback jams caused by time misalignment, and improving user experience. Optimize the efficiency of decoding resources: Through intelligent scheduling of decoding resources, reasonable allocation of computing power, improved decoding efficiency, reduced latency, and ensure high-quality panoramic video playback.

[0078] A second aspect of an embodiment of the present invention provides a system for adaptively adjusting a rate of a multi-user VR video streaming media, including: The first unit is used to construct a two-layer architecture of the edge layer and the user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the video at the edge layer to obtain video slice data, collect translational motion information and rotational motion information of the user terminal in the user layer, and selectively transmit the video slice data based on the translational motion information and the rotational motion information to achieve adaptive video transmission based on the user perspective; The second unit is used to collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; The third unit is used to evaluate the video quality difference between user terminals based on the user experience evaluation result, constrain the video quality difference by combining the balance constraint in the time dimension and the minimum quality level, construct the user terminals that meet the constraint conditions into a quality-balanced user subset, dynamically optimize the resource allocation of the quality-balanced user subset under the condition of meeting multiple constraints, and realize service quality balance in a multi-user scenario; The fourth unit is used to obtain the proposed quality level of each user terminal in the quality balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize and allocate the video quality level of the user terminal based on the verification result; The fifth unit is used to perform integrity check and timing synchronization processing on the video slice data transmitted after dynamic optimization allocation, decode the video slice data in combination with the intelligent scheduling mechanism of hardware decoding resources, and input the decoded video slice data into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

[0079] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0080] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0081] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adaptively adjusting the rate of multi-user VR video streaming media, characterized in that: include: Construct a two-layer architecture of edge layer and user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the edge layer to obtain video slice data, collect translation and rotation information of user terminals in the user layer, and selectively transmit video slice data based on translation and rotation information to achieve adaptive video transmission based on user perspective; Collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; Based on the user experience evaluation results, the video quality differences between user terminals are evaluated. The video quality differences are constrained by combining the time dimension balance constraints and the minimum quality level. The user terminals that meet the constraints are constructed as quality-balanced user subsets. Under the conditions of meeting multiple constraints, the resource allocation of the quality-balanced user subsets is dynamically optimized to achieve service quality balance in multi-user scenarios. Obtain the proposed quality level of each user terminal in the quality-balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize the video quality level of the user terminal based on the verification result; The video slice data transmitted after dynamic optimization allocation is subjected to integrity check and timing synchronization processing, and the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources. The decoded video slice data is input into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

2. The method according to claim 1, characterized in that: Construct a two-layer architecture of edge layer and user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the edge layer to obtain video slice data, collect translation and rotation information of user terminals in the user layer, and selectively transmit video slice data based on translation and rotation information to achieve adaptive video transmission based on user perspective, including: The edge server is connected to the wireless network access point via an Ethernet cable. The edge server and the wireless network access point form an edge layer. The edge layer is connected to the user layer via wireless communication. The user layer includes multiple user terminals. The edge server performs time segmentation processing on the stored video, and divides the stored video into multiple video segments according to a preset duration; the edge server performs space segmentation processing on each frame of the video segment, and divides each frame into multiple rectangular tiles to obtain a first tile, a second tile, a third tile, and a fourth tile; The edge server uses multiple preset quality levels to encode the first tile, the second tile, the third tile, and the fourth tile respectively, and the multiple preset quality levels correspond to multiple different CRF values ​​of constant rate factors, and the CRF values ​​are set in ascending order from low to high; Each of the multiple user terminals collects motion information of the user, the motion information including translation information and rotation information, wherein the translation information includes the translation position along the X-axis, the translation position along the Y-axis and the translation position along the Z-axis, and the rotation information includes the pitch angle, the yaw angle and the roll angle; the user terminal sends the motion information to the edge layer according to a preset sampling period; The edge server determines the current viewing angle of the user terminal according to the received motion information, selects a target tile of a corresponding quality level from tiles encoded at multiple preset quality levels according to the current viewing angle, and sends the target tile to the corresponding user terminal through a wireless network access point; The user terminal plays the video according to the received target tile, and continuously collects and sends motion information to the edge layer during the video playback process. The edge server updates the selection of the target tile in real time according to the continuously received motion information, realizing adaptive video transmission based on the motion information from the user's perspective.

3. The method according to claim 1, characterized in that Collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize the system resource utilization efficiency based on the user experience evaluation result, including: Collecting multi-dimensional video playback status information of a user terminal, the multi-dimensional video playback status information including video playback indication information and bandwidth resource cost information, and dividing the video playback indication information into a playback quality parameter and a playback stability parameter; Normalizing the playback quality parameter and the playback stability parameter to obtain a normalized quality parameter and a normalized stability parameter, respectively, calculating a positive eigenvalue based on the normalized quality parameter and the normalized stability parameter, and calculating a negative eigenvalue based on bandwidth resource cost information; Multiply the positive eigenvalue by the first weighting coefficient to obtain a positive experience evaluation value, and multiply the negative eigenvalue by the second weighting coefficient to obtain a negative experience evaluation value, wherein the sum of the first weighting coefficient and the second weighting coefficient is 1, and input the positive experience evaluation value and the negative experience evaluation value into the positive and negative weighted user experience evaluation model to obtain a comprehensive user experience evaluation value; A resource allocation optimization objective function is constructed based on the comprehensive evaluation value of user experience to calculate the current resource utilization efficiency of the system. When the resource utilization efficiency is less than the preset efficiency threshold, a resource optimization allocation plan is generated using the resource allocation optimization objective function. The resource optimization allocation plan includes bandwidth allocation parameters and cache allocation parameters. The system resource utilization efficiency of the user terminal is dynamically optimized and allocated according to the bandwidth allocation parameters and the cache allocation parameters.

4. The method according to claim 1, characterized in that: Based on the user experience evaluation results, the video quality differences between user terminals are evaluated. The video quality differences are constrained by combining the time dimension balance constraints and the minimum quality level. The user terminals that meet the constraints are constructed as quality-balanced user subsets. Under the conditions of meeting multiple constraints, the resource allocation of the quality-balanced user subsets is dynamically optimized to achieve service quality balance in multi-user scenarios, including: Calculating a video quality difference of multiple user terminals, where the video quality difference is determined by a deviation between a current video quality level of each user terminal and an average video quality level of all user terminals; Establishing a quality balancing constraint condition in the time dimension, the quality balancing constraint condition includes that the difference in video quality levels between any two user terminals within a preset video duration does not exceed a preset quality difference threshold; Obtaining a minimum video quality level among multiple user terminals, and constructing a quality-balanced user subset according to the minimum video quality level, wherein a difference between a user terminal in the quality-balanced user subset and the minimum video quality level does not exceed a preset maximum quality difference value; Obtaining bandwidth resource utilization efficiency of each user terminal in the quality-balanced user subset, and optimizing resource allocation for the user terminals in the quality-balanced user subset according to the bandwidth resource utilization efficiency; Determine whether the resource allocation optimization satisfies bandwidth resource constraints, quality balancing constraints, and minimum quality assurance constraints, where the bandwidth resource constraint is that the sum of bandwidth resources of all user terminals does not exceed the upper limit of system bandwidth resources, the quality balancing constraint is that the difference in video quality levels of any two user terminals in the quality balancing user subset does not exceed a preset maximum quality difference value, and the minimum quality assurance constraint is that the video quality level of all user terminals is not lower than the minimum video quality level; When bandwidth resource constraints, quality balance constraints, and minimum quality assurance constraints are met, the video quality level and resource allocation status of the user terminal are updated; The video quality difference is recalculated according to the updated video quality level and resource allocation status, and the resource allocation strategy is adjusted based on the video quality difference to achieve dynamic balance of service quality in multi-user scenarios.

5. The method according to claim 1, characterized in that Obtain the proposed quality level of each user terminal in the quality-balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification, and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize the video quality level of the user terminal based on the verification result, including: Obtaining a proposed quality level of the user terminal, where the proposed quality level is determined based on the current video playback quality level of the user terminal, and determining whether the proposed quality level exceeds a preset quality threshold. When the proposed quality level exceeds the preset quality threshold, the user terminal is removed from the user set; The bandwidth resource demand of the user terminal is calculated based on the proposed quality level. The bandwidth resource demand is calculated based on the bandwidth demand of the user terminal, the transmission delay requirement and the proposed quality level. It is determined whether the bandwidth resource demand exceeds the individual throughput upper limit of the user terminal. When the bandwidth resource demand exceeds the individual throughput upper limit, the user terminal is removed from the user set. Calculate the total resource requirements of all user terminals in the user set at the proposed quality level, determine whether the total resource requirements exceed the total available throughput of the edge server, and remove the user terminal from the user set when the total resource requirements exceed the total available throughput; Construct a multi-dimensional resource verification model, which includes quality threshold verification, individual resource verification, and system resource verification. The resource availability of the user terminal is determined based on the multi-dimensional resource verification model. The video quality level of the user terminal is updated according to the resource availability. When the resource availability meets the preset conditions, the video quality level of the user terminal is increased by a preset level based on the current video quality level. The acquisition of the proposed quality level, the calculation of the bandwidth resource requirement, the calculation of the overall resource requirement, the determination of resource availability, and the update of the video quality level are executed cyclically until all user terminals in the user set complete resource allocation.

6. The method according to claim 1, characterized in that The video slice data transmitted after dynamic optimization allocation is subjected to integrity check and timing synchronization processing, the video slice data is decoded in combination with the intelligent scheduling mechanism of hardware decoding resources, and the decoded video slice data is input into the spherical projection model to realize the stereoscopic reconstruction of the panoramic frame, including: Receiving panoramic video slice data, performing integrity check on the panoramic video slice data, obtaining a check result by calculating a hash value of the panoramic video slice data, and determining the validity of the panoramic video slice data based on the check result; Performing time synchronization processing on the panoramic video slice data, determining the alignment time of the panoramic video slice data according to the reference timestamp and the time offset of the panoramic video slice data, and sorting the panoramic video slice data according to the alignment time; Obtain processing capability parameters of the decoding device, the processing capability parameters include the processing capability of the central processing unit and the processing capability of the graphics processing unit, determine the resource allocation weight of the decoding task according to the processing capability parameters, and call the hardware decoder to decode the panoramic video slice data based on the resource allocation weight; A spherical projection model is constructed to map the decoded panoramic video fragment data from plane coordinates to spherical coordinates, the pixel fusion weights are calculated based on the spherical coordinates, and the panoramic video fragment data are reconstructed according to the pixel fusion weights to obtain a panoramic frame.

7. A system for adaptively adjusting the rate of multi-user VR video streaming media, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit is used to construct a two-layer architecture of the edge layer and the user layer, perform spatiotemporal segmentation and multi-quality encoding processing on the video at the edge layer to obtain video slice data, collect translational motion information and rotational motion information of the user terminal in the user layer, and selectively transmit the video slice data based on the translational motion information and the rotational motion information to achieve adaptive video transmission based on the user perspective; The second unit is used to collect multi-dimensional video playback status information after the user terminal receives the video segment data, establish a forward and reverse weighted user experience evaluation model, input the multi-dimensional video playback status information and bandwidth resource cost information into the forward and reverse weighted user experience evaluation model to obtain a user experience evaluation result, and dynamically optimize and allocate system resource utilization efficiency based on the user experience evaluation result; The third unit is used to evaluate the video quality difference between user terminals based on the user experience evaluation result, constrain the video quality difference by combining the balance constraint in the time dimension and the minimum quality level, construct the user terminals that meet the constraint conditions into a quality-balanced user subset, dynamically optimize the resource allocation of the quality-balanced user subset under the condition of meeting multiple constraints, and realize service quality balance in a multi-user scenario; The fourth unit is used to obtain the proposed quality level of each user terminal in the quality balanced user subset, build a multi-dimensional resource verification model by combining quality threshold verification, individual resource verification and system resource verification, input the proposed quality level into the multi-dimensional resource verification model for verification, and dynamically optimize and allocate the video quality level of the user terminal based on the verification result; The fifth unit is used to perform integrity check and timing synchronization processing on the video slice data transmitted after dynamic optimization allocation, decode the video slice data in combination with the intelligent scheduling mechanism of hardware decoding resources, and input the decoded video slice data into the spherical projection model to realize stereoscopic reconstruction of the panoramic frame.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images