Video playing bandwidth allocation method, computer equipment and storage medium

Abstract

The invention relates to a video playing bandwidth allocation method, computer equipment and a storage medium. The method comprises the following steps: receiving a plurality of video playing requests sent by a user side, and determining a plurality of videos to be played corresponding to the plurality of video playing requests; respectively determining a user feature and a video feature corresponding to each video to be played, wherein the video feature comprises video complexity; determining the experience quality gain of each to-be-played video according to the user features and the video features; and according to the experience quality gain and the bandwidth consumption of each to-be-played video, bandwidth allocation of the to-be-played video is carried out. Therefore, the resource efficiency and the user experience can be effectively balanced within the limited bandwidth prediction, so that the decision of bandwidth allocation can well serve the experience quality gain, the optimization of bandwidth resources is realized, the bandwidth resources are combined with the requirements of actual video services, and the user experience quality is improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for allocating video playback bandwidth. Background Art

[0002] With the rapid development of video processing technology, various video applications are becoming more and more integrated into people's lives, such as video conferencing, video surveillance, and short videos on mobile devices. As the video-oriented trend of Internet services becomes more and more obvious, how to better improve user satisfaction with video services has become increasingly important.

[0003] In traditional technologies, objective network performance indicators are generally used to evaluate video services. For example, quality of service (QoS) is used to allocate video service traffic.

[0004] However, the QoS-based approach is not precise enough in allocating video service traffic. It focuses more on network performance and cannot effectively balance resource efficiency and user experience. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for allocating video playback bandwidth that can effectively balance resource efficiency and user experience within a limited bandwidth budget to address the above technical problems.

[0006] In a first aspect, the present application provides a method for allocating video playback bandwidth, comprising:

[0007] Receiving multiple video playback requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video playback requests;

[0008] Respectively determining user features and video features corresponding to each of the videos to be played, wherein the video features include video complexity;

[0009] Determining, according to the user characteristics and the video characteristics, a quality of experience gain of each of the videos to be played;

[0010] Bandwidth allocation for the videos to be played is performed according to the experience quality gain and bandwidth consumption of each of the videos to be played.

[0011] In one of the embodiments, determining the experience quality gain of each of the to-be-played videos according to the user characteristics and the video characteristics includes:

[0012] Extracting and fusing the user features and the video features to obtain weighted features;

[0013] Extracting and concatenating the weighted features to obtain a vector for representing the experience quality gain;

[0014] The experience quality gain of each to-be-played video is determined according to the vector used to characterize the experience quality gain.

[0015] In one embodiment, the user characteristics include: user identity portrait, user historical consumption data; wherein the user historical consumption data includes: user experience data in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode, and the user experience data includes: playback time, interaction index, e-commerce and commercial revenue, any one of them, or any combination of two or more;

[0016] The video features also include: video content, and the difference in image quality when the video is played in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode.

[0017] In one embodiment, the extracting and fusing the user features and the video features to obtain weighted features includes:

[0018] Performing vector extraction on the user features and the video features to obtain a user feature vector and a video feature vector;

[0019] Performing collaborative attention fusion processing on the user feature vector and the video feature vector to obtain an attention weight of the user feature to the video feature, and an attention weight of the video feature to the user feature;

[0020] By using the attention weight of the user feature on the video feature, the video feature is weightedly aggregated to obtain the video feature perceived by the user;

[0021] The user features are weightedly aggregated by the attention weights of the video features to the user features to obtain the video-aware user features.

[0022] In one of the embodiments, the extracting and concatenating the weighted features to obtain a vector for characterizing the experience quality gain includes:

[0023] Performing vector extraction on the video features perceived by the user and the user features perceived by the video to obtain a user-perceived video feature vector and a video-perceived user feature vector;

[0024] The user-perceived video feature vector and the video-perceived user feature vector are concatenated to obtain a vector for representing the experience quality gain.

[0025] In one of the embodiments, before allocating bandwidth of the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played, the method further includes: obtaining bandwidth consumption of each of the videos to be played, wherein:

[0026] The method of obtaining the bandwidth consumption of each video to be played includes any of the following methods:

[0027] Determine the bandwidth required for each video to be played when it is played in a preset high-definition bandwidth mode, and use the bandwidth required for playing in the preset high-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0028] Determine the bandwidth required for each video to be played when it is played in a preset low-definition bandwidth mode, and use the bandwidth required for playing in the preset low-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0029] The number of times each video to be played is estimated according to the historical playback data of each video to be played, and the bandwidth consumption of the corresponding video to be played is determined according to the estimated number of times each video to be played.

[0030] In one of the embodiments, allocating bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played includes:

[0031] Taking the ratio between the experience quality gain of the video to be played and the bandwidth consumption of the video to be played as the playback efficiency of the video to be played;

[0032] According to the playback efficiency of each video to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of each video to be played;

[0033] Bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

[0034] In one embodiment, allocating bandwidth to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played includes:

[0035] According to the total bandwidth budget, starting from the to-be-played video corresponding to the highest priority, the preset high-definition bandwidth is allocated one by one from high to low according to the bandwidth allocation priority of each to-be-played video, until the total bandwidth budget is allocated.

[0036] In one of the embodiments, allocating bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played includes:

[0037] respectively setting a first weight corresponding to the experience quality gain of the video to be played and a second weight corresponding to the bandwidth consumption of the video to be played;

[0038] Calculating a comprehensive score corresponding to each of the to-be-played videos based on the first weight and the second weight;

[0039] According to the comprehensive scores of the videos to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of the videos to be played;

[0040] Bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

[0041] In a second aspect, the present application further provides a video playback device, comprising:

[0042] A receiving module, used for receiving multiple video playback requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video playback requests;

[0043] A feature determination module, used to respectively determine user features and video features corresponding to each of the videos to be played, wherein the video features include video complexity;

[0044] An experience quality gain determination module, used to determine the experience quality gain of each of the videos to be played according to the user characteristics and the video characteristics;

[0045] The bandwidth allocation module is used to allocate bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played.

[0046] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any one of the embodiments of the first aspect is implemented.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any embodiment of the first aspect.

[0048] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any embodiment of the first aspect.

[0049] The above-mentioned method, device, computer equipment, computer-readable storage medium and computer program product for allocating video playback bandwidth receive multiple video playback requests sent by the user end, and determine multiple videos to be played corresponding to the multiple video playback requests; thereby, multiple video playback requests can be processed to improve processing efficiency. By respectively determining the user characteristics and video characteristics corresponding to each of the videos to be played, the video characteristics include video complexity; thereby, the influence of the user and the video itself on the experience quality gain can be comprehensively considered, providing a basis for the subsequent bandwidth allocation, and balancing the user experience and the consumption of bandwidth resources as much as possible. By determining the experience quality gain of each of the videos to be played according to the user characteristics and the video characteristics; thereby, the user experience and resource efficiency can be quantified through the experience quality gain. By allocating bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played. Thus, within the limited bandwidth budget, resource efficiency and user experience can be effectively balanced, so that the decision of bandwidth allocation serves the experience quality gain well. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 A diagram of an application environment of a method for allocating bandwidth for video playback in one embodiment;

[0052] Figure 2A One of the interactive signaling flow charts between a terminal and a server in one embodiment;

[0053] Figure 2B FIG2 is a second flow chart of interactive signaling between a terminal and a server in an embodiment;

[0054] Figure 3 A schematic diagram of an interface for playing multiple videos in split screen on a terminal in one embodiment;

[0055] Figure 4 It is a flowchart of the method for allocating video playback bandwidth in the first embodiment;

[0056] Figure 5 This is a flow chart of step 401 in the first embodiment;

[0057] Figure 6 is a schematic diagram of a processing process of a target model in one embodiment;

[0058] Figure 7 It is a flow chart of the method for allocating video playback bandwidth in the second embodiment;

[0059] Fig. 8A The third is a flow chart of interactive signaling between a terminal and a server in one embodiment;

[0060] Figure 8B FIG4 is a fourth flow chart of interactive signaling between a terminal and a server in one embodiment;

[0061] Fig. 9 It is a flowchart of the method for allocating video playback bandwidth in the third embodiment;

[0062] Fig.10 is a flow chart of a method for allocating video playback bandwidth in a fourth embodiment;

[0063] Fig.11 A schematic diagram of a flow chart of a method for determining a video experience quality gain in one embodiment;

[0064] Fig.12 is a schematic flow chart of step 1102 in one embodiment;

[0065] Fig.13A FIG5 is a fifth flow chart of interactive signaling between a terminal and a server in one embodiment;

[0066] Fig. 13B FIG6 is a sixth flow chart of interactive signaling between a terminal and a server in one embodiment;

[0067] Fig. 13C FIG7 is a flow chart of interactive signaling between a terminal and a server in one embodiment;

[0068] Fig.13D FIG8 is an eighth flowchart of interactive signaling between a terminal and a server in one embodiment;

[0069] Fig.14 It is a structural block diagram of a device for allocating bandwidth for video playback in one embodiment;

[0070] Fig.15 A structural block diagram of an apparatus for determining a video experience quality gain in one embodiment;

[0071] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0073] The video playback method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 (that is, the user terminal) communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process (for example, user features and video features). The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Exemplarily, the server 104 receives multiple video playback requests sent by the terminal 102, and determines multiple videos to be played corresponding to the multiple video playback requests; respectively determines the user features and video features corresponding to each of the videos to be played, and the video features include video complexity; according to the user features and the video features, determines the experience quality gain of each of the videos to be played; according to the experience quality gain and bandwidth consumption of each of the videos to be played, the bandwidth of the videos to be played is allocated. Finally, different bandwidths are used to play videos on the terminal 102, so that within the limited bandwidth budget, resource efficiency and user experience can be effectively balanced, so that the decision of bandwidth allocation serves the experience quality gain well. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0074] For example, Figure 2A As shown, combined Figure 1 The server and terminal shown in the figure describe the signaling interaction process between the terminal and the server when the method for allocating video playback bandwidth in the embodiment of the present application is applied. Figure 2AIn the embodiment, there are terminal 1, terminal 2, ..., terminal N, a total of N terminals, where N is a natural number greater than 2. Assume that there is a special case, that is, there are multiple terminals among the N terminals that send video playback requests to the server, and the sending time of these video playback requests is within a preset time period. At this time, since the total bandwidth budget of the server is limited, it is necessary to first determine the user characteristics and video characteristics corresponding to each video to be played, and then determine the experience quality gain of each video to be played based on the user characteristics and video characteristics. Furthermore, the server obtains the bandwidth consumption of each video to be played, and allocates the bandwidth of the video to be played based on the experience quality gain and bandwidth consumption of each video to be played. This allocation method can well balance user experience and resource efficiency.

[0075] For example, Figure 2B As shown, combined Figure 1 The server and terminal shown in the figure describe the signaling interaction process between the terminal and the server when the method for allocating video playback bandwidth in the embodiment of the present application is applied. Figure 2A Unlike the embodiment in FIG. 1 , there is only one terminal in this embodiment. However, the terminal has Figure 3 The situation shown is that two or more videos are played simultaneously on the interface of a terminal. Assume that there is a special situation, that is, the user triggers multiple video playback requests on the terminal interface, and the terminal device supports split-screen playback mode. At this time, the server first determines the user characteristics and video characteristics corresponding to each video to be played, and then determines the experience quality gain of each video to be played based on the user characteristics and video characteristics. Furthermore, the server obtains the bandwidth consumption of each video to be played, and allocates the bandwidth of the video to be played according to the experience quality gain and bandwidth consumption of each video to be played. This allocation method can well balance user experience and resource efficiency.

[0076] It should be noted that in Figure 3 Only a schematic diagram of the interface for playing video 1 and video 2 simultaneously is shown. In actual applications, the number of videos played simultaneously on the terminal interface may be greater than two. Therefore, the number of videos that can be played simultaneously by the terminal is not limited in this embodiment, and can be adjusted according to the performance of the terminal itself or the application function used.

[0077] In a first exemplary embodiment, if Figure 4 As shown, a method for allocating video playback bandwidth is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 401 to 404. Among them:

[0078] Step 401: receiving multiple video play requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video play requests.

[0079] In this embodiment, the user terminal corresponds to Figure 1 In the terminal shown in , the server receives multiple video play requests sent by the terminal, and determines the to-be-played video corresponding to each video play request. It should be noted that the user terminal can be one or more than one. When there is one user terminal, it is generally considered that the device of the user terminal has a window for playing two or more videos at the same time.

[0080] Step 402: Determine the user features and video features corresponding to each video to be played.

[0081] Wherein, the video features include video complexity.

[0082] Exemplarily, the user characteristics include: user identity portrait, user historical consumption data; wherein, the user historical consumption data includes: user experience data in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode, and the user experience data includes: playback time, interaction indicators, any one of e-commerce and commercial revenue, or any combination of two or more.

[0083] Exemplarily, the video features may also include: video content, and a difference in image quality when the video is played in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode.

[0084] It should be noted that the picture quality resolution corresponding to the preset high-definition bandwidth mode and the picture quality resolution corresponding to the preset low-definition bandwidth mode are not limited in this embodiment. In the embodiment of the present application, it is only used to characterize two different bandwidth modes, one corresponding to high definition and the other to low definition. For example, the picture quality resolution corresponding to the preset high-definition bandwidth mode is 1080p, and the picture quality resolution corresponding to the preset low-definition bandwidth mode is 720p. For another example, the picture quality resolution corresponding to the preset high-definition bandwidth mode can be set to 2K, and the picture quality resolution corresponding to the preset low-definition bandwidth mode can be set to 1080p. The high and low in the embodiment of the present application are a relative probability, and their specific values ​​are not limited.

[0085] Step 403: Determine the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0086] Quality of Experience (QoE) refers to the user's comprehensive subjective feeling about the quality and performance (including effectiveness and availability) of equipment, network and system, application or service, that is, it is defined from the comfort of service application. Through QoE scoring, operators can comprehensively evaluate the quality and performance of video services and optimize the network.

[0087] In this embodiment, the quality of experience gain (QoE gain for short) refers to the increment of user experience data in the preset high-definition bandwidth mode compared with the preset low-definition bandwidth mode. For example, in the preset high-definition bandwidth mode, the increment of video service in interactive indicators such as playback time, likes, reposts and comments, as well as e-commerce and commercial revenue.

[0088] It should be noted that this embodiment does not limit the type of video services, nor the type of user experience data, and any data related to user experience evaluation can be applied in the embodiments of this application.

[0089] Step 404: Allocate bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0090] In this embodiment, the bandwidth allocation priority of each video to be played is determined by comprehensively considering the experience quality gain and bandwidth consumption of each video to be played. Then, bandwidth allocation is performed based on the bandwidth allocation priority of each video to be played.

[0091] In an optional implementation, the ratio between the experience quality gain of the video to be played and the bandwidth consumption of the video to be played can be used as the playback efficiency of the video to be played; according to the playback efficiency of each video to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of each video to be played; and bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

[0092] For the above implementation, according to the total bandwidth budget, starting from the to-be-played video corresponding to the highest priority, the preset high-definition bandwidth can be allocated one by one from high to low according to the bandwidth allocation priority of each to-be-played video until the total bandwidth budget is allocated.

[0093] In this embodiment, the experience quality gain of the video to be played is used to characterize the increase in QoE (for example, the increase in play time, the increase in interactive indicators such as likes, forwarding and comments, and the increase in revenue from e-commerce and commercialization, etc.) when the video is played in a preset high-definition bandwidth mode compared to when the video is played in a preset low-definition bandwidth mode. Bandwidth consumption generally refers to the bandwidth required in different playback modes. The playback efficiency of the video to be played is comprehensively evaluated based on these two indicators.

[0094] Exemplarily, the ratio between the experience quality gain of the video to be played and the bandwidth consumption of the video to be played can be used as the playback efficiency of the video to be played.

[0095] This embodiment reflects the balance between limited bandwidth budget and experience quality, that is, the higher the experience quality gain and the lower the corresponding bandwidth consumption, the higher the playback efficiency of the video to be played.

[0096] In this embodiment, after obtaining the playback efficiency of each video to be played, the bandwidth allocation priority of the corresponding video to be played is determined based on the playback efficiency. The higher the bandwidth allocation priority of the video to be played, the higher the priority of the bandwidth allocation of the high-definition mode bandwidth is allocated.

[0097] Exemplarily, the videos to be played may be sorted from high to low according to the playback efficiency of each video to be played, so as to obtain the bandwidth allocation priority of each video to be played.

[0098] In this embodiment, the total bandwidth budget is generally given as a monthly budget, that is, the bandwidth budget is determined at the beginning of the month. A greedy algorithm can be used to allocate the total bandwidth budget.

[0099] In another optional implementation, a first weight corresponding to the experience quality gain of the video to be played and a second weight corresponding to the bandwidth consumption of the video to be played may be set respectively; based on the first weight and the second weight, a comprehensive score corresponding to each of the videos to be played is calculated; according to the comprehensive scores of the videos to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of each video to be played; and bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

[0100] It should be noted that the present embodiment does not limit specific values ​​of the first weight and the second weight, and their specific values ​​can be set according to actual test results or previous experience.

[0101] Exemplarily, the product of the first weight and the experience quality gain of the video to be played is added to the product of the second weight and the bandwidth consumption of the video to be played to obtain a comprehensive score corresponding to the video to be played.

[0102] Exemplarily, for the convenience of calculation, the experience quality gain of the video to be played and the bandwidth consumption of the video to be played may be normalized and converted in advance, and then the comprehensive score corresponding to the video to be played may be calculated.

[0103] In the above embodiment, the preset high-definition bandwidth may also be allocated one by one from high to low according to the bandwidth allocation priority of each video to be played, starting from the video to be played corresponding to the highest priority, until the total bandwidth budget is allocated.

[0104] In one possible scenario, assuming that after the total bandwidth budget is allocated to the preset high-definition bandwidth for the first four videos to be played, the remaining bandwidth is insufficient to allocate the preset high-definition bandwidth to the fifth video to be played, then the fifth video to be played and subsequent videos to be played are all played in the preset low-definition bandwidth mode.

[0105] Exemplarily, before executing step 404, the method may further include: obtaining bandwidth consumption of each video to be played, wherein:

[0106] The method of obtaining the bandwidth consumption of each video to be played includes any of the following methods:

[0107] Method 1: determining the bandwidth required for each video to be played when it is played in a preset high-definition bandwidth mode, and using the bandwidth required for playing in the preset high-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0108] Method 2: Determine the bandwidth required for each video to be played when it is played in a preset low-definition bandwidth mode, and use the bandwidth required for playing in the preset low-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0109] Method 3: The number of times each video to be played is estimated based on the historical playback data of each video to be played, and the bandwidth consumption of the corresponding video to be played is determined based on the estimated number of times each video to be played.

[0110] The estimated number of video playbacks is obtained based on the time series prediction of historical data. For example, a past total period is divided into several sub-periods, and the time and number of video playbacks in each sub-period are determined. Based on the number and time of video playbacks in this total period, the number of video playbacks in the next total period (or each sub-period in the next total period) is estimated.

[0111] It should be noted that the bandwidth consumption in this embodiment is not limited to the above three definitions, and can also be a weighted processing of any two or more of the above three methods to obtain the bandwidth consumption of multiple data.

[0112] For example, Figure 5 As shown, the above step 403 may include the following steps:

[0113] Step 4031, extract and fuse the user features and video features to obtain weighted features.

[0114] In this embodiment, historical user data and video data of each video to be played can be collected, wherein the historical user data can be the characteristics of users who watched the video in the past preset time period; the video data can be the video content, the number of times the video was played in the past preset time period, and the expected number of times the video was played in the future preset time period, etc.

[0115] It should be noted that the present embodiment does not limit the specific types and contents of the user history data and video data, which can be flexibly set according to different video services.

[0116] Exemplarily, vector extraction is performed on the user features and the video features to obtain a user feature vector and a video feature vector; collaborative attention fusion processing is performed on the user feature vector and the video feature vector to obtain the attention weights of the user features to the video features, and the attention weights of the video features to the user features; the video features are weightedly aggregated using the attention weights of the user features to the video features to obtain user-perceived video features; the user features are weightedly aggregated using the attention weights of the video features to the user features to obtain video-perceived user features.

[0117] Step 4032: extract and concatenate the weighted features to obtain a vector for representing the experience quality gain.

[0118] Exemplarily, vector extraction is performed on the user-perceived video features and the video-perceived user features to obtain a user-perceived video feature vector and a video-perceived user feature vector; the user-perceived video feature vector and the video-perceived user feature vector are concatenated to obtain a vector for characterizing the experience quality gain.

[0119] Step 4033: Determine the experience quality gain of each video to be played according to the vector used to characterize the experience quality gain.

[0120] In this embodiment, the vector eigenvalue corresponding to the vector used to characterize the experience quality gain can be calculated, and the experience quality gain of each video to be played can be characterized by the vector eigenvalue, so that the experience quality gain of each video to be played can be quantified.

[0121] Exemplarily, in order to facilitate calculation, the vector eigenvalues ​​corresponding to the vector used to characterize the experience quality gain can also be normalized, so that the experience quality gain of each video to be played is more intuitive, and it is also convenient to subsequently determine the bandwidth allocation priority based on the experience quality gain of each video to be played.

[0122] The specific implementation process of steps 4031 to 4033 in this embodiment can be implemented based on a pre-established target model, wherein the target model is established based on a causal inference mechanism and an attention mechanism.

[0123] For ease of understanding, the idea of ​​establishing the target model in this embodiment is briefly described. The objective fact based on which the target model is established in this embodiment is that when a user is watching a video, it is determined whether to allocate a preset high-definition bandwidth mode or a preset low-definition bandwidth mode to the video. Before allocating these two modes, the following two aspects are mainly considered:

[0124] 1) The difference in video quality at different bitrates. For example, for different videos, the image quality changes at 1080p and 720p resolutions vary due to factors such as video content or video complexity, and the impact on user experience is also different.

[0125] 2) Users also have different requirements for image quality when playing videos. For example, young users hope to enjoy higher definition when watching photography videos, while older users may not have very high requirements for clarity when watching political videos.

[0126] For example, based on the above two points, an AB experiment can be conducted (randomly sample the evaluation objects into two groups, one is the control group and the other is the experimental group; impose strategic intervention on the experimental group and compare the effects between the two groups; because the control group and the experimental group are randomly divided, the experimental objects in the two groups can be considered to be homogeneous, so the effect of the experiment can be considered to be the effect brought about by the strategy), where the A experimental group plays in high definition and the B control group plays in low definition. Collect consumption data in both low-definition and high-definition modes (including interactive indicators such as playback time, likes, reposts and comments, and multi-dimensional user experience data such as e-commerce and commercial revenue). And based on business understanding (convert interactive indicators such as playback time, likes, reposts and comments into revenue and normalize them into money), perform a comprehensive weighted calculation of QoE indicators.

[0127] Furthermore, based on the above description, the design ideas of the target model can be established.

[0128] Exemplarily, first collect video features, such as video content, video complexity, and the diff value (difference) of video quality in high and low definition; and collect user features, such as user basic identity portrait, user historical consumption data, etc. Then, based on the model idea of ​​causal inference, model the random experiment. Specifically, the attention mechanism (which enables the neural network to focus on its input or feature subset and select specific input) is used for video features and user features to perform feature crossover, capturing the QoE changes brought about by the two strategies of high-definition mode and low-definition mode when different users watch different videos.

[0129] For example, Figure 6 The target model processing is shown in Figure 6 It can be seen that there are first two input branches. The first input branch: user features are sequentially passed through multiple multilayer neural networks (Multilayer Perceptron, MLP) to obtain user feature vectors; the second input branch: video features are sequentially passed through multiple MLPs to obtain video feature vectors. The user feature vector and the video feature vector are input into the collaborative attention mechanism network (Co Attention) for weighted processing. The results of the weighted processing are output by the first output branch and the second output branch respectively, that is, the first output branch outputs video-perceived user features, and the second output branch outputs user-perceived video features. Furthermore, the video-perceived user features are sequentially passed through multiple MLPs to obtain video-perceived user feature vectors, and the user-perceived video features are sequentially passed through multiple MLPs to obtain user-perceived video feature vectors. Finally, the video-perceived user feature vector and the user-perceived video feature vector are spliced ​​to obtain the QoE grading result.

[0130] Exemplarily, the input user features and video features are first extracted and fused to obtain weighted features; then the weighted features are extracted and concatenated to obtain a vector for characterizing the experience quality gain; finally, based on the vector for characterizing the experience quality gain, the experience quality gain of each video to be played is determined.

[0131] In this embodiment, user features are features that represent user interests and behaviors, and video features are features that describe video content and attributes. User features and video features are extracted to obtain corresponding user feature vectors and video feature vectors. Secondly, a collaborative attention mechanism is used, and the attention weights of user features to video features and the attention weights of video features to user features are calculated. This process allows the module to dynamically determine the bidirectional importance relationship between user features and video features. After calculating the bidirectional attention weights, the features are mutually enhanced. For example, attention weighted aggregation is used to weight the user features using the attention weights of video features to user features, which will produce a "video-aware" user feature representation. Similarly, for video features, the video features are weightedly aggregated using the attention weights of user features to video features. This will produce a "user-aware" video feature representation. Finally, feature vectors are extracted for the "video-aware" user features and the "user-aware" video features, and the two extracted feature vectors are spliced ​​to obtain a vector for characterizing the experience quality gain.

[0132] The above-mentioned method for allocating bandwidth for video playback receives multiple video playback requests sent by the user end, and determines multiple videos to be played corresponding to the multiple video playback requests; thereby, multiple video playback requests can be processed to improve processing efficiency. By respectively determining the user characteristics and video characteristics corresponding to each of the videos to be played, the video characteristics include video complexity; thereby, the influence of the user and the video itself on the experience quality gain can be comprehensively considered, providing a basis for the subsequent bandwidth allocation, and balancing the user experience and the consumption of bandwidth resources as much as possible. By determining the experience quality gain of each of the videos to be played according to the user characteristics and the video characteristics; thereby, the user experience and resource efficiency can be quantified through the experience quality gain. By allocating bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played. Thus, within the limited bandwidth budget, resource efficiency and user experience can be effectively balanced, so that the decision on bandwidth allocation serves the experience quality gain well.

[0133] In a second exemplary embodiment, if Figure 7 As shown, a method for allocating video playback bandwidth is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 701 to 705. Among them:

[0134] Step 701: receiving multiple video play requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video play requests.

[0135] Step 702: Determine the user features and video features corresponding to each video to be played.

[0136] Step 703: Determine the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0137] Step 704: Allocate bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0138] In this embodiment, the specific implementation process and technical effects of steps 701 to 704 are shown in Figure 4 The descriptions of steps 401 to 404 are not repeated here.

[0139] Step 705: If the video playback using the preset high-definition bandwidth ends, the preset high-definition bandwidth is allocated to the video played using the preset low-definition bandwidth according to the released bandwidth.

[0140] In one possible case, assuming that after allocating the preset high-definition bandwidth of the total bandwidth budget to the first M videos to be played, the remaining bandwidth is insufficient to allocate the preset high-definition bandwidth to the M+1th video to be played, the preset low-definition bandwidth is first allocated to the M+1th video to be played, and if any of the first M videos to be played is finished, the released bandwidth is allocated to the M+1th video to be played, so that the M+1th video to be played is switched from the preset low-definition mode to the preset high-definition mode. Wherein, M is a natural number greater than 1.

[0141] In this embodiment, the playback mode can be switched dynamically in real time according to the playback status of each video to be played, so that the video originally played in the preset low-definition bandwidth mode can be switched to the preset high-definition bandwidth mode within a limited bandwidth budget, thereby optimizing bandwidth resources, combining it with the actual needs of video services, and improving the user experience quality.

[0142] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0143] For example, Fig. 8AAs shown, a method for allocating video playback bandwidth is provided and applied to the interaction scenario between the server and the terminal. Figure 1 Take the server in as an example to illustrate, assuming that there are N terminals, and the server allocates video playback bandwidth to the N terminals. First, multiple terminals send video playback requests to the server within a preset time period (for example, a 10-second time segment is set). The server analyzes the multiple video playback requests received to determine the user characteristics and video characteristics corresponding to these requests. Exemplarily, the user identity can be determined based on the IP address, terminal device number, etc. The user characteristics are determined based on the user's historical consumption data and user portraits, and the video characteristics are determined based on the content, complexity, and AB experiment results of the video to be played. Furthermore, the server determines the experience quality gain of each video to be played and the bandwidth consumption of each video to be played based on the user characteristics and video characteristics. The server allocates bandwidth for the videos to be played based on the experience quality gain and bandwidth consumption of each video to be played.

[0144] In an optional implementation, each terminal plays the video according to the allocated bandwidth. Since the lengths of the videos to be played are different, it may happen that the video playback of some terminals is completed while the video playback of other terminals is not yet completed. In response to this situation, the server will reallocate the bandwidth to the remaining terminals that have not yet completed the playback according to the bandwidth released by the terminals. In this way, the limited bandwidth resources can be fully utilized to improve the user experience.

[0145] For example, Figure 8B As shown, another method for allocating video playback bandwidth is provided and applied to the interactive scene between the server and the terminal. Figure 1 The server in the example is used to illustrate the Fig. 8A Unlike the embodiment shown, there is only one terminal in this embodiment. For example, the terminal sends a video playback request to the server in a preset time period (for example, a 10-second time segment is set). The server analyzes the multiple video playback requests received to determine the user characteristics and video characteristics corresponding to these requests. Exemplarily, the user identity can be determined based on the IP address, terminal device number, etc. The user characteristics are determined based on the user's historical consumption data and user portraits, and the video characteristics are determined based on the content, complexity, and AB experiment results of the video to be played. Furthermore, the server determines the experience quality gain of each video to be played and the bandwidth consumption of each video to be played based on the user characteristics and video characteristics. The server allocates bandwidth for the videos to be played based on the experience quality gain and bandwidth consumption of each video to be played.

[0146] Similarly, in an optional implementation, the terminal plays in split-screen mode, and the bandwidth of each played video is the bandwidth allocated by the server. Since the video lengths of the videos to be played are different, it may happen that the video playback of some split-screen windows is completed, while the video playback of other split-screen windows has not yet been completed. In response to this situation, the server will reallocate the bandwidth of the remaining videos that have not yet been completed according to the bandwidth released by the terminal. In this way, limited bandwidth resources can be fully utilized to improve the user experience.

[0147] In a third exemplary embodiment, Fig. 9 As shown, a method for allocating video playback bandwidth is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 901 to 909. Among them:

[0148] Step 901: receiving multiple video play requests sent by a user terminal, and determining multiple to-be-played videos corresponding to the multiple video play requests.

[0149] Step 902: Determine the user features and video features corresponding to each video to be played.

[0150] Step 903: extract and fuse the user features and video features to obtain weighted features.

[0151] Step 904: extract and concatenate the weighted features to obtain a vector for representing the experience quality gain.

[0152] Step 905: Determine the experience quality gain of each to-be-played video according to the vector used to characterize the experience quality gain.

[0153] Step 906: Obtain the bandwidth consumption of each video to be played.

[0154] Step 907: The ratio between the experience quality gain of the video to be played and the bandwidth consumption of the video to be played is used as the playback efficiency of the video to be played.

[0155] Step 908: sort the videos to be played from high to low according to the playback efficiency of each video to be played, and obtain the bandwidth allocation priority of each video to be played.

[0156] Step 909: Allocate bandwidth to the videos to be played according to the bandwidth allocation priority of each video to be played.

[0157] In this embodiment, the concept of playback efficiency is introduced, that is, when allocating bandwidth to a video, not only the experience quality gain of the video is considered, but also the bandwidth consumption of the video to be played. For example, assuming that there is a user A, based on the user characteristics and video characteristics of user A, it can be determined that the experience quality gain of user A is very high, but the video played by A consumes a lot of bandwidth. If the server allocates bandwidth to user A's terminal first, it will affect other users such as user B, user C, user D, etc., so that the videos watched by these users are often in the preset low-definition mode, and the user experience is not good. Using the playback efficiency method to determine the bandwidth priority can well avoid this situation, so that more users can get a good experience.

[0158] In a fourth exemplary embodiment, as Fig.10 As shown, a method for allocating video playback bandwidth is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 1001 to 1010. Among them:

[0159] Step 1001: receiving multiple video play requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video play requests.

[0160] Step 1002, respectively determine the user features and video features corresponding to each video to be played.

[0161] Step 1003: extract and fuse the user features and video features to obtain weighted features.

[0162] Step 1004: extract and concatenate the weighted features to obtain a vector for representing the experience quality gain.

[0163] Step 1005 : Determine the experience quality gain of each to-be-played video according to the vector used to characterize the experience quality gain.

[0164] Step 1006, obtaining the bandwidth consumption of each video to be played.

[0165] Step 1007: Set a first weight corresponding to the experience quality gain of the video to be played and a second weight corresponding to the bandwidth consumption of the video to be played.

[0166] Step 1008: Calculate a comprehensive score corresponding to each video to be played based on the first weight and the second weight.

[0167] Step 1009: sort the videos to be played from high to low according to their comprehensive scores, and obtain the bandwidth allocation priority of each video to be played.

[0168] Step 1010: Allocate bandwidth to the videos to be played according to the bandwidth allocation priority of each video to be played.

[0169] In this embodiment, compared with Fig. 9 The method of the embodiment introduces a first weight and a second weight, so that the influence ratio of the experience quality gain of the video to be played and the bandwidth consumption of the video to be played can be more flexibly configured, so that when the bandwidth allocation priority is determined, it can be flexibly adjusted according to different application scenarios or user needs, and has better applicability.

[0170] In an exemplary embodiment, Fig.11 As shown, a method for determining a video experience quality gain is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 1101 to 1102. Among them:

[0171] Step 1101 , extracting features of each video to be played, and obtaining user features and video features corresponding to each video to be played.

[0172] In this embodiment, historical user data and video data of each video to be played can be collected, wherein the historical user data can be the characteristics of users who watched the video in the past preset time period; the video data can be the video content, the number of times the video was played in the past preset time period, and the expected number of times the video was played in the future preset time period, etc.

[0173] Exemplarily, the user characteristics include: user identity portrait, user historical consumption data; wherein, the user historical consumption data includes: user experience data in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode, and the user experience data includes: playback time, interaction indicators, any one of e-commerce and commercial revenue, or any combination of two or more; the video characteristics include: video content, video complexity, and the difference in picture quality when the video is played in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode.

[0174] In this embodiment, an AB experiment can be conducted (randomly sampling the evaluation objects into two groups, one is the control group and the other is the experimental group; applying strategic intervention to the experimental group and comparing the effects between the two groups; because the control group and the experimental group are randomly divided, the experimental objects in the two groups can be considered to be homogeneous, so the effect of the experiment can be considered to be the effect brought about by the strategy), where the A experimental group plays in high definition and the B control group plays in low definition. Consumption data in both low-definition and high-definition modes are collected (including interactive indicators such as playback time, likes, reposts, and comments, and multi-dimensional user experience data such as revenue from e-commerce and commercialization). And based on business understanding (converting interactive indicators such as playback time, likes, reposts, and comments into revenue and normalizing them into amounts), a comprehensive weighted calculation of QoE indicators is performed.

[0175] It should be noted that the picture quality resolution corresponding to the preset high-definition bandwidth mode and the picture quality resolution corresponding to the preset low-definition bandwidth mode are not limited in this embodiment. In the embodiment of the present application, it is only used to characterize two different bandwidth modes, one corresponding to high definition and the other to low definition. For example, the picture quality resolution corresponding to the preset high-definition bandwidth mode is 1080p, and the picture quality resolution corresponding to the preset low-definition bandwidth mode is 720p. For another example, the picture quality resolution corresponding to the preset high-definition bandwidth mode can be set to 2K, and the picture quality resolution corresponding to the preset low-definition bandwidth mode can be set to 1080p. The high and low in the embodiment of the present application are a relative probability, and their specific values ​​are not limited.

[0176] Step 1102: Input the user features and the video features into the target model for processing respectively, and output the experience quality gain of each video to be played through the target model.

[0177] In this embodiment, the design concept of the target model can be established based on the above AB experiment.

[0178] Exemplarily, first collect video features, such as video content, video complexity, and the diff value (difference) of video quality in high and low definition; and collect user features, such as user basic identity portrait, user historical consumption data, etc. Then, based on the model idea of ​​causal inference, model the random experiment. Specifically, the attention mechanism (which enables the neural network to focus on its input or feature subset and select specific input) is used for video features and user features to perform feature crossover, capturing the QoE changes brought about by the two strategies of high-definition mode and low-definition mode when different users watch different videos.

[0179] For example, Fig.12 As shown, the above step 1102 may include the following steps:

[0180] Step 11021, extract and fuse the input user features and video features to obtain weighted features.

[0181] In this embodiment, user features are features that represent user interests and behaviors, and video features are features that describe video content and attributes. User features and video features are extracted to obtain corresponding user feature vectors and video feature vectors. Secondly, a collaborative attention mechanism is used, and the attention weights of user features to video features, as well as the attention weights of video features to user features, are calculated. This process allows the module to dynamically determine the bidirectional importance relationship between user features and video features. After calculating the bidirectional attention weights, the features are mutually enhanced. For example, attention weighted aggregation is used to perform weighted aggregation of user features using the attention weights of video features to user features, which will produce a "video-aware" user feature representation. Similarly, for video features, the attention weights of user features to video features are used to perform weighted aggregation of video features. This will produce a "user-aware" video feature representation.

[0182] Exemplarily, vector extraction can be performed on the input user features and the video features to obtain a user feature vector and a video feature vector; collaborative attention fusion processing can be performed on the user feature vector and the video feature vector to obtain the attention weights of the user features to the video features, and the attention weights of the video features to the user features; the video features are weightedly aggregated using the attention weights of the user features to the video features to obtain user-perceived video features; the user features are weightedly aggregated using the attention weights of the video features to the user features to obtain video-perceived user features.

[0183] Step 11022: extract and concatenate the weighted features to obtain a vector for representing the experience quality gain.

[0184] In this embodiment, feature vectors are extracted from the “video-perceived” user features and the “user-perceived” video features obtained in the aforementioned step 11021, and the two extracted feature vectors are concatenated to obtain a vector for characterizing the experience quality gain.

[0185] Exemplarily, vector extraction is performed on the user-perceived video features and the video-perceived user features to obtain a user-perceived video feature vector and a video-perceived user feature vector; the user-perceived video feature vector and the video-perceived user feature vector are concatenated to obtain a vector for characterizing the experience quality gain.

[0186] Step 11023: Determine the experience quality gain of each video to be played according to the vector used to characterize the experience quality gain.

[0187] In this embodiment, eigenvalue calculation may be performed on the vector used to characterize the experience quality gain, and the experience quality gain of each to-be-played video may be characterized by the obtained vector eigenvalue.

[0188] This embodiment can find a balance between playback granularity, playback quality and user experience, and determine the experience quality gain of each video to be played through the established target model within a limited bandwidth budget. Therefore, the bandwidth allocation priority of each video to be played can be determined by means of the quantified experience quality gain and bandwidth consumption, and the optimal allocation plan for high-definition bandwidth can be obtained, so that the experience quality gain of video services can be maximized under limited bandwidth resources.

[0189] For example, Fig.13A As shown in FIG. 1 , the interaction process between the server and the user end of the first method for allocating video playback bandwidth is shown. The method is applied to Figure 1 The server and the terminal (ie, the user end) in FIG. 1 are used as an example to illustrate the method, which includes the following steps 1301 to 1306. Among them:

[0190] Step 1301: multiple clients send multiple video playback requests to a server.

[0191] Among them, user terminal 1, user terminal 2, ..., user terminal N sends a video playback request to the server within a preset time period, wherein N is a natural number greater than 2.

[0192] Step 1302: The server determines user features and video features corresponding to each video to be played.

[0193] Step 1303: The server determines the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0194] Step 1304: The server obtains the bandwidth consumption of each video to be played.

[0195] Step 1305: The server allocates bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0196] Step 1306: The client plays the video according to the allocated bandwidth.

[0197] In this embodiment, steps 1301 to 1306 combine the above Figure 4 , Figure 7 , Fig. 9 , Fig.10 For the steps in the method embodiment shown in the figure, and the specific implementation process and technical effects thereof, please refer to Figure 4 , Figure 7 , Fig. 9 , Fig.10The relevant description in the illustrated method embodiment will not be repeated here.

[0198] For example, Fig. 13B As shown in FIG. 1 , the interaction process between the server and the user end of the second method for allocating video playback bandwidth is shown. The method is applied to Figure 1 The server and terminal (ie, the user end) in FIG. 1 are used as an example to illustrate the method, which includes the following steps 1301 to 1308. Among them:

[0199] Step 1301: multiple clients send multiple video playback requests to a server.

[0200] Among them, user terminal 1, user terminal 2, ..., user terminal N sends a video playback request to the server within a preset time period, wherein N is a natural number greater than 2.

[0201] Step 1302: The server determines user features and video features corresponding to each video to be played.

[0202] Step 1303: The server determines the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0203] Step 1304: The server obtains the bandwidth consumption of each video to be played.

[0204] Step 1305: The server allocates bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0205] Step 1306: The client plays the video according to the allocated bandwidth.

[0206] In step 1307, the user terminal determines whether at least one video has been played and at least one video has not been played. If so, step 1308 is executed; if not, step 1307 is continued after waiting for a preset time period.

[0207] Step 1308: The server allocates bandwidth for the videos to be played according to the experience quality gains and bandwidth consumption of the remaining videos to be played.

[0208] In this embodiment, compared with Fig.13AThe method shown introduces a dynamic discrimination mechanism, that is, the user end will determine whether there is at least one video that has been played and at least one video that has not been played. If so, an indication message can be sent to the server (of course, the server can also automatically sense that the user end has released the bandwidth). At this time, the server allocates bandwidth for the remaining videos to be played based on the experience quality gain and bandwidth consumption of each remaining video to be played. Therefore, the released bandwidth can be used to improve the clarity of the remaining at least one video to be played, thereby further improving the efficiency of bandwidth resource utilization.

[0209] For example, Fig. 13C As shown in FIG. 1 , the interaction process between the server and the user end of the third method for allocating video playback bandwidth is shown. The method is applied to Figure 1 The server and the terminal (ie, the user end) in FIG. 1 are used as an example to illustrate the method, which includes the following steps 1301 to 1306. Among them:

[0210] Step 1301: The client sends multiple video playback requests to the server.

[0211] Among them, the user terminal has the function of playing two or more videos at the same time (see Figure 3 interface shown).

[0212] Step 1302: The server determines user features and video features corresponding to each video to be played.

[0213] Step 1303: The server determines the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0214] Step 1304: The server obtains the bandwidth consumption of each video to be played.

[0215] Step 1305: The server allocates bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0216] Step 1306: The client plays the video according to the allocated bandwidth.

[0217] In this embodiment, compared with Fig.13A The main difference between the methods shown is that multiple video playback requests are sent for the same user terminal.

[0218] For example, Fig.13D As shown in FIG. 4 , the interaction process between the server and the user end of the fourth method for allocating video playback bandwidth is shown. The method is applied to Figure 1 The server and the terminal (ie, the user end) in FIG. 1 are used as an example to illustrate the method, which includes the following steps 1301 to 1308. Among them:

[0219] Step 1301: The client sends multiple video playback requests to the server.

[0220] Among them, the user terminal has the function of playing two or more videos at the same time (see Figure 3 interface shown).

[0221] Step 1302: The server determines user features and video features corresponding to each video to be played.

[0222] Step 1303: The server determines the experience quality gain of each video to be played according to the user characteristics and the video characteristics.

[0223] Step 1304: The server obtains the bandwidth consumption of each video to be played.

[0224] Step 1305: The server allocates bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each video to be played.

[0225] Step 1306: The client plays the video according to the allocated bandwidth.

[0226] In step 1307, the user terminal determines whether at least one video has been played and at least one video has not been played. If so, step 1308 is executed; if not, step 1307 is continued after waiting for a preset time period.

[0227] Step 1308: The server allocates bandwidth for the videos to be played according to the experience quality gains and bandwidth consumption of the remaining videos to be played.

[0228] In this embodiment, compared with Fig. 13B The method shown is applicable to the case where a single client sends multiple video playback requests to the server at the same time.

[0229] It should be noted that in Fig.13A , Fig. 13B There may also be a user end using Fig. 13C , Fig.13D The implementation principle is similar and will not be described here.

[0230] The above embodiment combines the above Figure 4 , Figure 7 , Fig. 9 , Fig.10In each step of the method embodiment shown, the user terminal sends multiple video playback requests to the server. The server determines the user characteristics and video characteristics corresponding to each video to be played, and after obtaining the bandwidth consumption of each video to be played, allocates the bandwidth of the video to be played according to the experience quality gain and bandwidth consumption of each video to be played. The user terminal plays the video according to the allocated bandwidth. Thus, within the limited bandwidth budget, it is possible to effectively balance resource efficiency and user experience, so that the decision of bandwidth allocation serves the experience quality gain well. When the video playback using the preset high-definition bandwidth ends, the preset high-definition bandwidth is allocated to the video played using the preset low-definition bandwidth according to the released bandwidth. Thus, according to the playback status of each video to be played, the playback mode can be switched in real time and dynamically, so that the video originally played in the preset low-definition bandwidth mode can be switched to the preset high-definition bandwidth mode under the limited bandwidth budget, so as to optimize the bandwidth resources, combine it with the needs of the actual video service, and improve the user experience quality.

[0231] Based on the same inventive concept, the embodiment of the present application also provides a video playback device for implementing the video playback method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more video playback device embodiments provided below can refer to the limitations of the video playback method above, and will not be repeated here.

[0232] In an exemplary embodiment, Fig.14 As shown, a device for allocating video playback bandwidth is provided, including: a receiving module 1401, a feature determination module 1402, an experience quality gain determination module 1403 and a bandwidth allocation module 1404, wherein:

[0233] The receiving module 1401 is used to receive multiple video playback requests sent by the user terminal, and determine multiple videos to be played corresponding to the multiple video playback requests;

[0234] A feature determination module 1402 is used to respectively determine user features and video features corresponding to each of the videos to be played, wherein the video features include video complexity;

[0235] The experience quality gain determination module 1403 is used to determine the experience quality gain of each of the videos to be played according to the user characteristics and the video characteristics;

[0236] The bandwidth allocation module 1404 is used to allocate bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played.

[0237] Exemplarily, the experience quality gain determination module 1403 is specifically used to: extract and fuse the user features and the video features to obtain weighted features; extract and splice the weighted features to obtain a vector for characterizing the experience quality gain; and determine the experience quality gain of each video to be played based on the vector for characterizing the experience quality gain.

[0238] Exemplarily, the user characteristics include: user identity portrait, user historical consumption data; wherein, the user historical consumption data includes: user experience data in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode, and the user experience data includes: playback time, interaction indicators, any one of e-commerce and commercial revenue, or any combination of two or more; the video characteristics include: video content, video complexity, and the difference in picture quality when the video is played in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode.

[0239] Exemplarily, the extracting and fusing the user features and the video features to obtain weighted features includes: performing vector extraction on the user features and the video features to obtain a user feature vector and a video feature vector; performing collaborative attention fusion processing on the user feature vector and the video feature vector to obtain an attention weight of the user feature on the video feature, and an attention weight of the video feature on the user feature; performing weighted aggregation on the video features through the attention weight of the user features on the video features to obtain user-perceived video features; performing weighted aggregation on the user features through the attention weight of the video features on the user features to obtain video-perceived user features.

[0240] Exemplarily, the extracting and splicing the weighted features to obtain a vector for characterizing the experience quality gain includes: extracting vectors of the user-perceived video features and the video-perceived user features to obtain a user-perceived video feature vector and a video-perceived user feature vector; and splicing the user-perceived video feature vector and the video-perceived user feature vector to obtain a vector for characterizing the experience quality gain.

[0241] Exemplarily, the above device may further include a bandwidth consumption determination module 1405, wherein the bandwidth consumption determination module 1405 is configured to obtain the bandwidth consumption of each to-be-played video in any of the following ways:

[0242] Determine the bandwidth required for each video to be played when it is played in a preset high-definition bandwidth mode, and use the bandwidth required for playing in the preset high-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0243] Determine the bandwidth required for each video to be played when it is played in a preset low-definition bandwidth mode, and use the bandwidth required for playing in the preset low-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played;

[0244] The number of times each video to be played is estimated according to the historical playback data of each video to be played, and the bandwidth consumption of the corresponding video to be played is determined according to the estimated number of times each video to be played.

[0245] Exemplarily, the bandwidth allocation module 1404 is specifically used to: allocate preset high-definition bandwidth one by one from high to low according to the bandwidth allocation priority of each video to be played, starting from the video to be played corresponding to the highest priority according to the total bandwidth budget, until the total bandwidth budget is allocated.

[0246] In an optional implementation, the bandwidth allocation module 1404 is further configured to allocate a preset high-definition bandwidth to a video played using a preset low-definition bandwidth according to the released bandwidth when the video playing using the preset high-definition bandwidth ends.

[0247] Each module in the above video playback device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0248] In an exemplary embodiment, Fig.15 As shown, a device for determining a video experience quality gain is provided, comprising: a feature extraction module 1501 and a processing module 1502, wherein:

[0249] The feature extraction module 1501 is used to extract features of each video to be played, and obtain user features and video features corresponding to each video to be played;

[0250] The processing module 1502 is used to input the user features and the video features into the target model for processing respectively, and output the experience quality gain of each video to be played through the target model, wherein the target model is established based on a causal inference mechanism and an attention mechanism.

[0251] Exemplarily, the user characteristics include: user identity portrait, user historical consumption data; wherein, the user historical consumption data includes: user experience data in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode, and the user experience data includes: playback time, interaction indicators, any one of e-commerce and commercial revenue, or any combination of two or more; the video characteristics include: video content, video complexity, and the difference in picture quality when the video is played in a preset high-definition bandwidth mode and a preset low-definition bandwidth mode.

[0252] Exemplarily, the processing module 1502 is specifically used to: extract and fuse the input user features and video features to obtain weighted features; extract and splice the weighted features to obtain a vector for characterizing the experience quality gain; and determine the experience quality gain of each video to be played based on the vector for characterizing the experience quality gain.

[0253] Exemplarily, the extraction and fusion processing of the input user features and video features to obtain weighted features includes: performing vector extraction on the input user features and the video features to obtain user feature vectors and video feature vectors; performing collaborative attention fusion processing on the user feature vectors and the video feature vectors to obtain the attention weights of user features to video features, and the attention weights of video features to user features; performing weighted aggregation of the video features through the attention weights of the user features to the video features to obtain user-perceived video features; performing weighted aggregation of the user features through the attention weights of the video features to the user features to obtain video-perceived user features.

[0254] Exemplarily, the weighted features are extracted and concatenated to obtain a vector for characterizing the experience quality gain, including: extracting vectors of the user-perceived video features and the video-perceived user features to obtain a user-perceived video feature vector and a video-perceived user feature vector; and concatenating the user-perceived video feature vector and the video-perceived user feature vector to obtain a vector for characterizing the experience quality gain.

[0255] Each module in the above-mentioned apparatus for determining the video experience quality gain may be implemented in whole or in part by software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor may call and execute operations corresponding to each module.

[0256] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.16As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a video playback method or a method for determining the quality gain of video experience is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0257] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0258] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0259] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0260] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0261] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0262] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0263] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0264] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for allocating video playback bandwidth, characterized in that: The method comprises: Receiving multiple video playback requests sent by a user terminal, and determining multiple videos to be played corresponding to the multiple video playback requests; Respectively determining user features and video features corresponding to each of the videos to be played, wherein the video features include video complexity; Determining, according to the user characteristics and the video characteristics, a quality of experience gain of each of the videos to be played; Bandwidth allocation for the videos to be played is performed according to the experience quality gain and bandwidth consumption of each of the videos to be played.

2. The method according to claim 1, characterized in that The determining, according to the user characteristics and the video characteristics, the experience quality gain of each of the to-be-played videos includes: Extracting and fusing the user features and the video features to obtain weighted features; Extracting and concatenating the weighted features to obtain a vector for representing the experience quality gain; The experience quality gain of each to-be-played video is determined according to the vector used to characterize the experience quality gain.

3. The method according to claim 2, characterized in that The extracting and fusing the user features and the video features to obtain weighted features includes: Performing vector extraction on the user features and the video features to obtain a user feature vector and a video feature vector; Performing collaborative attention fusion processing on the user feature vector and the video feature vector to obtain an attention weight of the user feature to the video feature, and an attention weight of the video feature to the user feature; By using the attention weight of the user feature on the video feature, the video feature is weightedly aggregated to obtain the video feature perceived by the user; The user features are weightedly aggregated by the attention weights of the video features to the user features to obtain the video-aware user features.

4. The method according to claim 3, characterized in that The extracting and concatenating the weighted features to obtain a vector for characterizing the experience quality gain includes: Performing vector extraction on the video features perceived by the user and the user features perceived by the video to obtain a user-perceived video feature vector and a video-perceived user feature vector; The user-perceived video feature vector and the video-perceived user feature vector are concatenated to obtain a vector for representing the experience quality gain.

5. The method according to any one of claims 1 to 4, characterized in that: Before allocating bandwidth for the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played, the method further includes: obtaining bandwidth consumption of each of the videos to be played, wherein: The method of obtaining the bandwidth consumption of each video to be played includes any of the following methods: Determine the bandwidth required for each video to be played when it is played in a preset high-definition bandwidth mode, and use the bandwidth required for playing in the preset high-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played; Determine the bandwidth required for each video to be played when it is played in a preset low-definition bandwidth mode, and use the bandwidth required for playing in the preset low-definition bandwidth mode as the bandwidth consumption of the corresponding video to be played; The number of times each video to be played is estimated according to the historical playback data of each video to be played, and the bandwidth consumption of the corresponding video to be played is determined according to the estimated number of times each video to be played.

6. The method according to any one of claims 1 to 4, characterized in that: The allocating bandwidth of the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played includes: Taking the ratio between the experience quality gain of the video to be played and the bandwidth consumption of the video to be played as the playback efficiency of the video to be played; According to the playback efficiency of each video to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of each video to be played; Bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

7. The method according to claim 6, characterized in that The allocating bandwidth to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played includes: According to the total bandwidth budget, starting from the to-be-played video corresponding to the highest priority, the preset high-definition bandwidth is allocated one by one from high to low according to the bandwidth allocation priority of each to-be-played video, until the total bandwidth budget is allocated.

8. The method according to any one of claims 1 to 4, characterized in that: The allocating bandwidth of the videos to be played according to the experience quality gain and bandwidth consumption of each of the videos to be played includes: respectively setting a first weight corresponding to the experience quality gain of the video to be played and a second weight corresponding to the bandwidth consumption of the video to be played; Calculating a comprehensive score corresponding to each of the to-be-played videos based on the first weight and the second weight; According to the comprehensive scores of the videos to be played, the videos to be played are sorted from high to low to obtain the bandwidth allocation priority of the videos to be played; Bandwidth is allocated to the videos to be played according to the total bandwidth budget and the bandwidth allocation priority of each video to be played.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images