Distribution Bitrate Determination Device and Its Program

Abstract

To provide a delivery bit rate determination device that determines a delivery bit rate at which sensory quality is suppressed from decreasing.SOLUTION: A delivery bit rate determination device 3 comprises: a reproduction state gathering part 30 which gathers reproduction state information from a terminal device 2; a mathematical modeling part 31a which generates, based upon the reproduction state information, a mathematical model including a variable of which of a predetermined delivery bit rate and a candidates for a standby time is selected; a model variable calculation part 31b which calculates a solution for the variable so that a value of the mathematical model meets a condition of a predetermined threshold; a delivery indication generation part 31c which generates, as delivery indication information, a delivery bit rate of a next segment for the terminal device 2 and the standby time, the delivery bit rate being specified with the solution calculated by the model variable calculation part 31b; and a delivery indication notification part 32 which notifies the terminal device 2 of the delivery indication information.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a distribution bitrate determination device and a program thereof for determining a bitrate when streaming-distributing content.

Background Art

[0002] In recent years, adaptive streaming of videos on the Internet has mainly used adaptive streaming delivered by a general-purpose web server using HTTP (Hypertext Transfer Protocol). Specifically, protocols such as MPEG-DASH (MPEG Dynamic Adaptive Streaming over HTTP) and HLS (HTTP Live Streaming) are widely used. For example, in MPEG-DASH (see Non-Patent Document 1), video streams encoded with multiple qualities (resolution, bitrate, etc.) are prepared on a distribution server as segments each divided into several seconds to about several tens of seconds, and a manifest file describing the attributes and URLs (Uniform Resource Locators) of these video contents. Then, the terminal device determines the distribution bitrate of the segment to be received next at the start of playback and during playback from the candidates described in the manifest file in consideration of the screen size, network bandwidth, etc. Then, the terminal device sequentially receives the determined segments from the distribution server and plays them back by connecting them into one video content.

[0003] In recent years, in adaptive streaming, a method of determining a distribution bitrate that provides the highest possible perceived quality using model predictive control is known (see Non-Patent Document 2). In this method, video quality, variation in video quality, and playback stop time are used as indices of perceived quality. Note that in order to determine the distribution bitrate, it is necessary to complete the determination process within the segment length. Performing a process of determining a distribution bitrate that strictly optimizes the perceived quality within the segment length on a general terminal device involves an enormous amount of calculation and is often difficult.

[0004] Therefore, Non-Patent Document 2 proposes a method of approximately expressing the buffer occupancy of the terminal device and the reception bandwidth as discrete values with a predetermined quantization width in advance, and preparing a list of optimal distribution bitrates calculated for all combinations of possible values of the buffer occupancy and the bandwidth. By providing this list to the terminal device before the start of video playback, the terminal device only needs to select the optimal distribution bitrate corresponding to the combination of the buffer occupancy and the bandwidth during video playback from the list, reducing the processing load required for the determination process of the distribution bitrate.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the conventional method of discretizing the buffer occupancy and the bandwidth with a predetermined quantization width, a quantization error, which is the difference from the original value, occurs. Therefore, the desired perceived quality cannot be presented, and it may not be possible to determine the optimal distribution bitrate.

[0007] In addition, in the conventional method, if the quantization width is set small, the quantization error becomes small, and there is a high possibility of presenting the desired tactile quality. However, as the combination of buffer occupancy and bandwidth increases, problems may occur such as it becoming difficult in terms of time to generate a list of delivery bitrates to be provided to a terminal device before the start of video playback for a plurality of terminal devices, and the data size of the list becoming large and difficult to store in the memory of a general terminal device.

[0008] The present invention has been made in view of such problems of the prior art, and an object thereof is to provide a delivery bitrate determination device and a program thereof that can determine a delivery bitrate that suppresses a decrease in tactile quality due to quantization error.

Means for Solving the Problems

[0009] In order to solve the above problems, a delivery bitrate determination device according to the present invention is a delivery bitrate determination device that determines the delivery bitrate of a segment for a terminal device that receives a content segmented into a predetermined length for each segment, and includes a playback status collection unit, a mathematical modeling unit, a model variable calculation unit, a delivery instruction generation unit, and a delivery instruction notification unit.

[0010] In such a configuration, the delivery bitrate determination device collects, as playback status information, from the terminal device, the delivery bitrate at which the immediately preceding segment was received, the buffer occupancy of the playback buffer when starting to receive the next segment, and a bandwidth prediction value for a predetermined prediction interval, by the playback status collection unit.

[0011] Then, the delivery bitrate determination device generates, by the mathematical modeling unit, a mathematical model having as a variable which one to select from a predetermined delivery bitrate and candidates for waiting time based on the playback status information. This mathematical model can be expressed in a quadratic binary variable optimization expression or an Ising model expression.

[0012] Furthermore, the distribution bitrate determination device calculates the solution of the variables of the mathematical model by the model variable calculation unit so that the value of the mathematical model satisfies a predetermined threshold condition. This model variable calculation unit obtains the solution of the variables by performing a calculation for solving the optimization problem of the mathematical model. Thereby, the model variable calculation unit can obtain the solution of the variables of the mathematical model based on model predictive control.

[0013] Then, the distribution bitrate determination device generates, by the distribution instruction generation unit, the distribution bitrate of the next segment for the terminal device and the waiting time after receiving the next segment, which are specified by the solution of the variables calculated by the model variable calculation unit, as distribution instruction information. Then, the distribution bitrate determination device notifies, by the distribution instruction notification unit, the distribution instruction information generated by the distribution instruction generation unit to the terminal device.

[0014] Thereby, the terminal device repeatedly transmits the reproduction status information to the distribution bitrate determination device and receives the distribution instruction information, and repeatedly performs the operation of receiving the segment at the instructed distribution bitrate and waiting for the instructed waiting time, so that it becomes possible to reproduce the content considering the quality.

[0015] Note that the distribution bitrate determination device can operate a computer with a distribution bitrate determination program for causing the computer to function as each of the above-described functional units.

Advantages of the Invention

[0016] According to the present invention, without distributing a list to the terminal device as in the prior art, it is possible to suppress the load on the terminal device and suppress the deterioration of quality due to quantization error.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0018] [Overall Configuration of a Distribution System Including a Distribution Bitrate Determination Device] First, with reference to FIG. 1, the overall configuration of a distribution system including a distribution bitrate determination device according to an embodiment of the present invention will be described.

[0019] The distribution system S distributes content from the distribution server 1 to the terminal device 2 segment by segment. As shown in FIG. 1, the distribution system S includes a distribution server 1, a terminal device 2, a distribution bitrate determination device 3, and a computer 4.

[0020] The distribution server 1 stores content and distributes segments corresponding to requests from the terminal device 2 via the network N. The content is video, audio, or both, or data including them. A segment is data obtained by dividing the content into a predetermined length (segment length). Here, it is assumed that the distribution server 1 segments and distributes video content.

[0021] The terminal device 2 requests content from the distribution server 1 segment by segment and receives it via the network N. Here, one terminal device 2 is illustrated as an example, but the number of devices is not limited to one. While receiving a segment, the terminal device 2 notifies the distribution bitrate determination device 3 of the playback status information via the network N. Then, the terminal device 2 receives the segment based on the distribution instruction information notified from the distribution bitrate determination device 3. This terminal device 2 is not particularly limited as long as it is a terminal having a communication function. For example, it can be a smartphone terminal, a tablet terminal, a personal computer, a wearable terminal, etc.

[0022] The playback status information that the terminal device 2 notifies the distribution bitrate determination device 3 is information indicating the distribution bitrate, the buffer occupancy, and the bandwidth prediction value. The distribution bitrate of the playback status information is the bitrate when the terminal device 2 received the previous segment from the distribution server 1. The buffer occupancy is the occupancy of the playback buffer when the terminal device 2 starts receiving the next segment. Here, the buffer occupancy is the time length (seconds) for playing back the segment. The bandwidth prediction value is the predicted value of the bandwidth when the terminal device 2 receives segments after the next segment. Note that the prediction of the bandwidth in the terminal device 2 may use general methods such as a method based on moving average or a method based on time series analysis, so the description is omitted here.

[0023] The distribution instruction information that the terminal device 2 is notified from the distribution bitrate determination device 3 is information indicating the distribution bitrate and the waiting time. The distribution bitrate of the distribution instruction information is the bitrate when the terminal device 2 receives the next segment from the distribution server 1. The waiting time is the time that the terminal device 2 waits after receiving a segment until it starts receiving the next segment.

[0024] The distribution bitrate determination device 3 determines the distribution bitrate of segments for the terminal device 2 that receives the content in segments divided into a predetermined length. Based on the playback status information notified from the terminal device 2 via the network N, the distribution bitrate determination device 3 determines the distribution bitrate and the waiting time, and notifies the terminal device 2 as distribution instruction information. The distribution bitrate determination device 3 formulates a mathematical model of a combinatorial optimization problem with a variable of which candidate of the predetermined distribution bitrate and waiting time to select, and obtains the solution of the mathematical model by the computer 4. Here, the distribution bitrate determination device 3 constructs the mathematical model H in the QUBO (Quadratic unconstrained binary optimization) expression shown in the following formula (1).

[0025]

Equation

[0026] Here, z(i) and z(j) are the i-th and j-th elements of the vector z, which are binary variables taking "0" or "1". Σ i , Σ j is an operator that takes the sum of all elements of the vector z. The symbol T in the upper right of the vector represents the transpose of the vector. Q(i,j) is the element in the i-th row and j-th column of the parameter matrix Q, and becomes the coefficient of the product z(i)z(j) of the binary variables. The distribution bitrate determination device 3 calculates the set (z(i), z(j)) of z that minimizes the mathematical model H of formula (1) using the computer 4. Note that the mathematical model H may use an Ising model expression equivalent to it in addition to the QUBO expression. This distribution bitrate determination device 3 will be described in detail later.

[0027] The computer 4 obtains the solution of the mathematical model generated by the distribution bitrate determination device 3 by calculation. The computer 4 may be any computer as long as it can calculate the solution of the mathematical model generated by the distribution bitrate determination device 3. For example, the computer 4 is a computer such as an annealing machine that solves a QUBO or an Ising model. Note that the computer 4 does not necessarily have an independent configuration and may be provided inside the distribution bitrate determination device 3.

[0028] [Regarding the variation in the buffer occupancy of the terminal device] Here, with reference to FIG. 2, the variation in the buffer occupancy in the terminal device 2 due to the distribution instruction information (distribution bitrate and waiting time) instructed by the distribution bitrate determination device 3 will be described. Assume that the video content is composed of K segments with a segment length of L. Among the K segments, let the distribution bitrate of segment k be r(k), the buffer occupancy at the start of download (reception) be b(k), the predicted bandwidth value during download be c(k), and the waiting time be d(k).

[0029] As shown in FIG. 2, when the terminal device 2 starts downloading segment k at the playback time t(k), the download time is L(r(k) / c(k)), which depends on the distribution bitrate r(k) of the distribution instruction information and the predicted bandwidth value c(k). Thereafter, the terminal device 2 waits for the waiting time d(k) of the distribution instruction information until the download of the next segment k + 1. During this period, since playback continues, the buffer occupancy decreases by t(k + 1) - t(k), and the segment length L of the downloaded segment k is added. Note that if L(r(k) / c(k)) exceeds the buffer occupancy b(k), playback interruption (rebuffering) occurs. That is, in the terminal device 2, the state equation of the buffer occupancy in a situation where the playback buffer does not run out and playback stop does not occur is represented by the following equation (2).

[0030] [Equation]

[0031] Based on the state equation shown in this formula (2), the distribution bitrate determination device 3 determines the distribution bitrate so as not to deplete the playback buffer of the terminal device 2 and to improve the perceived quality. Hereinafter, the configuration and operation of the distribution bitrate determination device according to the embodiment of the present invention will be described.

[0032] [Configuration of Distribution Bitrate Determination Device] With reference to FIG. 3 (and appropriately refer to FIG. 1), the configuration of the distribution bitrate determination device 3 according to the embodiment of the present invention will be described. The distribution bitrate determination device 3 includes a storage unit ME, a communication unit NC, and a control unit CL.

[0033] The storage unit ME is a storage medium such as a ROM (Read Only Memory) that stores the distribution bitrate determination program P. The distribution bitrate determination program P is expanded and operates in a RAM (Random Access Memory) (not shown) to realize the functions of the control unit CL.

[0034] The communication unit NC communicates with the terminal device 2 via the network N. The communication unit NC outputs the playback status information received from the terminal device 2 to the control unit CL. Also, the communication unit NC transmits the distribution instruction information input from the control unit CL to the terminal device 2.

[0035] The control unit CL is a functional block that functions by the operation of the distribution bitrate determination program P. The control unit CL includes, as functional blocks, a playback status collection unit 30, a distribution instruction optimization unit 31, and a distribution instruction notification unit 32.

[0036] The playback status collection unit 30 collects, as playback status information, from the terminal device, the distribution bitrate at which the immediately preceding segment was received, the buffer occupancy of the playback buffer when starting to receive the next segment, and the predicted bandwidth value for a predetermined prediction interval. The playback status collection unit 30 collects the playback status information from the terminal device 2 and outputs it to the distribution instruction optimization unit 31.

[0037] The distribution instruction optimization unit 31 constructs a mathematical model from the playback status information collected by the playback status collection unit 30 and obtains its solution, thereby determining distribution instruction information indicating the distribution bitrate and waiting time information.

[0038] The distribution instruction optimization unit 31 includes a mathematical modeling unit 31a, a model variable calculation unit 31b, and a distribution instruction generation unit 31c.

[0039] Based on the playback status information collected by the playback status collection unit 30, the mathematical modeling unit 31a generates a mathematical model of the perceived quality with a variable indicating which one to select from the predetermined distribution bitrate and waiting time candidates.

[0040] Here, with reference to FIG. 4 (refer to FIGS. 1 and 3 as appropriate), the configuration of the mathematical modeling unit 31a will be described in more detail. As shown in FIG. 4, the mathematical modeling unit 31a includes a variable constraint model formulation unit 310, a quality evaluation model formulation unit 311, a quality variation evaluation model formulation unit 312, and a prediction control evaluation model formulation unit 313.

[0041] The variable constraint model formulation unit 310 generates a constraint function for constraining variables. The variable constraint model formulation unit 310 formulates a constraint function as a mathematical model that constrains selecting one from the distribution bitrate candidates. As described above, assume that the video content is composed of K segments with a segment length of L. Let the next segment to be received be k. Let r(k) be the delivery bitrate determined for segment k, and r(k) is to be selected from the R delivery bitrates {r1, r2, …, rR} prepared in the delivery server 1. Also, let np be the number of prediction intervals of the bandwidth when determining the delivery bitrate r(k) for segment k. That is, for segment k, r(k), r(k + 1), …, r(k + np - 1) are the variables to be determined.

[0042] When selecting a candidate r ∈ {r1, r2, …, rR} for the delivery bitrate for segment k, let the binary variable that takes “1” when selected and “0” when not selected be x(k, r). In this case, the delivery bitrate r(k) selected for segment k is represented by the following formula (3).

[0043]

Equation

[0044] Here, Σ r is an operator that takes the sum of all candidates for the delivery bitrate. Here, the variable constraint model formulation unit 310 generates a variable constraint model H1 shown in the following formula (4) as a constraint function that restricts selecting one from the candidates for the delivery bitrate.

[0045]

Equation

[0046] The variable constraint model formulation unit 310 outputs the variable constraint model H1 shown in formula (4) to the model variable calculation unit 31b.

[0047] Next, the waiting time, which is one of the variables, will be described below. Let d(k) be the waiting time from the completion of receiving segment k to the start of receiving segment k + 1. As candidates for the waiting time, here, 2 of the Δ intervals nd discrete values {0, Δ, 2Δ, …, (2 nd - 1)Δ} are assumed. Here, nd is the number of binary variables that are auxiliary variables for expressing the waiting time. When selecting a candidate d ∈ {0, 1, 2, …, nd - 1} of the auxiliary variable for the waiting time for segment k, let it be a binary variable y(k, d) that takes “1” when selected and “0” when not selected. In this case, the waiting time d(k) selected for segment k is expressed by the following formula (5).

[0048]

Equation

[0049] Here, Σ d is an operator that takes the sum of all candidates of the auxiliary variable for the waiting time. As described above, by assuming equally spaced discrete values as candidates for the waiting time, for any y(k, d), d(k) expressed by formula (5) becomes one of {0, Δ, 2Δ, …, (2 nd - 1)Δ}, so that only one waiting time is selected without adding new constraints. Also, since 2 nd candidates for the waiting time can be expressed by nd binary variables, the number of binary variables to be determined can be reduced.

[0050] Note that in video streaming distribution, in many cases, it is not difficult to assume the waiting time as an equally spaced discrete set, but any value can also be assumed. In that case, the variable constraint model formulation unit 310 may also generate a model that constrains to select one from a plurality of candidates for the waiting time, similar to the constraint on the distribution bit rate of formula (4) for the waiting time. In this case, for segment k, in addition to the distribution bit rates r(k), r(k + 1), …, r(k + np - 1), the waiting times d(k), d(k + 1), …, d(k + np - 1) become variables to be determined.

[0051] The quality evaluation model formulation unit 311 generates an evaluation function (first evaluation function) for evaluating the quality of segments (video quality). The quality evaluation model formulation unit 311 formulates the evaluation function as a mathematical model aiming to maximize the quality. Here, let the quality of segment k when the delivery bitrate is r be q(k, r). q(k, r) may be calculated using, for example, the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), Video Multimethod Assessment Fusion (VMAF), etc., or the delivery bitrate collected by the playback status collection unit 30 may be used as q(k, r)=r(k) as it is. When using PSNR, SSIM, VMAF, etc., the playback status collection unit 30 measures PSNR, SSIM, VMAF, etc. as the collection result and outputs them to the quality evaluation model formulation unit 311. The quality evaluation model formulation unit 311 generates a quality evaluation model H2 shown in the following formula (6) as an evaluation function for evaluating the quality.

[0052]

Equation

[0053] The quality evaluation model H2 shown in this formula (6) is the sum of the qualities q(k, r) of segments k, k + 1,..., k + np - 1 at the delivery bitrate r with a negative sign attached. Thus, the quality evaluation model H2 functions to maximize the quality by being minimized. Note that the following formula (7) may also be used as the quality evaluation model H2.

[0054]

Equation

[0055] The quality evaluation model H2 shown in this equation (7) is the sum of the squares of the differences between the quality q(k, r) of segments k, k + 1, …, k + np - 1 and the quality q(k, max{r1, r2, …, rR}) of the maximum delivery bitrate. By doing this, the quality evaluation model H2 functions to minimize the difference and bring the quality as close as possible to the quality of the maximum delivery bitrate. The quality evaluation model formulation unit 311 outputs the quality evaluation model H2 shown in equation (6) or equation (7) to the model variable calculation unit 31b.

[0056] The quality fluctuation evaluation model formulation unit 312 generates an evaluation function (second evaluation function) for evaluating the fluctuation of the quality (video quality). The quality fluctuation evaluation model formulation unit 312 formulates the evaluation function as a mathematical model aimed at minimizing the fluctuation of the quality. Here, let the quality of segment k when the delivery bitrate is r be q(k, r). Note that for the quality, similar to the quality evaluation model formulation unit 311, PSNR, SSIM, VMAF, etc. may be used. The quality fluctuation evaluation model formulation unit 312 generates the quality fluctuation evaluation model H3 shown in the following equation (8) as an evaluation function for evaluating the fluctuation of the quality.

[0057]

Equation

[0058] In this equation (8), when the first segment k = 1, there is no previous segment, so it is divided into cases where k = 1 and k ≠ 1. The quality fluctuation evaluation model H3 shown in this equation (8) is the sum of the squares of the differences in quality between a certain segment and its previous segment. By doing this, the quality fluctuation evaluation model H3 functions to minimize the difference in quality fluctuations by being minimized. The quality fluctuation evaluation model formulation unit 312 outputs the quality fluctuation evaluation model H3 shown in equation (8) to the model variable calculation unit 31b.

[0059] The prediction control evaluation model formulation unit 313 generates an evaluation function (third evaluation function) for evaluating the difference between the buffer occupancy and a predetermined target value. The prediction control evaluation model formulation unit 313 generates an evaluation function for bringing the buffer occupancy as close as possible to the target value from the buffer occupancy b(k) at the start of receiving the next segment k among the reproduction status information collected by the reproduction status collection unit 30, and the predicted bandwidth values c(k), c(k + 1), …, c(k + np - 1).

[0060] The evaluation function generated by the prediction control evaluation model formulation unit 313 will be described by the following mathematical formulae. In the terminal device 2, the state equation of the buffer occupancy in a situation where the buffer occupancy does not run out and reproduction stop does not occur is represented by the above formula (2). Also, here, B, RD, Φ, Ξ are represented by the following formulae (9) to (12).

[0061]

Equation

[0062] However, the symbol T in the upper right of the vectors B, RD, Ξ represents transpose. At this time, the following relationship of formula (13) holds.

[0063]

Equation

[0064] However, 1vec represents a vector in which all elements of np rows and 1 column are "1". Here, assuming that the target value of the buffer occupancy is bref, the sum h4 of the squares of the differences between the buffer occupancies of segments k + 1, k + 2, …, k + np and the target value is represented by the following formula (14).

[0065]

Equation

[0066] This equation (14) serves as an evaluation function for assessing the buffer occupancy. However, since it is not a binary variable, it cannot be converted into a QUBO representation. Therefore, as shown below, the predictive control evaluation model formulation unit 313 generates a predictive control evaluation model H4 with h4 in equation (14) being a binary variable. Here, X(k), Y(k), Rvec, and Dvec are represented by the following equations (15) to (18).

[0067]

Number

[0068] Furthermore, Z and RDdiag are represented by the following equations (19) and (20).

[0069]

Number

[0070] However, diag(A1, A2, …) represents a block diagonal matrix formed by arranging the matrices A1, A2, … on the diagonal part. The RDdiag in this equation (20) is a block diagonal matrix formed by arranging the matrices Rvec and Dvec on the diagonal part, and further, a block diagonal matrix formed by arranging np diagonal parts. At this time, RD in equation (10) is represented by the following equation (21) using Z in equation (19) and RDdiag in equation (20).

[0071]

Number

[0072] By substituting equation (21) into the right side of equation (14), the predictive control evaluation model H4 of binary variables shown in the following equation (22) is obtained.

[0073]

Number

[0074] The predictive control evaluation model formulation unit 313 generates the predictive control evaluation model H4 of Expression (22) and outputs it to the model variable calculation unit 31b.

[0075] As described above, since the variable constraint model H1 (the above Expression (4)), the quality evaluation model H2 (the above Expression (6) or the above Expression (7)), the quality variation evaluation model H3 (the above Expression (8)), and the predictive control evaluation model H4 (the above Expression (22)) generated by the mathematical modeling unit 31a are expressed as quadratic expressions for the binary variables x(k,r) and y(k,d), they can be converted into the QUBO expression represented by the above Expression (1) by formula transformation.

[0076] It is obvious from, for example, <https: / / blueqat.readthedocs.io / ja / latest / ising.html> etc. that if the degree for binary variables is quadratic or less, it is possible to convert them into the quadratic form of QUBO (the degree is all quadratic or less). That is, for a binary variable z that takes "0" or "1", since z and the square of z are equivalent (both "0" and "1" remain unchanged when squared), the first-order term and the second-order term can be equivalently transformed. Also, since the constant term does not affect the evaluation function no matter how the binary variables are selected, it may be ignored. Thus, if the degree for binary variables is quadratic or less, it can be converted into the QUBO format. Returning to FIG. 3, the description of the configuration of the distribution bitrate determination device 3 will be continued.

[0077] The model variable calculation unit 31b calculates, by the computer 4, the solution of the variable that presents the optimal perceived quality from the mathematical model generated by the mathematical modeling unit 31a. The model variable calculation unit 31b outputs the overall function obtained by adding the mathematical models generated by the mathematical modeling unit 31a to the computer 4 and acquires the solution of the variable that is the calculation result. Here, the model variable calculation unit 31b calculates the overall function H of the following equation (23) obtained by adding the mathematical models (variable constraint model H1, quality evaluation model H2, quality variation evaluation model H3, predictive control evaluation model H4) using the balance coefficients λ1, λ2, λ3, λ4 ALL and instructs the computer 4 to calculate the solutions of the variables x(k,r) and y(k,d) that minimize it.

[0078]

Number

[0079] However, the balance coefficients λ1, λ2, λ3, λ4 are the weights of the mathematical models H1, H2, H3, H4, and the greater their values, the greater the weight placed on the mathematical model related to those values. The model variable calculation unit 31b sets initial values for the balance coefficients λ1, λ2, λ3, λ4, causes the computer 4 to execute the calculation, and obtains the solutions of the variables x(k,r) and y(k,d).

[0080] The model variable calculation unit 31b calculates the mathematical models H1, H2, H3, H4 using the obtained variable solutions, and adjusts the balance coefficients so that the values (calculation results) of the respective mathematical models are below a predetermined threshold. Note that since the variable constraint model H1 needs to strictly satisfy the constraints, its threshold is set to "0". The thresholds for the quality evaluation model H2, the quality variation evaluation model H3, and the predictive control evaluation model H4 can be freely set according to the desired perceived quality. And while adjusting the balance coefficients, the computer 4 is caused to ALL repeatedly perform the process of recalculating (solving) the overall function H.

[0081] That is, the model variable calculation unit 31b uses the mathematical model H1 (constraint function), the mathematical model H2 (first evaluation function), the mathematical model H3 (second evaluation function), and the mathematical model H4 (third evaluation function) as the overall function obtained by multiplying and adding the balance coefficients, and under the constraints of the mathematical model H1, repeatedly calculates to obtain the solution of the mathematical model so as to increase the quality value (H2), decrease the variation in the quality value (H3), and decrease the difference between the buffer occupancy and the target value (H4).

[0082] For example, when the value of a certain mathematical model is not less than a predetermined threshold value, by increasing the balance coefficient corresponding to this mathematical model, the weight imposed on this mathematical model is increased. As a result, the value of this mathematical model can be decreased. Note that for the recalculation of this overall function H ALL an upper limit on the number of repetitions may be set. As a result, depending on the value of the set upper limit on the number of repetitions, even when the value of the mathematical model does not become less than the threshold value, the calculation can be terminated. The model variable calculation unit 31b outputs the solutions of the variables x(k, r) and y(k, d) to the distribution instruction generation unit 31c.

[0083] The distribution instruction generation unit 31c generates distribution instruction information to be notified to the terminal device 2 based on the solutions of the variables calculated by the model variable calculation unit 31b. Here, the distribution instruction generation unit 31c determines the distribution bit rate r(k) of the next segment k to be received and the waiting time d(k) until the reception of segment k+1 starts after the reception of segment k is completed based on the solutions of the variables x(k, r) and y(k, d), and generates distribution instruction information. The distribution instruction generation unit 31c calculates the distribution bit rate r(k) and the waiting time d(k) from the variable vector Z of the above formula (19) that combines the variables x(k, r) and y(k, d) according to the following formula (24).

[0084]

Equation

[0085] However, 0mat represents a matrix in which all elements of 2 rows and R nd(np-1) columns are "0".

[0086] The distribution instruction generation unit 31c outputs the calculated distribution bit rate r(k) and the waiting time d(k) as distribution instruction information to the distribution instruction notification unit 32.

[0087] The distribution instruction notification unit 32 notifies the terminal device 2 of the distribution instruction information generated by the distribution instruction generation unit 31c via the communication unit NC.

[0088] With the configuration described above, as shown in Fig. 5(a), the distribution bitrate determination device 3 determines the distribution bitrate r(k) that can achieve the highest possible perceived quality from the distribution bitrate r(k - 1), which is the playback status information collected from the terminal device 2, and the bandwidth prediction values c(k), c(k + 1), … of the prediction interval np. high , r mid , r low )

[0089] Also, as shown in Fig. 5(b), the distribution bitrate determination device 3 determines the waiting time d(k) such that the playback buffer of the terminal device 2 does not run out and the perceived quality is as high as possible, based on the preset buffer occupancy bref. With the configuration as described above, the distribution bitrate determination device 3 can determine the distribution bitrate and waiting time that achieve the desired perceived quality and notify the terminal device 2.

[0090] [Operation of Distribution Bitrate Determination Device] Next, with reference to Fig. 6 (appropriately refer to Figs. 1, 3, and 4), the operation of the distribution bitrate determination device 3 according to the embodiment of the present invention will be described.

[0091] In step S1, the playback status collection unit 30 collects, as playback status information, the distribution bitrate r(k - 1) when the previous segment was received, the buffer occupancy b(k) when the reception of the next segment starts, and the bandwidth prediction values c(k), c(k + 1), …, c(k + np - 1) of the prediction interval from the terminal device 2 via the communication unit NC.

[0092] In step S2, the mathematical modeling unit 31a of the distribution instruction optimization unit 31 generates a mathematical model of the perceived quality, using as a variable which one to select from candidate values of a predetermined distribution bit rate and a waiting time based on the playback status information collected in step S1. At this time, the mathematical modeling unit 31a generates a variable constraint model H1 (the above formula (4)) by the variable constraint model formulation unit 310. Further, the mathematical modeling unit 31a generates a quality evaluation model H2 (the above formula (6) or formula (7)) by the quality evaluation model formulation unit 311. Further, the mathematical modeling unit 31a generates a quality variation evaluation model H3 (the above formula (8)) by the quality variation evaluation model formulation unit 312. Further, the mathematical modeling unit 31a generates a predictive control evaluation model H4 (the above formula (22)) by the predictive control evaluation model formulation unit 313.

[0093] In step S3, the model variable calculation unit 31b of the distribution instruction optimization unit 31 determines initial values of balance coefficients λ1, λ2, λ3, λ4 when calculating the overall function H of the above formula (23) from the mathematical models (variable constraint model H1, quality evaluation model H2, quality variation evaluation model H3, predictive control evaluation model H4) generated in step S2. ALL

[0094] In step S4, the model variable calculation unit 31b calculates, using the computer 4, solutions of variables x(k, r) and y(k, d) that minimize the overall function H of the above formula (23), using the mathematical models H1, H2, H3, H4 generated in step S3 and the balance coefficients set in step S3 or adjusted in step S7 described later. ALL

[0095] In step S5, the model variable calculation unit 31b determines whether or not the number of calculation times for calculating the variable that minimizes the overall function H ALL is equal to or less than a predetermined threshold value. ​​Here, when the number of calculations is equal to or less than a predetermined threshold (Yes in step S5), the distribution bitrate determination device 3 proceeds to step S6. On the other hand, when the number of calculations is not equal to or less than the predetermined threshold (No in step S5), the distribution bitrate determination device 3 proceeds to step S8.

[0096] In step S6, the model variable calculation unit 31b calculates the mathematical models H1, H2, H3, and H4 using the solutions of the variables obtained in step S4, and determines whether the values are equal to or less than a predetermined threshold. Since the variable constraint model H1 needs to strictly satisfy the constraints, the threshold is set to "0". Note that the thresholds for the quality evaluation model H2, the quality variation evaluation model H3, and the predictive control evaluation model H4 may be freely set according to the desired perceived quality. Here, when the values of the mathematical models H1, H2, H3, and H4 are equal to or less than the predetermined threshold (Yes in step S6), the distribution bitrate determination device 3 proceeds to step S8. On the other hand, when the values of the mathematical models H1, H2, H3, and H4 are not equal to or less than the predetermined threshold (No in step S6), the distribution bitrate determination device 3 proceeds to step S7.

[0097] In step S7, the model variable calculation unit 31b increases the balance coefficient by adding a predetermined positive real number to the corresponding balance coefficient for the mathematical model whose value of the mathematical models H1, H2, H3, and H4 is equal to or less than the predetermined threshold. Alternatively, the model variable calculation unit 31b decreases the balance coefficient by adding a predetermined negative real number to the corresponding balance coefficient for the mathematical model that is not less than the threshold. For example, when only the value of the variable constraint model H1 is not less than the threshold, the model variable calculation unit 31b either increases only the balance coefficient λ1 or decreases the balance coefficients λ2, λ3, and λ4. As a result, relatively, the proportion of the variable constraint model H1 increases. Then, the distribution bitrate determination device 3 returns to step S4.

[0098] In step S8, the distribution instruction generation unit 31c of the distribution instruction optimization unit 31 determines the distribution bitrate and the waiting time using the solutions of the variables x(k,r) and y(k,d) obtained in step S4, and generates distribution instruction information (see the above formula (24)).

[0099] In step S9, the distribution instruction notification unit 32 notifies the terminal device 2 of the distribution instruction information generated in step S8 via the communication unit NC. Through the above operations, the distribution bitrate determination device 3 can determine the distribution bitrate and the waiting time that result in the desired perceived quality and notify the terminal device 2.

Explanation of Signs

[0100] 1 Distribution server 2 Terminal device 3 Distribution bitrate determination device 30 Playback status collection unit 31 Distribution instruction optimization unit 31a Mathematical modeling unit 310 Variable constraint model formulation unit 311 Quality evaluation model formulation unit 312 Quality variation evaluation model formulation unit 313 Prediction control evaluation model formulation unit 31b Model variable calculation unit 31c Distribution instruction generation unit 32 Distribution instruction notification unit 4 Computer S Distribution system

Claims

1. A delivery bitrate determination device that determines the delivery bitrate of segments for a terminal device that receives segments obtained by dividing content into segments of a predetermined length, comprising: a playback status collection unit that collects, as playback status information, the delivery bitrate at which the immediately preceding segment was received, the buffer occupancy of the playback buffer when starting to receive the next segment, and a bandwidth prediction value for a predetermined prediction interval, from the terminal device; a mathematical modeling unit that generates a mathematical model having, as a variable, which one to select from a predetermined delivery bitrate and candidate waiting times, based on the playback status information; a model variable calculation unit that calculates a solution of the variable of the mathematical model so that the value of the mathematical model satisfies a predetermined threshold condition; a delivery instruction generation unit that generates, as delivery instruction information, the delivery bitrate of the next segment for the terminal device and the waiting time after receiving the next segment, specified by the solution of the variable calculated by the model variable calculation unit; a delivery instruction notification unit that notifies the terminal device of the delivery instruction information generated by the delivery instruction generation unit; A delivery bitrate determination device, characterized by comprising the above.

2. The mathematical modeling unit includes: a variable constraint model formulation unit that generates a constraint function that has a constraint of selecting one from candidate predetermined delivery bitrates; a quality evaluation model formulation unit that generates a first evaluation function for evaluating a value indicating a predetermined quality of the segment; a quality variation evaluation model formulation unit that generates a second evaluation function for evaluating the variation in the value of the quality for each segment; a prediction control evaluation model formulation unit that generates a third evaluation function for evaluating the difference between the buffer occupancy and a predetermined target value, and The model variable calculation unit uses the constraint function, the first evaluation function, the second evaluation function, and the third evaluation function as a whole function obtained by multiplying by a balance coefficient and adding them as the mathematical model, and repeats calculations to obtain a solution of the mathematical model so as to increase the value of the quality, reduce the variation in the value of the quality, and reduce the difference between the buffer occupancy and the target value under the constraint. The delivery bitrate determination device according to claim 1.

3. The delivery bitrate determination device according to claim 2, wherein the value of the quality is the bitrate, PSNR, SSIM, or VMAF of the segment.

4. The distribution bit rate determination device according to any one of claims 1 to 3, wherein the mathematical model is a model represented by a quadratic form binary variable optimization expression without constraints or an Ising model expression.

5. A distribution bit rate determination program for causing a computer to function as the distribution bit rate determination device according to any one of claims 1 to 4.

Images