Bit rate determination device

Abstract

This bit rate determination device comprises: a quality-of-experience calculation unit configured to calculate an estimated value of quality of experience when a terminal receives a video at a certain bit rate; an available time calculation unit configured to calculate, on the basis of the remaining battery capacity of the terminal, the available time of the terminal when the terminal receives a video at a certain bit rate and transmits a video at a certain bit rate; and a bit rate determination unit configured to determine a pair of a future transmission bit rate and a future reception bit rate for each of a plurality of terminals that transmit and receive videos in a many-to-many manner, while setting a goal that the estimated value calculated by the quality-of-experience calculation unit for the reception bit rate is greater than or equal to a target value and the available time calculated by the available time calculation unit for the pair of the reception bit rate and the transmission bit rate is longer than or equal to a target available time. This makes it possible to implement bit rate control that takes into account power consumption and quality of experience.

Description

Bit rate determination device

[0001] The present invention relates to a bit rate determination device.

[0002] With the increase in remote work, web conferencing services are becoming widely used on a variety of devices. To satisfy the intent of web conferencing service providers to reduce operational costs (network costs for distribution) and the intent of service users to use services with sufficient quality, a "video bitrate control technology based on target quality" that takes into account quality and data transfer volume has been proposed (Non-Patent Document 1). In Non-Patent Document 1, the user's quality of experience is estimated from the audio and video information received by each user. Based on this information, a future video bitrate required to achieve a pre-set target quality of experience is predicted, and a bitrate that minimizes the amount of data transferred while still achieving the target quality of experience is selected, and the bitrate for the transmitted video is instructed to each user.

[0003] In addition, in recent years, there has been an increase in situations where people are using mobile devices to participate in web conferences while on the go. When using a mobile device while on the go, it is important to be aware of the remaining battery level and power consumption so that the battery does not run out during the web conference and you do not have to leave the conference unintentionally.

[0004] Therefore, a method has been proposed for determining the number of transmission bit rates (number of types of transmission bit rates) and bit rate values, taking into consideration the power consumption of the transmitting terminal and the network bandwidth utilization efficiency on the receiving side (Non-Patent Document 2).

[0005] M. Yokota and K. Yamagish, "Quality-based video bitrate control for WebRTC-based tele-conference services", Electronic imaging 2022, IQSP-333, Jan. 2022R. Geng and H. Li, "Bitrate Adaptive Scheme in Real-time Video Conference System", 2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC)2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC), Beijing, China, 2021, pp. 44-47

[0006] The technology described in Non-Patent Document 1 takes into consideration quality and bit rate, but does not take into consideration remaining battery power or power consumption, and therefore cannot respond to user intent regarding remaining battery power or power consumption.

[0007] Furthermore, the technology described in Non-Patent Document 2 only considers bandwidth utilization efficiency, which is the difference between the receiver's bit rate and the reception bandwidth, and power consumption, but does not consider the quality of experience or the bit rate itself that affects it. Furthermore, it does not specify how to weight the terms for bandwidth utilization efficiency and power consumption.

[0008] For reference, the formula for power consumption expressed as a function of bit rate described in Non-Patent Document 2 is shown below. The first term takes into account the difference between the rate and the receiving bandwidth (=bandwidth utilization efficiency), and the second term takes into account the increase in the transmission rate (=power consumption). In this paper, power consumption is linearly related to the bit rate (b), so it is a simple addition.

[0009]

[0010] The following constraints are imposed on this formula:

[0011] α+β=1: The sum of the weights of the bandwidth efficiency and cost terms is constrained to be 1. n≦m: The number of transmission bit rates (n) is constrained to be smaller than the number of receivers (m). n≦N ｍａｘ : n is the maximum number that the terminal can withstand (N ｍａｘ ) Constraint b ｋ ≦B ｋ，ｌ The constraint is that the transmission rate (bk) of stream k is smaller than the corresponding receiver's bandwidth rate and reception bandwidth (Bk, l). The present invention has been made in consideration of the above points, and aims to enable bit rate control that takes into account power consumption and quality of experience.

[0012] In order to solve the above problem, the bit rate determination device has a quality of experience calculation unit configured to calculate an estimated value of quality of experience when a terminal receives video at a certain bit rate; an available time calculation unit configured to calculate the available time of a terminal when the terminal receives video at a certain bit rate and transmits video at a certain bit rate based on the remaining battery capacity of the terminal; and a bit rate determination unit configured to determine a pair of future transmission bit rate and reception bit rate for each of a plurality of terminals that transmit and receive video to and from each other in a many-to-many manner, with the goal that the estimated value calculated by the quality of experience calculation unit for the reception bit rate is equal to or greater than a target value and the available time calculated by the available time calculation unit for the pair of reception bit rate and transmission bit rate is equal to or greater than a target available time.

[0013] It is possible to control the bit rate while taking into consideration power consumption and quality of experience.

[0014] FIG. 1 is a diagram showing a data flow relating to calculation of available time. FIG. 2 is a diagram showing an example of the configuration of the system in an embodiment of the present invention. FIG. 3 is a diagram showing an example of the hardware configuration of the control server 10 in an embodiment of the present invention. FIG. 4 is a flowchart for explaining an example of the processing procedure executed by the quality of experience calculation unit 12. FIG. 5 is a diagram showing an example of the data flow relating to the bit rate determination unit 14. FIG. 6 is a flowchart for explaining a second bit rate determination procedure. FIG. 7 is a diagram for explaining a specific example of the present embodiment.

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] In a situation where there are video senders and receivers, such as in a Web conference service, power consumption generally increases as the receiving bit rate and transmitting bit rate increase.

[0017] In this embodiment, the relationship between the transmission and reception bit rate and the power consumption is modeled and handled. Furthermore, the obtained model is separated into a model showing the relationship between the reception bit rate and the power consumption and a model showing the relationship between the transmission bit rate and the power consumption.

[0018] Although details will be described later, in this embodiment, the model parameters of the above model are obtained in advance by actually measuring the power consumption of the client (terminal used by the user).

[0019] In general, one of the main factors that causes the power consumption of a client to change in response to a change in bit rate is thought to be the characteristics of the client's video codec (for example, its internal processing and internal circuitry).

[0020] Therefore, among the model parameters of the above models, model parameters that depend on the characteristics of the video codec are likely to fit into one of the expected video codec types and models if a set corresponding to each type is prepared.

[0021] Considering the above, even when measuring power consumption for each client and determining model parameters, it may be possible to use a simple determination procedure that simply involves determining which of the prepared model parameter sets best matches the measurement results.

[0022] [Modeling of the Relationship Between Transmission and Reception Bit Rates and Power Consumption] Modeling of the relationship between transmission and reception bit rates and power consumption of a client will be described. Transmission bit rate b ｕ , receiving bit rate b ｄ and power consumption p ｕｄ The relationship between these is modeled as in the following equation (1): Note that the direction in which a bit stream is sent from the client to the outside is referred to as "transmission," and the direction in which the client receives a bit stream from the outside is referred to as "reception."

[0023]

[0024] Equation (1) is a model showing the relationship between the transmission and reception bit rate and power consumption, where b ｄ is the received bit rate, b ｕ is the transmission bit rate, p ｕｄ is the power consumed by the transmission and reception. ０ ~α ３ , β ０ ~β ３ is a model parameter for each transmission and reception rate (b ｄ and b ｕ Appropriate values ​​for these model parameters are determined based on the actual measured values ​​of power consumption (for each pair of ). As an example, the actual power consumption is measured while changing the transmission and reception bit rate, and the obtained data on the transmission bit rate, reception bit rate, and power consumption are used to determine the appropriate values ​​for these model parameters using a technique such as curve fitting.

[0025] Generally, the sending and receiving bit rates of a client are controlled by a web conferencing system including the client, and are also affected by external, uncontrollable factors such as network congestion. Therefore, it is difficult to carry out the above-mentioned actual measurements while keeping one of the sending and receiving bit rates completely fixed and changing the other. Therefore, it is appropriate to calculate the model parameters based on the above formula (1), which takes into account the influence of both the sending and receiving bit rates.

[0026] However, the constant term α in equation (1) ０ and β ０ To determine this, actual measurements under conditions where only one of the receiving or transmitting bit rates changes are required. To achieve this, it is possible to select from multiple actual measurements that satisfy this condition within a specified error, or to use a method such as improving the measurement environment and eliminating external, uncontrollable factors such as those mentioned above.

[0027] Since the form of equation (1) is a sum of terms that affect the reception bit rate and the transmission bit rate, this sum can be separated into equations (2) and (3), and equation (2) can be expressed as the reception bit rate and the power consumed by reception at that rate p ｄ The model (3) shows the relationship between the transmission bit rate and the power consumed by transmission at that rate p ｕ This can be used as a model showing the relationship between

[0028]

[0029] Hereinafter, the relationship (estimation model) of equation (2) will be written as equation (4) below.

[0030]

[0031] Furthermore, the relationship (estimation model) of equation (3) is written as equation (5) below.

[0032]

[0033] [Calculation of Available Time] The available time of the client is calculated using equation (4) (model showing the relationship between the reception bit rate and the power consumption) and equation (5) (model showing the relationship between the transmission bit rate and the power consumption).

[0034] Generally, a Web conference client uses multiple transmission streams and multiple reception streams, so the power consumption P ｉ is expressed as an aggregate of the estimated power consumption values ​​for each stream as follows:

[0035]

[0036] P ｉ : Power consumption of client i p ｕ，ｎ : Power consumption of client i for transmission stream n p ｄ，ｒ : Power consumption of client i for received stream r b ｕ，ｎ : Transmission bit rate of transmission stream n of client i b ｄ，ｒ : Receiving bit rate of receiving stream r of client i N ｍａｘ : Number of transmittable bit rates R: Number of receivetable bit rates f ｕｐ (b ｕ，ｎ ): Model equation showing the relationship between transmission bit rate and power consumption f ｄｏｗｎ (b ｄ，ｒ ): A model equation showing the relationship between the reception bit rate and power consumption. The available time of client i is T ｉ Then, it is expressed by the following formula (8).

[0037]

[0038] Here, the remaining battery capacity ｉ The unit is [Wh], P ｉ If [W] is used as the unit of ｉ is expressed as [h] (hours). Although formula (8) is a simple equation obtained by dividing the capacity by the power consumption, any equation that can calculate the available time using the remaining battery capacity and power consumption may be used instead of formula (8).

[0039] As described above, the remaining battery capacity is a value related to the remaining energy stored in the client's battery, but it does not have to be a value that accurately represents the remaining energy stored in the battery. For example, the remaining battery capacity may be a value that becomes 0 when the energy stored in the battery has decreased to a point where it is not yet 0 but is so low that it is practically impossible to participate in a Web conference.

[0040] Remaining battery capacity ｉ may be obtained from information collected from client i, or may be estimated using information other than the remaining battery capacity. As an example, the current remaining battery capacity may be estimated using the average set bit rate for a past period, the decrease in the remaining battery capacity during that period, and information on the battery capacity at the start of the conference. As another example, the current remaining battery capacity may be estimated using the battery voltage or battery current. In addition, if it is difficult to collect the necessary information on the battery capacity at the start of the conference, a default value may be provided. Note that for clients not running on battery power (for example, desktop clients running on external power such as AC power, or mobile clients connected to an external power source and not running on battery power), T may be calculated without using the remaining battery capacity. ｉ may be set to a sufficiently large value.

[0041] The data flow relating to the calculation of the available time is shown in FIG.

[0042] Next, an example of a specific computer system (hereinafter referred to as "this system") that utilizes the above will be described.

[0043] [System Overview] Fig. 2 is a diagram showing an example of the configuration of the system according to an embodiment of the present invention. In Fig. 2, the system includes a conference server 30, a control server 10, and a plurality of clients 20. Each client 20 is capable of communicating with the conference server 30 and the control server 10.

[0044] The conference server 30 is one or more computers that relay the transmission and reception of video between the clients 20 .

[0045] The clients 20 are terminals used by users who participate in a web conference. In Fig. 2, the clients have a data collection unit 21 and a quality setting unit 22. Although not shown, each client 20 has a power source (battery or external power source).

[0046] The data collection unit 21 collects information such as the receiving bit rate, transmitting bit rate, frame rate, resolution, display screen size, and remaining battery capacity from the client 20, and transmits the collected information to the control server 10. Here, the information transmitted by the client 20 corresponds to the transmitted stream or the received stream and is information for each stream. The conference server 30 may also have the data collection unit 21. In this case, the collected information may be transmitted from the conference server 30 to the control server 10.

[0047] The quality setting unit 22 receives from the control server 10 the bit rate determined by the control server 10 based on information sent by the data collection unit 21, etc., and sets the bit rate used by the client 20 to the received bit rate.

[0048] The control server 10 is one or more computers having a setting unit 11 , a quality of experience calculation unit 12 , an available time calculation unit 13 , a bit rate determination unit 14 , and a bit rate instruction unit 15 .

[0049] The setting unit 11 accepts in advance settings of a target QoE, which is a target value of the quality of experience to be achieved in a Web conference, and a target available time to be achieved (a target value for the available time of the client 20). The settings may be made for each Web conference. The target QoE and the target available time may be input by participants in the Web conference via a user interface (not shown) or the like.

[0050] The quality of experience calculation unit 12 calculates an estimate of the quality of experience (hereinafter referred to as "QoE") for a certain period based on the bit rate for that period. For example, the quality of experience calculation unit 12 estimates the QoE at time t (calculates the estimated QoE) from bit rate information for a period including 30 points in time from t-29 to t. The QoE is expressed as a numerical value, and the larger the numerical value, the better the quality of experience.

[0051] The available time calculation unit 13 receives the transmission and reception bit rate and remaining battery capacity of the client 20 as input and calculates an estimated value of the available time of the client 20 .

[0052] The bit rate determination unit 14 selects a future bit rate from a preset bit rate ladder that will achieve the set target QoE for each client 20 and ensure that the available time at that time is equal to or longer than the set target available time. Regarding the achievement of the target QoE, the future QoE of multiple clients 20 participating in the web conference may be achieved collectively.

[0053] The bit rate ladder is a list of available bit rate options (candidates) that are set in advance, taking into consideration the specifications of the conference server 30 to be used, the specifications of the participating clients 20, the processing capabilities of the web conference system, and the present system.

[0054] In addition, when it is not possible to simultaneously achieve both the target QoE and the target available time, which one is to be prioritized is set in advance. In this case, the bit rate determination unit 14 selects a bit rate that will achieve the prioritized one.

[0055] The bit rate instruction unit 15 transmits the bit rate selected by the bit rate determination unit 14 to each client 20. In addition to transmitting the bit rate, the bit rate instruction unit 15 may also transmit to each client 20 setting values ​​other than the bit rate used for controlling QoE (such as display resolution and display frame rate).

[0056] 3 is a diagram showing an example of the hardware configuration of the control server 10 according to an embodiment of the present invention. The control server 10 in FIG. 3 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, all of which are interconnected via a bus B.

[0057] A program that realizes processing in the control server 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0058] When an instruction to start a program is received, the memory device 103 reads and stores the program from the auxiliary storage device 102. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the control server 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0059] The client 20 and the conference server 30 may also have the same hardware configuration as that shown in Fig. 3. Each unit of the control server 10 in Fig. 2 is realized by a process in which one or more programs installed in the control server 10 are executed by the processor 104. Each unit of the client 20 in Fig. 2 is realized by a process in which one or more programs installed in the client 20 are executed by the processor of the client 20.

[0060] Next, each unit of the control server 10 will be described in more detail.

[0061] 4 is a flowchart illustrating an example of a processing procedure executed by the quality of experience calculation unit 12. In response to a request from the bit rate determination unit 14, the quality of experience calculation unit 12 executes the following processing.

[0062] In step S101, the quality of experience calculation unit 12 receives data necessary for estimating QoE from the bit rate determination unit 14. As described above, the quality of experience calculation unit 12 calculates the QoE at (the end of) a certain period from bit rates at multiple points in time within the period. Therefore, in step S101, the bit rates at these multiple points in time are input. As will be described later, when estimating QoE at a certain point in time in the future, the bit rate at a point in time before the current point in time among the multiple points in time is a past actual value, and the bit rate at a future point in time is a value to be searched for by the bit rate determination unit 14.

[0063] Next, the quality of experience calculation unit 12 calculates the QoE based on the input data (S102).

[0064] Known techniques are used to calculate QoE. The data required to calculate QoE differs depending on the technique used. Non-Patent Document 1 is an example of this technique, which calculates QoE using a model that can calculate the QoE of a user receiving video at a bit rate from the bit rate. When using the technique in Non-Patent Document 1, information such as resolution and frame rate is required in addition to the bit rate.

[0065] [Bit Rate Determining Unit 14] FIG. 5 is a diagram showing an example of a data flow related to the bit rate determining unit 14. As shown in FIG.

[0066] The bit rate determination unit 14 determines (searches for) a bit rate for each client 20, taking into consideration the QoE and available time estimated for each of the multiple clients 20, under the conditions (constraints) of the target QoE and target available time accepted by the setting unit 11.

[0067] The bit rate determination unit 14 determines a set of transmission bit rate and reception bit rate for each client 20 with the goal that the future QoE will achieve the set target QoE (become equal to or greater than the target QoE) and that the available time at that time will be equal to or greater than the set target available time.

[0068] An example of a procedure for determining the bit rate is shown below.

[0069] <First Bit Rate Determination Procedure> It is assumed that the number of clients 20 participating in the Web conference is M. In addition, the following procedure controls the bit rate, and the bit rate in this case is selected from a predetermined bit rate ladder.

[0070] Furthermore, it is assumed that streams including images of the clients 20 (for example, images of the users of the clients 20) are transmitted and received between the clients 20 participating in the Web conference in a many-to-many manner via the network.

[0071] In this case, the bit rate determination unit 14 selects future bit rates for all clients 20 with the goal of satisfying the following two objective functions:

[0072]

[0073] Here, equation (9) is the estimated future quality of experience (QoE) of client 20i. ｉ ) is equal to or greater than the target QoE (Target QoE). ｉ ) is calculated based on the bit rates received from M-1 clients 20 excluding the client 20 i itself.

[0074] The bit rate determination unit 14 determines the QoE ｉ The quality of experience calculation unit 12 calculates the QoE based on the formula (9). ｉ Calculate the QoE in Equation (9). ｉ (b_past ｊ , b_future ｕ，ｊ ) is the future QoE of the client 20i. ｉ is the actual measurement value b_past of the reception bit rate of the client 20i from the client 20j at multiple points in time during the most recent past fixed period T1. ｊ and the reception bit rate b_future of the client 20i from the client 20j at multiple points in time during a certain period T2 in the immediate future. ｕ，ｊ The resulting calculated QoE isｉ is the QoE of the client 20i at the end of the period T1+T2. ｕ，ｊ is assumed to be constant during the period T2. ｕ，ｊ Each b_future in the set ｕ，ｊ The value of b_future is unique. ｕ，ｊ is a value to be searched for by the bit rate determination unit 14, and one of the options (candidate values) in the bit rate ladder is substituted into equation (9) as a search hypothesis.

[0075] The maximum number of bit rates that can be simultaneously transmitted by the client 20 is N ｍａｘ The client 20 can transmit the same video at different bit rates depending on the situation on the receiving side, and the number of different bit rates is the "number of bit rates that can be transmitted simultaneously." ｍａｘ may be different for each client 20. ｍａｘ is equal to or less than the number of bit rates in the bit rate ladder. For example, if the number of bit rates in the bit rate ladder is 8, then N ｍａｘ is about 3.

[0076] u is the N ｍａｘ This is a variable for identifying which of the various bit rates the client 20i has selected.

[0077] Furthermore, the QoE of the client 20 is also affected by the frame rate and resolution when displaying video on the client 20. Generally, increasing the frame rate or resolution increases the QoE, while decreasing it decreases the QoE. In calculating the QoE in equation (9), not only the bit rate but also the resolution and frame rate may be used as parameters. In other words, when the frame rate and / or resolution are used to calculate the QoE, the above b_past ｊ , b_future ｕ，ｊ In addition to the above, these values ​​may be input. As with the bit rate, the past and future frame rates and resolutions may also be specified.

[0078] Equation (10) is the available time T ｉ is the target usage time (TargetT ｉ me) or more.

[0079] The bit rate determination unit 14 determines b_future ｕ，ｊ In the process of searching for b_future ｕ，ｊ T when the candidate value of ｉ In response to this request, the available time calculation unit 13 calculates T ｉ Calculate.

[0080] As explained in the above equations (6) to (8), T ｉ is affected by the transmission and reception bit rate. Therefore, for the client 20i, T ｉ When calculating the bit rate, both the transmission and reception bit rates are required. The reception bit rate is calculated using the formula (9) b_future ｕ，ｊ (j∈M, j≠i) can be used. Meanwhile, the transmission bit rate is not shown in equation (9). However, in a Web conference where clients 20 exchange bit streams directly with each other, this transmission bit rate is equal to the reception bit rate of the other client 20 via the bit stream. In other words, b_future ｕ，ｊ can also be treated as the transmission bit rate of j. If this is taken into consideration across the entire transmission and reception bit rates of each client 20, T ｉ As shown in equation (8), it is possible to calculate T ｉ The remaining battery capacity is used to calculate ｉ Remaining battery capacity ｉ is set to the remaining battery capacity last received (latest) from the client 20i.

[0081] The bit rate determination unit 14 determines the bit rate b_future that satisfies these two objective functions. ｕ，ｊis determined from the bit rate ladder set in advance. In other words, each bit rate constituting the bit rate ladder set in advance is b_future ｕ，ｊ The bit rate ladder may be common to all clients 20, or may be set individually.

[0082] At this time, the following constraints are taken into consideration:

[0083] Constraint 1: The total value of the receiving bit rates of all clients 20 is set to b_future as small as possible. ｕ，ｊ Select .

[0084] Generally, the higher the receiving bit rate of the client 20, the better the quality. ｕ，ｊ prevents the value from growing without limit.

[0085] Constraint 2: b_future ｕ，ｊ selects from a pre-determined bitrate ladder.

[0086] Constraint 3: For each client 20j, the number of transmission bit rates is equal to the number N of simultaneously transmittable bit rates. ｍａｘ In other words, when u is assigned consecutively from 1, u is N ｍａｘ The following applies.

[0087] Any existing method can be used to determine a combination of bit rates that satisfies the objective function and constraints. For example, a brute force approach may be used, in which the quality of experience calculation unit 12 calculates a set of QoEs for possible combinations of bit rates across multiple clients 20, the available time calculation unit 13 calculates a set of available times, and the bit rate determination unit 14 determines a combination of bit rates that satisfies the objective function and constraints from among the many combinations.

[0088] The combination of bit rates determined as described above consists of the receiving bit rate for each bit stream when a client 20 participating in the conference communicates with another client 20. Since the transmission and reception of bit streams are in a one-to-one correspondence, once the reception bit rate is determined, the corresponding transmission bit rate is also determined.

[0089] The bit rate determination unit 14 outputs the combination of bit rates determined as described above to the bit rate instruction unit 15. When the frame rate and the resolution are also used in calculating the QoE, the bit rate determination unit 14 determines the combination of these and outputs it to the bit rate instruction unit 15.

[0090] The bit rate instruction unit 15 creates information for each client 20 that is suitable for setting processing in the quality setting unit 22 of the client 20 from the received combination information such as bit rate, frame rate, and resolution, and transmits it to the client 20.

[0091] Since the battery capacity of the client 20 changes from time to time, it is preferable that the bit rate determination and the instruction by the bit rate instruction unit 15 be repeated at time intervals (for example, every few seconds).

[0092] <Second Bit Rate Determination Procedure> Next, an example of a bit rate determination procedure different from the bit rate determination procedure described in <First Bit Rate Determination Procedure> will be described. In the above description, the bit rate was determined so that the QoE of each client 20 is equal to or greater than the target QoE. In contrast, in the following example, the bit rate is determined so that the total QoE of all clients 20 is equal to or greater than the target value. Note that, as with <First Bit Rate Determination Procedure>, the number of clients 20 participating in the Web conference is M, and the bit rate is selected from a predetermined bit rate ladder.

[0093] FIG. 6 is a flowchart illustrating the second bit rate determination procedure.

[0094] In step S201, the bit rate determination unit 14 obtains the estimated available time of each client 20 participating in the conference from the available time calculation unit 13. At this time, the available time calculation unit 13 calculates the available time for each client 20 based on the last (i.e., latest) remaining battery capacity and the (latest) transmission and reception bit rate received from that client 20, and outputs the calculation result to the bit rate determination unit 14 as the estimated available time.

[0095] Next, the bit rate determination unit 14 determines whether or not there is a client 20 that satisfies the condition that the estimated available time is less than the target available time (hereinafter referred to as "Condition A") (S201). Satisfying Condition A means that there is a concern that the battery may run out during the web conference.

[0096] If there is no client 20 that satisfies the condition A (No in S202), the bit rate determination unit 14 ends the process.

[0097] If a client 20 that satisfies condition A exists (Yes in S202), the bit rate determination unit 14 determines that client 20 as a target for bit rate setting (S203). The client 20 (hereinafter referred to as "client 20X") for bit rate setting is determined from among the M clients 20, so the number of clients for bit rate setting is M or less.

[0098] Next, the bit rate determination unit 14 determines a pair of transmission and reception bit rates for each client 20X such that the estimated available time is equal to or greater than the target available time (S204). This determination is performed using a model showing the relationship between transmission / reception bit rates and power consumption, and an available time calculation formula. For example, the power consumption of each client 20X is estimated for all possible pairs of transmission and reception bit rates for all clients 20X to determine the available time, and a pair of transmission and reception bit rates is determined such that all clients 20X will have the target available time or greater. Note that the search is performed in order from highest to lowest bit rate, and if a pair is found that is equal to or greater than the target available time, step S205 is performed. If the pair is not equal to or greater than the target available time, step S204 is performed for the next pair in the order.

[0099] Next, the bit rate determination unit 14 causes the quality of experience calculation unit 12 to calculate the QoE of the M clients 20 assuming that the bit rate determined in step S204 is applied to each client 20X, and obtains the sum of the QoEs of the M clients 20 (hereinafter referred to as the "total QoE") (S205). At this time, the quality of experience calculation unit 12 calculates the QoE for (the end of) period T1+T2 for each client 20 based on the actual bit rate values ​​at multiple points in time during the most recent past period T1 and the bit rates at multiple points in time during the most recent future period T2. The quality of experience calculation unit 12 uses the bit rate determined in step S204 for the bit rate for period T2 of client 20X, and uses the bit rate last received from that client 20 (i.e., the latest bit rate) for the bit rate for period T2 of clients 20 other than client 20X. That is, for the bit rate during the period T2, the bit rate determined for the client 20X in step S204 is applied (overwritten) to the latest set of bit rates for all the clients 20.

[0100] Next, the bit rate determination unit 14 calculates the QoE shortage from the target QoE and the total QoE (S206). The formula for calculating the QoE shortage is as follows:

[0101] QoE Deficiency=M×Target QoE−Total QoE (11) The target QoE is the QoE that each individual client 20 should achieve. To achieve the overall QoE across all clients 20 participating in the Web conference, the total QoE value for M clients 20 is set to be equal to or greater than M×target QoE. If this total value is less than M×target QoE, the overall QoE is not achieved.

[0102] Therefore, the bit rate determination unit 14 checks whether the overall QoE is achieved by determining whether the value of QoE deficiency exceeds 0 (S207).

[0103] If the QoE deficiency value is 0 or less (No in S207), the overall QoS is achieved by the bit rate determined in step S204, so the bit rate determination unit 14 outputs the determination results of S203 and S204 (the bit rate determined in step S204 for each client 20X) to the bit rate instruction unit 15 and terminates the processing (S208).

[0104] If the QoE deficiency value exceeds 0 (Yes in S207), the overall QoE is not achieved by the bit rate determined in step S204, and so the bit rate determination unit 14 executes the processes from step S209 onwards. The processes from S209 onwards are aimed at achieving the overall QoE by increasing the QoE of the client 20 (hereinafter referred to as "client 20Y") that was not the target for bit rate setting in S203.

[0105] In step S209, the bit rate determination unit 14 determines the client 20Y as a target for which the QoE is to be increased.

[0106] Next, the bit rate determination unit 14 determines a receiving bit rate for client 20Y (S210). Specifically, the bit rate determination unit 14 determines the receiving bit rate for client 20Y so that the QoE deficiency calculated from the total QoE when the determined receiving bit rate is applied to client 20Y is 0 or less. Note that the determination of the receiving bit rate for client 20Y also includes the determination of the transmitting bit rates for the other clients 20 that are changed in accordance with the receiving bit rate. For example, the bit rate determination unit 14 increases the QoE of client 20Y by changing (increasing) the receiving bit rate of client 20Y, thereby increasing the overall QoE (total QoE in equation (11)), so that the QoE deficiency is 0 or less. At this time, in the calculation of the QoE for each client 20 performed by the quality of experience calculation unit 12, actual values ​​are used for the bit rates at multiple points in time during period T1. For the bit rate of the period T2, the bit rate determined in step S204 is used for the client 20X, and for the client 20Y, the bit rate is searched as described above in step S210. At this time, the search must be performed taking into account the above-mentioned constraints 1 to 3 and the fact that the available time and QoE of the client 20 change due to control.

[0107] However, there is not necessarily a bit rate that makes the QoE deficiency equal to or less than 0. Therefore, the bit rate determination unit 14 branches the process depending on whether or not a solution that makes the QoE deficiency equal to or less than 0 has been obtained (S211).

[0108] If a solution is obtained (Yes in S211), the bit rate determination unit 14 outputs the determination results of steps S203, S204, S209 and S210 (the bit rates determined in step S204 or S210 for each client 20) to the bit rate instruction unit 15 (S212), and ends the processing.

[0109] If no solution is obtained (No in S211), the bit rate determination unit 14 proceeds to step S213 to perform processing based on priority conditions, since it is not possible to simultaneously achieve both the target available time and the target QoE.

[0110] In step S213, the bit rate determination unit 14 determines whether to prioritize achievement of the target QoE or achievement of the target available time based on a preset priority condition.

[0111] If priority is given to achieving the target QoE (Yes in S213), the bit rate determination unit 14 ends the process without outputting anything to the bit rate instruction unit 15. In this case, the determination results of S203, S204, S209, and S210 may be discarded.

[0112] If achieving the target available time is prioritized (No in S213), the bit rate determination unit 14 outputs the determination results of S203 and S204 (the bit rate determined for client 20X) to the bit rate instruction unit 15 (S214), and ends the processing.

[0113] In the <Second bit rate determination procedure>, an example was shown in which the bit rate was changed to control the QoE of the client 20. As explained in the <First bit rate determination procedure>, it is also possible to control the QoE by controlling the resolution and frame rate of the client 20. This is also possible in the <Second bit rate determination procedure>. Therefore, in the <Second bit rate determination procedure>, it is also possible to use the resolution and frame rate in addition to the bit rate.

[0114] In the <second bit rate determination procedure>, as in the <first bit rate determination procedure>, the bit rate instruction unit 15 transmits the received information such as the bit rate to the client 20 to be instructed. Also, it is preferable to repeat the <second bit rate determination procedure> process at time intervals.

[0115] [Specific Example] An example in which this embodiment is applied to a web conference will be described below. The aim of control according to this embodiment is to reduce the delivery rate of a target user in order to reduce the power consumption of a client 20 with limited power, when the overall QoE, i.e., the QoE of all multiple clients 20 participating in a conference, can be increased by improving the QoE of clients 20 with sufficient power, such as remaining battery capacity. This situation is explained with reference to FIG. 7 . Assume that the clients 20 of users A, C, and D are in a state of Battery: High and have sufficient power, while user B is outdoors and not in an environment where charging is possible, and his client 20 is in a state of Battery: Low and has limited power. In this case, taking into account the battery status of user B's client 20, control is performed to reduce the transmission / reception bit rate of user B's client 20, thereby reducing the quality of the transmitted / received video. While this reduces user B's QoE, the reduced bit rate reduces power consumption, thereby preventing user B's client 20 from running out of battery (interrupting conference participation). On the other hand, if control is exercised to increase the QoE (i.e., increase the bit rate) for clients 20 other than user B (for example, user A), the overall QoE is compensated for.

[0116] [Effects] According to this embodiment including the <first bit rate determination procedure> or the <second bit rate determination procedure>, it is possible to control the bit rate while taking into consideration the available time of the client 20 (i.e., power consumption and remaining battery capacity) and QoE across multiple clients 20 participating in a conference. In other words, it is possible to perform bit rate control that takes into consideration power consumption and quality of experience (bit rate control that balances the power consumption of the client 20 and the QoE of the conference participants). Furthermore, this control can prevent the battery from running out during a web conference. For example, it can prevent a mobile terminal with limited remaining battery capacity from having its participation in a web conference interrupted due to a power shortage.

[0117] In this embodiment, the control server 10 is an example of a bit rate determination device. The target QoE is an example of a first target value. The client 20X is an example of a first terminal. The target QoE×M is an example of a second target value. The client 20Y is an example of a second terminal.

[0118] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0119] REFERENCE SIGNS LIST 10 control server 11 setting unit 12 quality of experience calculation unit 13 available time calculation unit 14 bit rate determination unit 15 bit rate instruction unit 20 multiple clients 21 data collection unit 22 quality setting unit 30 conference server 100 drive device 101 recording medium 102 auxiliary storage device 103 memory device 104 processor 105 interface device B bus

Claims

1. A bitrate determination device comprising: a quality of experience calculation unit configured to calculate an estimated value of quality of experience when a terminal receives video at a certain bitrate; an available time calculation unit configured to calculate the available time of a terminal when the terminal receives video at a certain bitrate and transmits video at a certain bitrate based on the remaining battery capacity of the terminal; and a bitrate determination unit configured to determine a pair of future transmission bitrate and reception bitrate for each of multiple terminals that transmit and receive video to and from each other in a many-to-many manner, with the goal that the estimated value calculated by the quality of experience calculation unit for the reception bitrate is equal to or greater than a target value, and the available time calculated by the available time calculation unit for the pair of reception bitrate and transmission bitrate is equal to or greater than a target available time.

2. The bit rate determination device according to claim 1, characterized in that the bit rate determination unit is configured to determine, for each terminal, a set of future transmission bit rate and reception bit rate for that terminal, with the goal that the estimated value for that terminal will achieve a first target value and that the available time for that terminal will be equal to or longer than a target available time.

3. The bit rate determination device according to claim 1, characterized in that the bit rate determination unit is configured to determine, for a first terminal among the plurality of terminals for which the available time calculated by the available time calculation unit for the latest pair of transmitting bit rate and receiving bit rate is less than the target available time, a pair of transmitting bit rate and receiving bit rate such that the available time calculated by the available time calculation unit is equal to or greater than the target available time.

4. The bit rate determination device according to claim 3, characterized in that, when the transmission bit rate and reception bit rate determined for the first terminal are applied to a set of the latest pairs of transmission bit rates and reception bit rates for the multiple terminals, if the sum of the estimated values ​​calculated for the multiple terminals by the quality of experience calculation unit is not greater than or equal to a second target value, the bit rate determination unit determines the reception bit rate for a second terminal other than the first terminal so that the sum of the estimated values ​​calculated for the multiple terminals by the quality of experience calculation unit is greater than or equal to a second target value.

Images