System and method for controlling streams of many-to-many streaming service

Abstract

A system for controlling streams according to the present invention comprises: a plurality of client devices that receive, from a media server, a bitrate map including identification information for each of one or more transmission client devices, rating information about a transmission stream transmitted by each of the one or more transmission client devices, and bitrate information about each rating of transmission streams, generate, on the basis of the bitrate map, a selection table composed of combinations of a plurality of reception streams including the ratings of the transmission streams to be received from the transmission client devices and a summed bitrate obtained by summing a bitrate of each of the ratings of the transmission streams, and select the combinations of the reception streams on the basis of a frame delay time; and the media server that receives media encoded from each of the one or more transmission client devices, and transmits the media to one or more reception client devices. The client devices determine the frame delay time on the basis of transmission start time information inserted into a media frame received from the transmission client devices and an arrival time of the media frame.

Description

Stream control system and method for many-to-many streaming services

[0001] The present invention relates to a stream control system and method for controlling a reception stream and a transmission stream in a many-to-many streaming service.

[0002] As multimedia communication services have become more popular, multimedia is developing based on video communication that allows face-to-face conversations with people who are geographically distant.

[0003] Among these, large-scale video conferencing systems allow multiple users to exchange conference information, such as voice and video, over a network. Video conferencing systems utilize a streaming architecture called Selective Forwarding Unit (SFU), which consists of a media server and clients. In this SFU architecture, each client transmits streams of its video encoded at multiple bitrates to the media server, which then forwards the received video streams to other clients in a stream appropriate for each client's network conditions.

[0004] At this time, since the operation of determining the bitrate setting value in the SFU structure and the operation of selecting the stream to be transmitted have a significant impact on the service quality (delay time and video quality) experienced by users, technologies have been developed to improve this.

[0005] These technologies have configured systems for their purposes using either a control method that achieves low latency performance by setting a low fixed bitrate, or a control method that optimizes bitrate decisions and stream selection by having the media server periodically collect network bandwidth information from all users.

[0006] However, these methods have limitations in achieving both low-latency and high-quality performance in large-scale video conferencing services. While methods that use a fixed bitrate reliably achieve low-latency performance, they inevitably result in a decrease in video quality. While methods that optimize bitrate determination and stream selection achieve low-latency and high-quality performance when the number of users is small, as the number of users increases, they have limitations in quickly responding to increases in latency for each user by changing bitrate and stream selection due to the time required to collect data and solve the optimization problem known as the Knapsack problem, which requires a large amount of computation.

[0007] Therefore, a technology is required to solve such problems.

[0008] In order to solve the above-mentioned problem, the present invention provides a stream control system and method in which a receiving client device selects a combination of receiving streams including transmission stream grade information of each transmitting client device based on a frame delay time, and a media server adjusts a bit rate of a transmission stream grade of a transmitting client device based on the combination of receiving streams selected by the receiving client device.

[0009] However, the technical tasks that this embodiment seeks to accomplish are not limited to the technical tasks described above, and other technical tasks may exist.

[0010] As a technical means for solving the above-described technical problem, a stream control system according to an embodiment of the present invention comprises: a plurality of client devices for receiving, from a media server, identification information for each of at least one transmitting client device, a bitrate map including grade information of a transmission stream transmitted by each transmitting client device, and bitrate information for each grade of the transmission stream; generating a selection table composed of combinations of a plurality of receiving streams including a grade of a transmission stream to be received from each transmitting client device and a summed bitrate obtained by adding the bitrates of each grade of the transmission stream based on the bitrate map; and a media server for receiving encoded media from each of at least one transmitting client device and transmitting the media to at least one receiving client device, wherein the client device determines the frame delay time based on transmission start time information inserted into a media frame received from the transmitting client device and an arrival time of the media frame.

[0011] In addition, a stream control method according to another embodiment of the present invention comprises the steps of: (a) receiving, by a receiving client device, a bitrate map including identification information for each of at least one transmitting client device, grade information of a transmission stream transmitted by each transmitting client device, and bitrate information for each grade of the transmission stream from a media server; (b) generating, by the receiving client device, a selection table including combinations of a plurality of receiving streams including a grade of a transmission stream to be received from each transmitting client device and a summed bitrate obtained by adding the bitrates of each grade of the transmission stream based on the bitrate map; and (c) selecting, by the receiving client device, a combination of receiving streams based on a frame delay time, wherein, in the step (c), the always-on receiving client device determines the frame delay time based on transmission start time information inserted into a media frame received from the transmitting client device and an arrival time of the media frame.

[0012] According to the above-described problem solving means of the present invention, each receiving client device directly selects a receiving stream to receive media based on the frame delay time without going through a media server, so that it can immediately respond to an increase in delay time.

[0013] Additionally, the media server adjusts the bitrate of the transmission stream class based on the selected stream combination information of each client device, thereby reflecting the preferences of all client devices and allowing each client device to select a transmission stream with a higher bitrate within its own bandwidth, thereby improving video quality.

[0014] Additionally, the media server can reduce the amount of computation by only performing the task of adjusting the bitrate, so bitrate adjustments can be updated quickly.

[0015] FIG. 1 is a block diagram schematically illustrating a stream control system according to one embodiment of the present invention.

[0016] Figure 2 is an example diagram for explaining the operation of a client device.

[0017] Figure 3 is an example of a bitrate map.

[0018] Figure 4 is an example of a selection table.

[0019] Figure 5 is a block diagram schematically showing the configuration of a client device.

[0020] Figure 6 is an example diagram for explaining the operation of a media server.

[0021] Figure 7 is a flowchart for explaining a stream control method according to one embodiment of the present invention.

[0022] Hereinafter, the present invention will be described in detail with reference to the attached drawings. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in the present specification, and the technical ideas disclosed in the present specification are not limited by the attached drawings. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and the size, shape, and shape of each component shown in the drawings can be variously modified. The same / similar drawing reference numerals are assigned to the same / similar parts throughout the specification.

[0023] The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. Furthermore, in describing the embodiments disclosed herein, detailed descriptions of related known technologies have been omitted if they are deemed to obscure the gist of the embodiments disclosed herein.

[0024] Throughout the specification, when a part is said to be "connected (connected, in contact with, or coupled)" to another part, this includes not only cases where it is "directly connected (connected, in contact with, or coupled)" but also cases where it is "indirectly connected (connected, in contact with, or coupled)" with another member in between. Furthermore, when a part is said to "include (have or provide)" a certain component, this does not mean that it excludes other components, but rather that it may "include (have or provide)" other components, unless otherwise specifically stated.

[0025] As used herein, ordinal terms such as "first," "second," etc., are used solely to distinguish one component from another and do not limit the order or relationship between the components. For example, the first component of the present invention may be referred to as the "second component," and similarly, the second component may also be referred to as the "first component."

[0026] FIG. 1 is a conceptual diagram schematically illustrating a stream control system according to one embodiment of the present invention. Referring to FIG. 1, a stream control system (100) according to one embodiment of the present invention will be described. The stream control system (100) is composed of a plurality of client devices (110) and a media server (120), and controls transmission streams and reception streams in a many-to-many streaming service in which a plurality of client devices (110) transmit and receive media to and from each other through the media server (120). Each client device (110) selects a reception stream based on a frame delay time and a bandwidth, and the media server (120) controls the bitrate of the transmission stream grade of each client device (110) based on the reception stream selected by each client device (110) and the bandwidth of each reception client device (110).

[0027] Here, each client device (110) performs both the operations of a transmitting client device that encodes media and transmits it to a media server (120) and a receiving client device that receives media from another client device (110) from the media server (120).

[0028] Next, the client device (110) and the media server (120) will be described in detail.

[0029] Referring to FIG. 2, a client device (110) is described.

[0030] A transmitting client device (111) encodes media into a plurality of transmission stream grades and transmits the media to a media server (120). A receiving client device (112) receives, from the media server (120), a bitrate map including identification information for at least one transmitting client device (111), grade information of a transmission stream transmitted by each transmitting client device (111), and bitrate information for each transmission stream grade. Then, based on the bitrate map, a selection table is generated consisting of combinations of a plurality of receiving streams including a grade of a transmission stream to be received from each transmitting client device (111) and a summed bitrate that is the sum of the bitrates of each transmission stream grade, and a receiving stream combination is selected based on a frame delay time.

[0031] Here, the first transmitting client device (111-1) and the first receiving client device (112-1) are the same device, and the media of the first transmitting client (111-1) is received by the receiving client device (112) excluding the first receiving client (112-1).

[0032] The bitrate map includes identification information of each transmitting client (111), a transmission stream class transmitted by each transmitting client (111), and bitrate information of each transmission stream class, as shown in FIG. 3. Each class can be set to a different bitrate depending on the bandwidth of each transmitting client device (111). The receiving client device (112) generates a selection table based on this.

[0033] The selection table generated based on the bitrate map is composed of multiple combinations of receiving streams, as shown in Fig. 4. The combination of receiving streams includes a combined bitrate that is the sum of the bitrates of the transmission stream classes to be received from each transmitting client device (111) and the overall transmission stream classes.

[0034] In the case of a many-to-many streaming service for n (n is a natural number greater than 2) client devices (110) as shown in Fig. 2, each receiving client device (112) receives media from n-1 transmitting client devices (111) excluding itself. If the receiving client device (112) receives media from n-1 transmitting client devices (111) and each transmitting client device (111) encodes media for three grades, the receiving stream combination is 3. n-1 Dogs can be created.

[0035] Additionally, if m (m is a natural number greater than 2) transmitting client devices (111) transmit media by transmitting client devices (111) that do not transmit media among n-1 transmitting client devices (111), the receiving stream combination is 3 m A dog can be generated. That is, the receiving stream combination is generated only for the transmitting client device (111) that transmits the media.

[0036] The receiving client device (112) selects one of these to receive the media of each transmitting client device (111). For example, if the first receiving stream combination is selected, the media for the first transmitting client device (111-1) and the media for the second transmitting client device (111-2) are received at the first level.

[0037] Additionally, the media frame received by the receiving client device (112) includes transmission start time information inserted from the transmitting client device (111), and the receiving client device (112) determines the frame delay time based on the start time information included in the media frame and the arrival time of the media frame. The receiving client device (112) can measure the delay time for each media frame or at predetermined intervals.

[0038] The receiving client device (112) initially selects a combination of receiving streams with the lowest total bitrate among the multiple receiving streams included in the selection table and starts streaming. Thereafter, the receiving client device (112) maintains or changes the combination of receiving streams depending on whether the frame delay time exceeds a preset reference time.

[0039] The receiving client device (112) changes to a receiving stream combination having a lower total bit rate when the frame delay time exceeds the reference time, and changes to a receiving stream combination having a higher total bit rate when the frame delay time is less than the reference time.

[0040] Additionally, the receiving client device (112) can sort the combinations of received streams having the same aggregate bitrate based on a Quality of Experience (QoE) indicator, and select one of the combinations of received streams having the same aggregate bitrate based on the QoE indicator.

[0041] Referring to FIG. 5, the client device (110) is configured to perform operations of a transmitting client device (111) and a receiving client device (112), and includes a memory (113) and a processor (114).

[0042] Memory (113) stores a stream control program. Memory (113) may be interpreted as a general term for a non-volatile storage device that maintains stored information even when power is not supplied and a volatile storage device that requires power to maintain the stored information. Memory (113) may perform a function of temporarily or permanently storing data processed by the processor (114). Memory (113) may include a magnetic storage media or a flash storage media in addition to a volatile storage device that requires power to maintain the stored information, but the scope of the present invention is not limited thereto.

[0043] Then, the processor (114) executes a stream control program in the memory (113) to encode and transmit media to the media server (120), and receives a bitrate map for another client device (110) from the media server (120) to select a combination of receiving streams to stream media of the other client device (110).

[0044] Meanwhile, the processor (114) may perform hardware control functions such as a file system, memory allocation, network, basic library, timer, device control (display, media, input device, 3D, etc.), and other utilities required when executing a program. In the present embodiment, the processor (114) may be implemented in the form of a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

[0045] The communication module (115) may include a device including hardware and software necessary for performing data communication with an external device and transmitting and receiving signals such as control signals or data signals through wired or wireless connections with other network devices. The database (116) may store various operation data for the stream control program to operate. For example, a bitrate map received from a media server (120) and a selection table generated based on the bitrate map may be stored.

[0046] Next, the media server (120) will be described with reference to FIGS. 2 and 6.

[0047] Referring to FIG. 2, the media server (120) receives encoded media from each transmitting client device (111) during the operation process of the client device (110) described above, and transmits media of the corresponding transmitting stream grade to the receiving client device (112) based on the combination of receiving streams selected by the receiving client device (112).

[0048] Additionally, the media server (120) generates a bitrate map to be transmitted to the receiving client device (112). The media server (120) receives bandwidth from each client device (110) and, through this, sets a bitrate for each transmission stream class of the transmitting client device (111) to generate the bitrate map.

[0049] For example, a case is described where a media server (120) is connected to three client devices (110), the bandwidth of the first client device is set to 1 to 3 Mbps, the bandwidth of the second client device is set to 2 to 5 Mbps, and the bandwidth of the third client device is set to 2 to 6 Mbps, and each transmitting client device (111) transmits media with three transmitting stream grades of High, Medium, and Low.

[0050] Here, the minimum value of the bandwidth range for the entire client device (110) is 1 Mbps, the maximum value is 6 Mbps, and each receiving client device receives two streams, which are N-1 in number.

[0051] The media server (120) can set the High stream of each transmitting client device (111) to MAX / (N-1) = 6 / 2 = 3 Mbps, the Low stream to MIN / (N-1) = 1 / 2 = 0.5 Mbps, and the Medium stream to 1.75 Mbps, which is the middle value between the High stream and the Low stream, to create a bitrate map in order to satisfy the maximum and minimum ranges for the bandwidth of all client devices (110).

[0052] The media server (120) can perform this operation at the beginning of a many-to-many streaming service, thereby ensuring that all client devices (110) have a selectable combination of receive streams within their bandwidth range.

[0053] Also, referring to FIG. 6, the media server (120) receives a combination of reception streams selected by each reception client device (112) from each reception client device (112), and adjusts a bitrate for multiple transmission stream grades of each transmission client device (111) based on the selected combination of reception streams.

[0054] Specifically, the media server (120) receives the combination of receiving streams selected by each receiving client device (112) at regular intervals, and the stream preference of each transmitting client device (111) included in the selected combination of receiving streams. ) is used to calculate the utility value of each transmission stream class of each transmitting client device (111), and the bit rate of each transmission stream class is adjusted based on the utility value. Here, the utility value may represent the gain obtained by all receiving client devices (112) when the bit rate of each transmission stream class of each transmitting client device (111) is increased or decreased.

[0055] The media server (120) calculates an incremental utility value and a decremental utility value for each transmission stream class based on the preference of the transmission stream class. The incremental utility value is calculated through an operation using a bitrate that increases the current bitrate by Δ and a preference, and represents the possibility of selecting a transmission stream class set to an incremental bitrate in the next cycle. In addition, the decremental utility value is calculated through an operation using a bitrate that decreases the current bitrate by Δ and a preference, and represents the possibility of selecting a transmission stream class set to a decremental bitrate. As a result, the media server (120) adjusts the bitrate to a larger value between the incremental utility value and the decremental utility value for each transmission stream class. That is, if the possibility of selecting is higher when the bitrate is increased, the bitrate is increased, and if the possibility of selecting is higher when the bitrate is decreased, the bitrate is decreased.

[0056] For example, an operation is described in which a media server (120) receives a combination of reception streams from each reception client device (112) at a 1-second cycle and calculates a utility value of each transmission stream grade of the first transmission client device (111-1) based on the combination of reception streams of the second reception client device (112-2) for the first transmission client device (111-1).

[0057] If the first transmitting client device (111-1) encodes media into three levels of transmitting streams: High, Medium, and Low, and the second receiving client device (112-2) maintains the Low, Medium, and High streams of the first transmitting client device (111-1) for 0.2, 0.5, and 0.3 seconds, respectively, for 1 second, the preferences for each stream become 0.2, 0.5, and 0.3.

[0058] This preference affects the selection of the transmission stream of the next cycle, and the possibility of selecting the High stream in the next cycle may be 0.2. Accordingly, the media server (120) calculates the incremental utility value of the High stream by using the preference and the upstream bitrate that increases the current bitrate of the High stream by Δ, and calculates the decremental utility value of the High stream by using the preference and the downstream bitrate that decreases the current bitrate of the High stream by Δ. The incremental utility value and the decremental utility value are also calculated for the Medium stream and the Low stream. This is the utility value of the second receiving client device (112-2) for each transmission stream of the first transmitting client device (111-1).

[0059] The media server (120) performs these operations for all receiving client devices (112), and calculates the total increased utility value and the total decreased utility value of each transmission stream of the first transmitting client device (111-1) by synthesizing them. Then, the bitrate is increased or decreased by Δ depending on the larger value between the total increased utility value and the total decreased utility value.

[0060] At this time, if the delay time exceeds the target time in any one of the receiving client devices that selected the Low stream, the media server (120) immediately reduces the bitrate of the Low stream by setting the reduction utility value of the Low stream to the maximum value. This is to ensure that a sufficiently low bitrate can be secured within the selectable range in order to prevent frame delay of the receiving client devices due to a rapid decrease in network bandwidth. This rule can be equally applied even if the number of stream classes increases.

[0061] Additionally, the media server (120) can calculate the increased utility value using mathematical formula 1 and the decreased utility value using mathematical formula 2.

[0062] [Mathematical Formula 1]

[0063]

[0064] [Equation 2]

[0065]

[0066] Mathematical expression 1 shows that the bit rate (r) of the transmission stream of class s of the transmitting client device i is transmitted by the receiving client device j. s ) is a formula for calculating the utility of increasing the bit rate (r) by Δ, and mathematical formula 2 is a formula for calculating the utility of increasing the bit rate (r) by Δ. s ) is a formula for calculating the utility of reducing Δ. Here, is the time that the receiving client device maintains the transmission stream of class s of the jth transmitting client device i.

[0067] In addition, the media server (120) can receive the changed combination of received streams when the receiving client device (112) changes the combination of received streams, and perform an operation of calculating an increased utility value and a decreased utility value for the difference in reception time from the previously received combination of received streams.

[0068] Through this operation, the media server (120) can increase the bitrate of the transmission stream grade that is more frequently used by all receiving client devices (112), thereby achieving higher efficiency with the current combination of receiving streams. In addition, by lowering the bitrate of the transmission stream of a higher grade, the receiving client devices (112) can select a transmission stream of a grade higher than that of the current transmission stream, thereby expanding the range of choices available to the receiving client devices (112), thereby improving the reception efficiency of all receiving client devices (112).

[0069] Then, the media server (120) updates the bitrate map by adjusting the bitrate of each transmission stream class of each transmission client device (111), and transmits the updated bitrate map to each reception client device (112).

[0070] Each receiving client device (112) can update a plurality of receiving stream combinations included in the selection table through the updated bitrate map, and maintain or change the receiving stream combinations.

[0071] Thereafter, the media server (120) repeats the operation of adjusting the bit rate of the transmission stream grade of each transmission client device (111) based on the change in the combination of the reception streams.

[0072] FIG. 7 is a flowchart illustrating a stream control method according to an embodiment of the present invention. Referring to FIGS. 2, 6, and 7, a stream control method (S100) for a multi-to-many streaming service according to an embodiment of the present invention is described.

[0073] Referring to FIG. 2, a receiving client device (112) receives a bitrate map including identification information for each of at least one transmitting client device (111) from a media server (120), rating information of a transmission stream transmitted by each transmitting client device, and bitrate information for each transmission stream rating (step S110).

[0074] Thereafter, the receiving client device (112) generates a selection table consisting of a combination of a plurality of receiving streams including a grade of a transmission stream to receive media from each transmitting client device (111) based on a bitrate map and a summed bitrate that is the sum of the bitrates of each transmission stream grade (step S120), and the receiving client device (112) selects a combination of receiving streams based on a frame delay time (step S130).

[0075] And, referring to FIG. 6, the media server (120) receives information about a combination of reception streams selected from each reception client device (112), and adjusts the bitrate of at least one transmission stream grade of each transmission client device (111) based on the combination of reception streams selected by each reception client device (112) (step S140).

[0076] In step S130, the receiving client device (112) determines a frame delay time based on the transmission start time information inserted into the media frame received from the transmitting client device (111) and the arrival time of the media frame.

[0077] Next, each step is explained.

[0078] In step S110, a bitrate map is generated from the media server (120) and includes identification information of each transmitting client (111), a transmission stream grade transmitted by each transmitting client (111), and bitrate information of each transmission stream grade, as shown in FIG. 3.

[0079] Referring to FIGS. 1 and 2, the process of the media server (120) generating a bitrate map is explained. The media server (120) receives bandwidth from each client device (110), and sets a bitrate for each transmission stream grade of the transmitting client device (111) through the bandwidth to generate a bitrate map.

[0080] For example, a case is described where a media server (120) is connected to three client devices (110), the bandwidth of the first client device is set to 1 to 3 Mbps, the bandwidth of the second client device is set to 2 to 5 Mbps, and the bandwidth of the third client device is set to 2 to 6 Mbps, and each transmitting client device (111) transmits media with three transmitting stream grades of High, Medium, and Low.

[0081] Here, the minimum value of the bandwidth range for all client devices (110) is 1 Mbps, the maximum value is 6 Mbps, and each receiving client device receives two streams, which are N-1. In order to satisfy the maximum and minimum ranges for the bandwidth of all client devices (110), the media server (120) can set the High stream of each transmitting client device (111) to MAX / (N-1) = 6 / 2 = 3 Mbps, the Low stream to MIN / (N-1) = 1 / 2 = 0.5 Mbps, and the Medium stream to 1.75 Mbps, which is the middle value between the High stream and the Low stream, to generate a bitrate map.

[0082] The media server (120) can perform this operation at the beginning of a many-to-many streaming service, thereby allowing all client devices (110) to secure a selectable combination of receiving streams within their own bandwidth range.

[0083] Next, the process (step S120) in which the receiving client device (112) creates a selection table is described.

[0084] The receiving client device (112) generates a selection table based on the bitrate map received from the media server (120). The selection table is composed of a plurality of combinations of receiving streams, as shown in FIG. 4. The combinations of receiving streams include a combined bitrate that is the sum of the bitrates of the transmission stream classes for receiving media from each transmitting client device (111) and the entire transmission stream classes. The receiving client device (112) initially selects a combination of receiving streams with the lowest combined bitrate among the plurality of receiving streams included in the selection table and starts streaming.

[0085] In the case of a many-to-many streaming service for n (n is a natural number greater than 2) client devices (110) as in Fig. 2, each receiving client device (112) receives media from n-1 transmitting client devices (111) excluding itself. If the receiving client device (112) receives media from n-1 transmitting client devices (111) and each transmitting client device (111) encodes media for three grades, the receiving stream combination is 3. n-1 Dogs can be created.

[0086] Additionally, if m (m is a natural number greater than 2) transmitting client devices (111) transmit media by transmitting client devices (111) that do not transmit media among n-1 transmitting client devices (111), the receiving stream combination is 3 mA dog can be generated. That is, the receiving stream combination is generated only for the transmitting client device (111) that transmits the media.

[0087] The receiving client device (112) selects one of these and receives the media of each transmitting client device (111). For example, if the second receiving stream combination is selected, the media for the first transmitting client device (111-1) is received at the first level, and the media for the second transmitting client device (111-2) is received at the second level.

[0088] Next, a process (step S130) in which a receiving client device (112) selects a combination of receiving streams based on a frame delay time is described.

[0089] The media frame received by the receiving client device (112) includes transmission start time information inserted from the transmitting client device (111), and the receiving client device (112) determines the frame delay time based on the start time information included in the media frame and the arrival time of the media frame. The receiving client device (112) can measure the delay time for each media frame or at predetermined intervals.

[0090] The receiving client device (112) maintains or changes the combination of received streams depending on whether the frame delay time exceeds a preset reference time. If the frame delay time exceeds the reference time, the receiving client device (112) changes to a combination of received streams having a lower total bit rate, and if the frame delay time is less than the reference time, the receiving client device (112) changes to a combination of received streams having a higher total bit rate.

[0091] And, the process (step S140) in which the media server (120) adjusts the bitrate of the transmission stream grade is described.

[0092] The media server (120) receives a combination of reception streams selected by each reception client device (112) from each reception client device (112), and adjusts the bitrate for multiple transmission stream grades of each transmission client device (111) based on the selected combination of reception streams.

[0093] Specifically, the media server (120) receives the combination of receiving streams selected by each receiving client device (112) at regular intervals, and the stream preference of each transmitting client device (111) included in the selected combination of receiving streams. ) is used to calculate the utility value of each transmission stream class of each transmitting client device (111), and the bit rate of each transmission stream class is adjusted based on the utility value. Here, the utility value may represent the gain obtained by all receiving client devices (112) when the bit rate of each transmission stream class of each transmitting client device (111) is increased or decreased.

[0094] The media server (120) calculates an incremental utility value and a decremental utility value for each transmission stream class based on the preference of the transmission stream class. The incremental utility value is calculated through an operation using a bitrate that increases the current bitrate by Δ and a preference, and represents the possibility of selecting a transmission stream class set to an incremental bitrate in the next cycle. In addition, the decremental utility value is calculated through an operation using a bitrate that decreases the current bitrate by Δ and a preference, and represents the possibility of selecting a transmission stream class set to a decremental bitrate. As a result, the media server (120) adjusts the bitrate to a larger value between the incremental utility value and the decremental utility value for each transmission stream class. That is, if the possibility of selecting is higher when the bitrate is increased, the bitrate is increased, and if the possibility of selecting is higher when the bitrate is decreased, the bitrate is decreased.

[0095] For example, an operation is described in which a media server (120) receives a combination of reception streams from each reception client device (112) at a 1-second cycle and calculates a utility value of each transmission stream grade of the first transmission client device (111-1) based on the combination of reception streams of the second reception client device (112-2) for the first transmission client device (111-1).

[0096] If the first transmitting client device (111-1) encodes media into three levels of transmitting streams: High, Medium, and Low, and the second receiving client device (112-2) maintains the Low, Medium, and High streams of the first transmitting client device (111-1) for 0.2, 0.5, and 0.3 seconds, respectively, for 1 second, the preferences for each stream become 0.2, 0.5, and 0.3.

[0097] This preference affects the selection of the transmission stream of the next cycle, and the possibility of selecting the High stream in the next cycle may be 0.2. Accordingly, the media server (120) calculates the incremental utility value of the High stream by using the preference and the upstream bitrate that increases the current bitrate of the High stream by Δ, and calculates the decremental utility value of the High stream by using the preference and the downstream bitrate that decreases the current bitrate of the High stream by Δ. The incremental utility value and the decremental utility value are also calculated for the Medium stream and the Low stream. This is the utility value of the second receiving client device (112-2) for each transmission stream of the first transmitting client device (111-1).

[0098] The media server (120) performs these operations for all receiving client devices (112), and calculates the total increased utility value and the total decreased utility value of each transmission stream of the first transmitting client device (111-1) by synthesizing them. Then, the bitrate is increased or decreased by Δ depending on the larger value between the total increased utility value and the total decreased utility value.

[0099] At this time, if the delay time exceeds the target time in any one of the receiving client devices that selected the Low stream, the media server (120) immediately reduces the bitrate of the Low stream by setting the reduction utility value of the Low stream to the maximum value. This is to ensure that a sufficiently low bitrate can be secured within the selectable range in order to prevent frame delay of the receiving client devices due to a rapid decrease in network bandwidth. This rule can be equally applied even if the number of stream classes increases.

[0100] Through this operation, the media server (120) can increase the bitrate of the transmission stream grade that is more frequently used by all receiving client devices (112), thereby achieving higher efficiency with the current combination of receiving streams. In addition, by lowering the bitrate of the transmission stream of a higher grade, the receiving client devices (112) can select a transmission stream of a grade higher than that of the current transmission stream, thereby expanding the range of choices available to the receiving client devices (112), thereby improving the reception efficiency of all receiving client devices (112).

[0101] Then, the media server (120) updates the bitrate map by adjusting the bitrate of each transmission stream grade of each transmission client device (111), and transmits the updated bitrate map to each reception client device (112). Each reception client device (112) can update a plurality of reception stream combinations included in the selection table through the updated bitrate map, and maintain or change the reception stream combination. Thereafter, the media server (120) repeats the operation of adjusting the bitrate of the transmission stream grade of each transmission client device (111) based on the change in the reception stream combination of the reception client device.

[0102] The present invention may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and include both volatile and nonvolatile media, removable and non-removable media. Computer-readable media may also include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0103] Additionally, although the methods and systems of the present invention have been described with respect to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

[0104] Those skilled in the art will appreciate that, based on the above description, the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the following claims, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

[0105] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. In a stream control system for a many-to-many streaming service, A plurality of client devices, each of which receives a bitrate map including identification information for at least one transmitting client device from a media server, grade information of a transmission stream transmitted by each transmitting client device, and bitrate information for each grade of the transmission stream, and generates a selection table including combinations of a plurality of receiving streams including a summed bitrate that is the sum of the grades of the transmission streams to be received from each transmitting client device and the bitrates of each grade of the transmission stream based on the bitrate map, and selects a combination of receiving streams based on a frame delay time; and A media server comprising: a media server configured to receive encoded media from at least one transmitting client device and to transmit said media to at least one receiving client device; The above client device, A stream control system that determines the frame delay time based on transmission start time information inserted into a media frame received from a transmitting client device and the arrival time of the media frame.

2. In paragraph 1, The above client device, A stream control system, which initially selects and streams a combination of received streams having the lowest total bitrate among a plurality of combinations of received streams included in the selection table.

3. In paragraph 1, The above client device, A stream control system that measures the delay time of a received media frame and maintains or changes the combination of the received streams depending on whether the delay time exceeds a preset reference time.

4. In paragraph 3, The above client device, A stream control system that changes to a combination of received streams having a lower total bitrate if the above delay time exceeds the above reference time.

5. In paragraph 3, The above client device, A stream control system that changes to a combination of received streams having a higher total bitrate when the above delay time is less than the above reference time.

6. In paragraph 1, The above media server, A stream control system that receives information about a selected combination of receive streams from each receiving client device, and adjusts a bitrate of at least one class of transmit streams of each transmitting client device based on the selected combination of receive streams of each receiving client device.

7. In paragraph 1, The above media server, A stream control system that calculates a utility value based on the stream retention time of each transmitting client device included in the combination of receiving streams selected by each receiving client device, and adjusts the bitrate of each transmitting stream class of the transmitting client device based on the maximum utility.

8. In the stream control method of a many-to-many streaming service, (a) a step in which a receiving client device receives from a media server a bitrate map including identification information for each of at least one transmitting client device, rating information of a transmission stream transmitted by each transmitting client device, and bitrate information for each rating of each transmission stream; (b) a step in which the receiving client device generates a selection table consisting of combinations of a plurality of receiving streams including a grade of a transmission stream to be received from each transmitting client device based on the bitrate map and a summed bitrate that is the sum of the bitrates of each transmission stream grade; and (c) comprising a step of the receiving client device selecting a combination of receiving streams based on a frame delay time; In step (c) above, A stream control method, wherein a constantly receiving client device determines the frame delay time based on transmission start time information inserted into a media frame received from the transmitting client device and the arrival time of the media frame.

9. In paragraph 8, Step (c) above, A stream control method, wherein initially, a combination of received streams having the lowest total bitrate among a plurality of combinations of received streams included in the selection table is selected and streamed.

10. In paragraph 8, Step (c) above, A stream control system that measures the delay time of a received media frame and maintains or changes the combination of the received streams depending on whether the delay time exceeds a preset reference time.

11. In paragraph 10, Step (c) above, A stream control method for changing to a combination of received streams having a lower total bitrate when the above delay time exceeds the above reference time.

12. In paragraph 10, Step (c) above, A stream control method for changing to a combination of received streams having a higher total bitrate when the above delay time is less than the above reference time.

13. In paragraph 8, (d) a stream control method further comprising the step of the media server receiving information about the combination of received streams selected from each receiving client device, and adjusting the bitrate of at least one transmission stream grade of each transmitting client device based on the selected combination of received streams of each receiving client device.

14. In paragraph 13, Step (d) above, A stream control method, wherein the media server calculates a utility value based on the stream retention time of each transmitting client device included in the combination of receiving streams selected by each receiving client device, and adjusts the bitrate of each transmitting stream grade of the transmitting client device based on the utility value.

Images