Method and system for audio control utilizing asymmetric channels in audio conferences - Patents.com

Abstract

To provide a method and system for controlling audio using an asymmetric channel of an audio conference.SOLUTION: An audio control method according to an embodiment includes: establishing a bi-directional media session having an asymmetric channel structure between a first participant among a plurality of participants participating in an audio conference and a mixer; receiving audio data from the first participant through a media session of a first channel structure in the bi-directional media session; mixing audio data of at least two participants excluding the first participant from among the plurality of participants, as a multichannel; and transferring the mixed audio data of the participants mixed as the multichannel to the first participant through a media session of a second channel structure in the bi-directional media session.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The following description relates to a method and system for audio control utilizing asymmetric channels in an audio conference. [Background technology]

[0002] An audio conference is a type of remote conference where participants can hold a conference in their own offices or workplaces by transmitting audio, without having to gather at a conference location. Not only remote conferences that use only audio data, but also multi-party calls in which multiple people talk at the same time and video conferences that transmit audio data along with video or media files can be said to fall within the scope of audio conferences, considering that audio is transmitted. In other words, audio conferences include all types of calls in which audio is transmitted.

[0003] At this time, in the case of a conventional audio conference using mixing, there is a problem that a user in the position of a listener cannot control each audio source. For example, Patent Document 1 relates to a method and system for sharing CCTV video and audio conferencing among multiple parties, and discloses a configuration for providing an audio conference by transcoding audio streams transmitted between a first connection, a second connection, and a CCTV camera, and mixing the first audio stream, the second audio stream, and the audio stream on the CCTV camera side in real time. In this case, since audio streams of many different audio sources are mixed, it is possible for the server to individually control the audio streams before mixing the audio streams, but a user who is only provided with an already mixed audio stream cannot individually control the audio stream of a specific audio source. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Korean Patent No. 10-1134581 Summary of the Invention [Problem to be solved by the invention]

[0005] To provide an audio control method and system that allows a user in a listener position to control audio for each source of audio data while using mixing in an audio conference. [Means for solving the problem]

[0006] Provided is an audio control method for a computer device including at least one processor, the audio control method including: setting a two-way media session having an asymmetric channel structure between a first participant of a plurality of participants participating in an audio conference and a mixer, the at least one processor receiving audio data from the first participant via the media session of the first channel structure of the two-way media sessions; mixing audio data of at least two or more participants of the plurality of participants excluding the first participant as multi-channel; and transmitting the audio data of the participant mixed as multi-channel to the first participant via a media session of the two-way media sessions having a second channel structure.

[0007] According to one aspect, the first channel structure may include a mono-channel structure, and the second channel structure may include a multi-channel structure.

[0008] According to another aspect, setting up the two-way media session may be characterized by determining at least one of a channel type, a channel coding scheme, and a processing unit for each direction by exchanging media processing information with the first participant.

[0009] According to another aspect, setting up the interactive media session may include determining a channel type, either one of a fixed channel and a variable channel, and a number of channels.

[0010] According to another aspect, setting up the bidirectional media session may include determining, when the media session of the second channel structure is set as multi-channel, one of a codec coding scheme using a codec supporting multi-channel encoding and an iterated coding scheme expressing multi-channels in a processing unit of voice data.

[0011] According to another aspect, the setting up of the interactive media session may include determining a processing unit of audio data for the iterative coding when iterative coding is used as a channel coding scheme.

[0012] According to another aspect, the first participant and the mixer may communicate based on a transport protocol of RTP (Real-time Transport Protocol), and when the media session of the second channel structure is set as multi-channel, the order of the channels may be indicated by an order of synchronization sources (SSRCs) indicated in a contributing source (CSRC) predefined based on the transport protocol of the RTP, and the number of channels may be indicated by a CSRC count (CC) further defined based on the transport protocol of the RTP.

[0013] According to another aspect, the transmitting may be characterized by transmitting audio data of n participants (n being a natural number greater than or equal to 2) as the at least two or more participants via n channels as the multi-channel.

[0014] According to yet another aspect, the at least two or more participants may be classified into m groups (where m is a natural number equal to or greater than two), and the transmitting step may include transmitting the audio data of the participants mixed by group via m channels formed for the m groups.

[0015] Provided is an audio control method for a computer device including at least one processor, the audio control method including: setting up a two-way media session having an asymmetric channel structure between a mixer included in a server providing an audio conference and a first participant among a plurality of participants participating in the audio conference, the at least one processor transmitting audio data of the first participant to the mixer via a media session of the first channel structure of the two-way media session; receiving, from a mixer that mixes audio data of at least two or more participants among the plurality of participants excluding the first participant as multi-channel, the audio data of the participants mixed as multi-channel, via a media session of the second channel structure of the two-way media session; and individually controlling audio data of a channel selected from the multi-channel.

[0016] According to one aspect, the individually controlling may be characterized by controlling a volume of the audio data of the selected channel, removing noise from the audio data of the selected channel, or modulating the audio data of the selected channel.

[0017] A computer program is provided for causing a computer device to carry out the method.

[0018] A computer-readable recording medium is provided on which a program for causing a computer device to execute the method is recorded.

[0019] Provided is a computer device including at least one processor, wherein the at least one processor sets up a two-way media session having an asymmetric channel structure between a first participant among a plurality of participants participating in an audio conference and a mixer, receives audio data from the first participant via the media session of the first channel structure among the two-way media sessions, mixes audio data of at least two or more participants among the plurality of participants excluding the first participant as multi-channel, and transmits the audio data of the participant mixed as multi-channel to the first participant by the at least one processor via a media session of the second channel structure among the two-way media sessions.

[0020] Provided is a computer device including at least one processor, the at least one processor setting up a two-way media session having an asymmetric channel structure between a first participant and a mixer included in a server that provides an audio conference, transmitting audio data of the first participant to the mixer via a media session having a first channel structure among the two-way media sessions, receiving the audio data of the participants mixed as multi-channels from a mixer that mixes audio data of at least two or more participants excluding the first participant from among the multiple participants, via a media session having a second channel structure among the two-way media sessions, and individually controlling the audio data of a channel selected from the multi-channels. Effect of the Invention

[0021] While using mixing in an audio conference, a user at a listening position can control the audio for each source of audio data. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a computing device in accordance with an embodiment of the present invention. [Diagram 3] 1 is a schematic diagram illustrating an audio control system according to an embodiment of the present invention; [Figure 4] FIG. 2 illustrates an example of a two-way media session with an asymmetric channel structure in accordance with an embodiment of the present invention. [Diagram 5] FIG. 2 illustrates an example of receiving mixed audio data via a media session with a mono channel structure in accordance with an embodiment of the present invention. [Figure 6] FIG. 2 illustrates an example of receiving mixed audio data via a multi-channel structured media session in accordance with an embodiment of the present invention. [Figure 7] 4 is a flowchart illustrating an example of an audio control method of a server in accordance with an embodiment of the present invention. [Figure 8] 4 is a flowchart illustrating an example of a method for controlling audio by a client in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0024] An audio control system according to an embodiment of the present invention may be realized by at least one computer device, and an audio control method according to an embodiment of the present invention may be executed by at least one computer device included in the audio control system. In this case, a computer program according to an embodiment of the present invention may be installed and executed in the computer device, and the computer device may execute the audio control method according to an embodiment of the present invention according to the control of the executed computer program. The above-mentioned computer program may be recorded in a computer-readable recording medium in combination with the computer device to cause the computer to execute the audio control method.

[0025] Fig. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Fig. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Such Fig. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in Fig. 1. Furthermore, the network environment in Fig. 1 is merely an example of an environment applicable to this embodiment, and the environment applicable to this embodiment is not limited to the network environment in Fig. 1.

[0026] The electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer devices. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. As an example, FIG. 1 shows a smartphone as an example of the electronic device 110, but in the embodiment of the present invention, the electronic device 110 may mean one of various physical computer devices that can communicate with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0027] The communication method is not limited, and may include not only a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 can include, but also short-range wireless communication between devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Furthermore, the network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, and the like.

[0028] Each of the servers 150, 160 may be implemented by one or more computing devices that communicate with the multiple electronic devices 110, 120, 130, 140 via the network 170 to provide instructions, code, files, content, services, etc. For example, the server 150 may be a system that provides services (such as, for example, an audio conferencing service, a messaging service, an email service, a social networking service, a map service, a translation service, a financial service, a payment service, a search service, a content provision service, etc.) to the multiple electronic devices 110, 120, 130, 140 connected via the network 170.

[0029] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the electronic devices 110, 120, 130, and 140 and the servers 150 and 160 described above may be realized by a computer device 200 shown in FIG.

[0030] Such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240, as shown in FIG. 2. The memory 210 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a permanent mass storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate permanent storage device separate from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory 210 through a communication interface 230 that is not a computer-readable recording medium. For example, the software components may be loaded into the memory 210 of the computing device 200 based on a computer program installed by a file received over the network 170 .

[0031] The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processor 220 by the memory 210 or the communication interface 230. For example, the processor 220 may be configured to execute instructions received according to program code stored in a storage device, such as the memory 210.

[0032] The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (for example, the above-mentioned recording device) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 according to program codes recorded in a recording device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc. from other devices may be received by the computer device 200 via the communication interface 230 of the computer device 200 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and the files, etc. may be recorded on a recording medium (the above-mentioned permanent recording device) that the computer device 200 may further include.

[0033] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display and a speaker. As another example, the input / output interface 240 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer device 200.

[0034] Also, in other embodiments, computing device 200 may include fewer or more components than those of Figure 2. However, most prior art components need not be explicitly illustrated. For example, computing device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

[0035] FIG. 3 is an exemplary diagram illustrating an audio control system according to an embodiment of the present invention. The audio control system according to the present embodiment may be configured with a server 310 providing an audio conference service and a client device 320 for multiple participants. In this case, the server 310 may include a mixer 311, which may mix and transmit audio data of other participants participating in the same audio conference to a first participant who has participated in the audio conference. For example, it is considered that client device 1 (321), client device 2 (322), and client device 3 (323) among the client devices 320 participate in one audio conference instance. In this case, the server 310 may mix the first audio data received from client device 1 (321) and the second audio data received from client device 2 (322) by the mixer 311, and transmit the mixed audio data to client device 3 (323).

[0036] The mixer 311 may be, for example, a software module that is generated in the server 310 for one audio conference instance and relays audio data between participants participating in the audio conference instance. In this case, the mixer 311 may set up a bidirectional media session with each of the participants participating in the audio conference instance. In particular, the mixer 311 according to the present embodiment may set up a bidirectional media session with an asymmetric channel structure so that each of the participants can individually control the mixed audio data.

[0037] FIG. 4 is a diagram illustrating an example of a bidirectional media session with an asymmetric channel structure in an embodiment of the present invention. FIG. 4 illustrates an example in which a bidirectional media session with an asymmetric channel structure is set between the mixer 311 and the client device 3 (323). Since the client device 3 (323) only needs to transmit its own audio data to the mixer 311, a media session in a direction from the client device 3 (323) to the mixer 311 may be set in a mono-channel structure. On the other hand, a media session in a direction from the mixer 311 to the client device 3 (323) may be set in a mono-channel structure if only mixed audio data is received. However, in this case, since it is mixed single-channel audio data, it is not possible to distinguish the individual audio data of the participants, and it is not possible to control the individual audio data for each participant. Therefore, in order to control the individual audio data, according to this embodiment, a media session in a direction from the mixer 311 to the client device 3 (323) may be set in a multi-channel structure. Such a bidirectional media session with an asymmetric channel structure may be set for a client device that wishes to control the individual audio data. In other words, a client device that does not wish to control individual audio data may have a mono-channel structured two-way media session established with mixer 311 .

[0038] FIG. 5 is a diagram illustrating an example of receiving mixed audio data through a media session with a mono-channel structure in one embodiment of the present invention, and FIG. 6 is a diagram illustrating an example of receiving mixed audio data through a media session with a multi-channel structure in one embodiment of the present invention.

[0039] 5 shows an example in which a media session from the mixer 311 to the client device 3 (323) is set as a mono channel structure, and the mixer 311 mixes audio data of the client device 1 (321) and audio data of the client device 2 (322) through the media session of such mono channel structure and transmits the mixed data to the client device 3 (323). In this embodiment, the mixer 311 may mix a first packet 511 from the client device 1 (321) and a second packet 521 from the client device 2 (322) to generate a third packet 531 for the mono channel, and transmit the generated third packet 531 to the client device 3 (323). Also, an example is shown in which the mixer 311 mixes a fourth packet 512 from the client device 1 (321) and a fifth packet 522 from the client device 2 (322) to generate a sixth packet 532 for the mono channel, and transmits the generated sixth packet 532 to the client device 3 (323). In this case, because mixer 311 transmits third packet 531 and sixth packet 532 mixed for the mono channel, client device 3 (323) cannot individually control the audio data from client device 1 (321) and the audio data from client device 2 (322).

[0040] 6 shows an example in which a media session from the mixer 311 to the client device 3 (323) is set as a multi-channel structure, and the mixer 311 mixes audio data of the client device 1 (321) and audio data of the client device 2 (322) through the media session of the multi-channel structure and transmits the mixed data to the client device 3 (323). In this embodiment, the mixer 311 may mix a first packet 511 from the client device 1 (321) and a second packet 521 from the client device 2 (322) to generate a seventh packet 611 for multi-channels and transmit the generated seventh packet 611 to the client device 3 (323). Also, an example is shown in which the mixer 311 mixes a fourth packet 512 from the client device 1 (321) and a fifth packet 522 from the client device 2 (322) to generate an eighth packet 612 for multi-channels and transmits the generated eighth packet 612 to the client device 3 (323). In this case, the mixer 311 transmits the seventh packet 611 and the eighth packet 612 mixed for the multi-channel structure, so that the client device 3 (323) can individually control the audio data from the client device 1 (321) and the audio data from the client device 2 (322).

[0041] FIG. 7 is a flowchart showing an example of an audio control method of a server in an embodiment of the present invention. The audio control method according to this embodiment may be executed by a computer device 200 that realizes a server for an audio conference service. As an example, the server may be an entity that provides an audio conference service. In this case, the processor 220 of the computer device 200 may be realized to execute control instructions according to the operating system code and at least one program code included in the memory 210. Here, the processor 220 may control the computer device 200 so that the computer device 200 executes steps 710 to 740 included in the method of FIG. 7 according to the control instructions provided by the code recorded in the computer device 200.

[0042] In step 710, the computer device 200 may set up a bidirectional media session having an asymmetric channel structure between a first participant of multiple participants participating in the audio conference and a mixer, where the audio conference may correspond to one audio conference instance, and the mixer may be a software module generated by the computer device 200. Also, the multiple participants may be clients participating in the same audio conference instance among clients identified by an account of the audio conference service.

[0043] In this case, the computer device 200 may set up an interactive media session having an asymmetric channel structure for all participants, or in some embodiments, may set up an interactive media session having an asymmetric channel structure only for participants who wish to individually control audio data for each participant.

[0044] In one embodiment, the asymmetric channel structure may include a first channel structure including a mono-channel structure and a second channel structure including a multi-channel structure. In other words, a bidirectional media session of the asymmetric channel structure may be set between the first participant and the mixer, in which a media session in a direction from the first participant to the mixer and a media session in a direction from the mixer to the first participant have different channel structures. In this case, since the first participant only transmits its own audio data to the mixer, a media session of the first channel structure may be set as the media session in the direction from the first participant to the mixer. On the other hand, since the mixer can transmit audio data of multiple participants to the first participant, a media session of the second channel structure may be set as the media session in the direction from the mixer to the first participant.

[0045] To set up a two-way media session with such an asymmetric channel structure, the computing device 200 may determine at least one of a channel type, a channel coding scheme, and a processing unit for each direction by exchanging media processing information with the first participant.

[0046] Here, the channel type defines the type and number of channels, and the channel type may include fixed channels and variable channels. Fixed channels may mean a type having a fixed number of channels, and variable channels may mean a type in which the number of channels can be changed. As an example, a variable channel that allows up to 15 channels to be changed may be used in the transmission protocol of RTP (Real-time Transport Protocol). In other words, the computer device 200 may determine the type and number of channels of any one of fixed channels and variable channels in step 710. If the channel type is not specifically determined, "Fixed 1" having one fixed channel may be determined by default.

[0047] Also, in the case of multi-channel, the channel coding scheme may define a coding scheme of the multi-channels in the payload. If the channel type is "fixed 1", it means a mono channel, so there is no point in determining the channel coding scheme. Such a channel coding scheme may include a codec coding scheme using multi-channel encoding of a codec and iterated coding that expresses multi-channels in a processing unit. In other words, in the case where the media session of the second channel structure is set as multi-channel in step 710, the computer device 200 may determine one of the channel coding schemes of codec coding using a codec supporting multi-channel encoding and iterated coding that expresses multi-channels in a processing unit of audio data.

[0048] The processing unit may be determined to use the iterative coding scheme as the basic processing unit of the audio data as described above. In other words, when iterative coding is used as the channel coding scheme, the computer device 200 may determine the processing unit of the audio data for iterative coding.

[0049] In step 720, the computer device 200 may receive audio data from a first participant via a media session of a first channel structure of the two-way media session. As described above, the first channel structure may include a mono channel structure and may mean a case where the channel type is "fixed 1". As described above, when the channel type is "fixed 1", it is meaningless to determine a channel coding scheme, and therefore there is no need to determine a processing unit.

[0050] In step 730, the computer device 200 may mix audio data of at least two or more participants excluding the first participant as multi-channel. If there are only two participants in the audio conference instance, multi-channel is meaningless, but if there are three or more participants, the first participant may receive audio data from two or more participants, so multi-channel may be utilized. In this case, the computer device 200 may mix audio data received from two or more participants excluding the first participant as multi-channel. At this time, the mixed audio data may be coded according to the channel coding scheme previously determined in step 710.

[0051] In step 740, the computer device 200 may transmit the audio data of the participants mixed as multi-channels to the first participant through the media session of the second channel structure of the two-way media session. The audio data mixed as multi-channels may be transmitted to the first participant through the media session of the second channel structure, in other words, the media session of the multi-channel structure. In this case, since the audio data of the participants can be identified by each channel, the first participant can control the audio data of the other participants on a participant-by-participant basis.

[0052] For example, the computer device 200 may transmit audio data of at least two or more n participants (n is a natural number of 2 or more) excluding the first participant among the multiple participants, via n channels that are multi-channels. In this case, the first participant can control the audio data of the other participants excluding the first participant for each participant. As another example, the at least two or more participants may be classified into m groups (m is a natural number of 2 or more). In this case, the computer device 200 may transmit audio data of the participants mixed for each group via m channels formed for the m groups. In this case, the first participant can control the audio data mixed for each group for each group.

[0053] Meanwhile, the first participant and the mixer may communicate based on the transfer protocol of RTP (Real-time Transport Protocol). In this case, when the media session of the second channel structure is set as a multi-channel, the order of the channels may be indicated by the order of synchronization sources (SSRCs) indicated in a contributing source (CSRC) predefined based on the transfer protocol of RTP, and the number of channels may be indicated by a CSRC count (CC) further defined based on the transfer protocol of RTP. The SSRC may indicate a source of audio data, in other words, a participant who intends to transmit audio data to the first participant, and the CSRC may indicate a list of such SSRCs. In other words, the order of the SSRCs indicated in the CSRC may correspond to the order of the channels. Also, the CC indicates the number of SSRCs, and such a CC may correspond to the number of channels.

[0054] FIG. 8 is a flowchart showing an example of an audio control method of a client in an embodiment of the present invention. The audio control method according to this embodiment may be executed by a computer device 200 that realizes a client for an audio conference service. As an example, a server may be an entity that participates in an audio conference instance. In this case, the processor 220 of the computer device 200 may be realized to execute control instructions according to the operating system code and at least one program code included in the memory 210. Here, the processor 220 may control the computer device 200 so that the computer device 200 executes steps 810 to 840 included in the method of FIG. 8 according to the control instructions provided by the code recorded in the computer device 200.

[0055] In step 810, the computer device 200 may set up a two-way media session having an asymmetric channel structure between the first participant and a mixer included in a server providing an audio conference. Setting up a two-way media session having an asymmetric channel structure between the mixer and the first participant has been described in detail with reference to Fig. 7. The mixer and the first participant may exchange media processing information, such as a codec, to determine at least one of a channel type, a channel coding scheme, and a processing unit.

[0056] In step 820, the computer device 200 may transmit audio data of the first participant to the mixer via a media session of a first channel structure of the two-way media session. As described above, the first channel structure may include a mono channel structure. The computer device 200 of this embodiment, which is the client device of the first participant, may transmit the audio data of the first participant to the mixer via the media session of such a mono channel structure.

[0057] In step 830, the computer device 200 may receive audio data of participants mixed as multi-channels from a mixer that mixes audio data of at least two or more participants excluding the first participant as multi-channels, via a media session of a second channel structure among the two-way media sessions. The second channel structure may include a multi-channel structure. In this case, the computer device 200 may receive multi-channel audio data for each of the audio data of the two or more participants via a media session of such a multi-channel structure. For example, the computer device 200 may receive audio data of at least n participants (n is a natural number of 2 or more) as two or more participants via n channels as multi-channels. As another example, the computer device 200 may divide the two or more participants into two or more groups and receive audio data corresponding to each group via a media session of a multi-channel structure. For example, assuming an instance A of an audio conference in which participants 1, 2, 3, 4, 5, 6, 7, and 8 participate, the participant 1 may set a group 1 including participants 2, 3, and 4 and a group 2 including participants 5, 6, 7, and 8. In this case, the server may be associated with the instance A of the audio conference, and information about the group 1 and information about the group 2 may be set by the participant 1, and a mixer included in the server may set a media session including a first channel for mixing and transmitting audio data of the participants of group 1 (participants 2, 3, and 4) and a second channel for mixing and transmitting audio data of the participants of group 2 (participants 5, 6, 7, and 8) in association with the instance A of the audio conference for the participant 1. In this case, the participant 1 can individually control the audio data of the group 1 and the audio data of the group 2 in the instance A of the audio conference. From the above description, an embodiment in which other participants are set in three or more groups and audio data is controlled by group via three or more multi-channels can also be easily understood.

[0058] In step 840, the computer device 200 may individually control the audio data of the channel selected from the multiple channels. For example, the computer device 200 may control the volume of the audio data of the selected channel, remove noise from the audio data of the selected channel, or modulate the audio data of the selected channel. For this purpose, the computer device 200 may provide the user (first participant) with a user interface including at least one of a function of controlling the volume of the audio data of the selected channel, a function of removing noise from the audio data of the selected channel, and a function of modulating the audio data of the selected channel.

[0059] In this manner, according to an embodiment of the present invention, even when mixing is used in an audio conference, a user in a listener's position can control the audio for each source of audio data.

[0060] The above-described systems or devices may be realized by hardware components or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0061] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either independently or collectively. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted based on the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0062] The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The medium may be one that continuously records a computer-executable program, or one that temporarily records it for execution or download. The medium may be one that is a variety of recording or storage means in the form of a single or multiple hardware devices combined, and is not limited to a medium directly connected to a certain computer system, but may be one that is distributed over a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, and the like, which are configured to record program instructions. Other examples of the medium include recording media or storage media managed by application stores that distribute applications, or sites, servers, and the like that supply or distribute various other software. Examples of the program instructions include not only machine language codes such as those generated by compilers, but also high-level language codes that are executed by a computer using an interpreter or the like.

[0063] Although the embodiments have been described above based on the limited embodiments and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

[0064] Therefore, different embodiments are within the scope of the appended claims provided they are equivalent to the claims. [Explanation of symbols]

[0065] 310: Server 311: Mixer 320: Client device

Claims

1. 1. A method of audio control for a computing device including at least one processor, comprising: the at least one processor establishing a two-way media session having an asymmetric channel structure between a first participant of a plurality of participants participating in an audio conference and a mixer; receiving, by the at least one processor, audio data from the first participant via a media session of a first channel structure of the two-way media session; said at least one processor mixing audio data of at least two or more participants, excluding said first participant, from said plurality of participants as multi-channel; communicating, by the at least one processor, the multi-channel mixed participant audio data to the first participant via a second channel structured media session of the two-way media session; and setting up the two-way media session includes determining at least one of a channel type, a channel coding scheme, and a processing unit for each direction by exchanging media processing information with the first participant.

2. the first channel structure comprises a monochannel structure; 2. The audio control method of claim 1, wherein the second channel structure comprises a multi-channel structure.

3. Establishing the interactive media session comprises:

2. The audio control method of claim 1, further comprising determining the type of channel, either one of fixed channels and variable channels, and the number of channels.

4. Establishing the interactive media session comprises: The audio control method of claim 1, characterized in that, when the media session of the second channel structure is set as multi-channel, a channel coding scheme is determined to be one of codec coding using a codec that supports multi-channel encoding and repetition coding that expresses multi-channels in a processing unit of audio data.

5. Establishing the interactive media session comprises: The audio control method according to claim 1, further comprising determining a processing unit of the audio data for the repetition coding when the repetition coding is used as a channel coding method.

6. The first participant and the mixer communicate with each other based on a transfer protocol of RTP (Real-time Transport Protocol); 2. The audio control method of claim 1, wherein, when the media session of the second channel structure is set as a multi-channel, the order of the channels is indicated by an order of synchronization sources (SSRCs) indicated in a contributing source (CSRC) predefined based on the RTP transport protocol, and the number of channels is indicated by a CSRC count (CC) further defined based on the RTP transport protocol.

7. The communicating step includes: The audio control method according to claim 1, characterized in that audio data of n participants (n being a natural number equal to or greater than 2) as the at least two or more participants is transmitted via n channels as the multi-channel.

8. The at least two or more participants are classified into m groups (where m is a natural number equal to or greater than 2); The communicating step includes:

2. The audio control method according to claim 1, further comprising transmitting the audio data of the participants mixed for each group via m channels formed for the m groups.

9. 1. A method of audio control for a computing device including at least one processor, comprising: The at least one processor sets up a two-way media session having an asymmetric channel structure between a mixer included in a server that provides an audio conference and a first participant among a plurality of participants participating in the audio conference; the at least one processor transmitting audio data of the first participant to the mixer via a media session of a first channel structure of the two-way media session; receiving, from a mixer that mixes audio data of at least two or more participants, excluding the first participant, among the plurality of participants, as a multi-channel, audio data of the participants mixed as the multi-channel via a media session having a second channel structure among the two-way media sessions; said at least one processor individually controlling audio data for selected channels of said multi-channel; and establishing the interactive media session comprises: and determining at least one of a channel type, a channel coding scheme, and a processing unit for each direction by exchanging media processing information with the mixer.

10. the first channel structure comprises a monochannel structure; 10. The audio control method of claim 9, wherein the second channel structure comprises a multi-channel structure.

11. Receiving the audio data includes: The audio control method according to claim 9, characterized in that audio data of n participants (n being a natural number equal to or greater than 2) as the at least two or more participants is received via n channels as the multi-channel.

12. The at least two or more participants are classified into m groups (where m is a natural number equal to or greater than 2); Receiving the audio data includes: The audio control method according to claim 9, further comprising receiving the audio data of the participants mixed for each group via m channels formed for the m groups.

13. The audio control method includes: providing a user interface including at least one of a function for controlling a volume of the audio data of the selected channel, a function for removing noise from the audio data of the selected channel, and a function for modulating the audio data of the selected channel; The audio control method of claim 9, further comprising:

14. The individual control includes:

10. The audio control method of claim 9, further comprising controlling a volume of the audio data of the selected channel, removing noise from the audio data of the selected channel, or modulating the audio data of the selected channel.

15. A computer program product configured to cause a computer device to carry out the method according to any one of claims 1 to 14.

16. A computer-readable recording medium having a computer program recorded thereon for causing a computer device to execute the method according to any one of claims 1 to 14.

17. 1. A computing device including at least one processor, The at least one processor: establishing a two-way media session having an asymmetric channel structure between a first participant of a plurality of participants participating in an audio conference and a mixer; receiving audio data from the first participant via a media session of a first channel structure of the two-way media session; mixing audio data of at least two or more participants, excluding the first participant, from among the plurality of participants into multi-channel; communicating the multi-channel mixed participant audio data to the first participant via a second channel structured media session of the two-way media session; 4. The computer apparatus of claim 3, wherein setting up the two-way media session includes exchanging media processing information with the first participant to determine at least one of a channel type, a channel coding scheme, and a processing unit for each direction.

18. A computing device including at least one processor, comprising: The at least one processor: A two-way media session having an asymmetric channel structure is established between a mixer included in a server providing an audio conference and a first participant among a plurality of participants participating in the audio conference; transmitting audio data of the first participant to the mixer via a media session of a first channel structure of the two-way media session; receiving, from a mixer that mixes, as multi-channel, audio data of at least two or more participants, excluding the first participant, among the plurality of participants, via a media session having a second channel structure among the two-way media sessions, the audio data of the participants mixed as multi-channel; individually controlling audio data of channels selected from the multi-channel; The computer apparatus, wherein setting up the bidirectional media session comprises exchanging media processing information with the mixer to determine at least one of a channel type, a channel coding scheme, and a processing unit for each direction.

Images