Audio control method and system using asymmetric channels of voice conference

Abstract

The present invention discloses an audio control method and system using an asymmetric channel of a voice conference. According to one embodiment, the audio control method may include: a step of establishing an interactive media session with an asymmetric channel structure between a first participant among multiple participants in a voice conference and a mixer; a step of receiving audio data from the first participant through a media session with a first channel structure in the interactive media session; a step of mixing audio data of at least two participants other than the first participant among the multiple participants into multiple channels; and a step of transmitting the audio data of the at least two participants mixed into the multiple channels to the first participant through a media session with a second channel structure in the interactive media session.

Description

Technical Field

[0001] The following description relates to an audio control method and system utilizing an asymmetric channel for voice conferencing. Background Art

[0002] An audio conference is a remote conference in which participants do not gather at a venue, but conduct the conference through voice transmission in their respective offices or workplaces. Considering the transmission of voice, remote conferences that only use audio data, multi-person calls where many people talk at the same time, or video conferences that transmit audio data together with video and media files all fall within the scope of this type of audio conference. In other words, an audio conference can include all types of calls that transmit voice.

[0003] At this time, in the case of an existing voice conference using mixing, there is a problem that users in the audience position cannot control each voice source. For example, Korean Patent No. 10-1134581 relates to CCTV image sharing and multi-party voice conferencing method and system, and discloses the following technology, that is, transcoding the voice stream transmitted between the first terminal and the second terminal and the CCTV camera, and mixing the first voice stream and the second voice stream with the voice stream on the CCTV camera side in real time to provide a voice conference. In this case, since multiple voice streams of multiple different voice sources are mixed, although individual voice streams can be controlled before the multiple voice streams are mixed in the server, users who receive the mixed voice streams cannot control individual voice streams of specific voice sources. Summary of the invention

[0004] The object of the present invention is to provide an audio control method and system, that is, audio mixing is used in a voice conference, and a user in the position of an audience can control the audio of each audio data source.

[0005] The present invention provides an audio control method, which is an audio control method for a computer device including at least one processor, the audio control method comprising: a step of establishing an interactive media session with an asymmetric channel structure between a first participant among multiple participants in a voice conference and a mixer using the at least one processor; a step of receiving audio data from the first participant through a media session with a first channel structure in the interactive media session using the at least one processor; a step of mixing audio data of at least two participants among the multiple participants except the first participant into multiple channels using the at least one processor; and a step of transmitting the audio data of the at least two participants mixed into the multiple channels to the first participant through a media session with a second channel structure in the interactive media session using the at least one processor.

[0006] According to one embodiment, the first channel structure may include a single-channel structure, and the second channel structure may include a multi-channel structure.

[0007] According to yet another embodiment, in the step of establishing the above-mentioned interactive media session, at least one of the channel type, channel coding mode and processing unit in each direction may be determined by exchanging media processing information with the above-mentioned first participant.

[0008] According to another embodiment, in the step of establishing the above interactive media session, a channel type and a channel number of fixed channels and variable channels may be determined.

[0009] According to another embodiment, in the step of establishing the above-mentioned interactive media session, when the media session of the above-mentioned second channel structure is established as a multi-channel, a channel coding method of codec coding and iterated coding can be determined, the above-mentioned codec coding utilizes a codec supporting multi-channel coding, and the above-mentioned iterative coding expresses multiple channels through a processing unit of voice data.

[0010] According to yet another embodiment, in the step of establishing the above interactive media session, when iterative coding is used as a channel coding method, a processing unit for the above iteratively coded voice data may be determined.

[0011] According to another embodiment, the first participant and the mixer can communicate via a real-time transport protocol (RTP), and when the media session of the second channel structure is established as a multi-channel, the order of the channels is represented by the order of synchronization sources (SSRC) appearing in a predefined contributing source (CSRC) according to the real-time transport protocol, and the number of channels is represented by a CSRC count (CC) further defined according to the real-time transport protocol.

[0012] According to still another embodiment, in the transmitting step, the audio data of n participants as the at least two participants may be transmitted through n (the n is a natural number greater than or equal to 2) channels as the multi-channels.

[0013] According to another embodiment, the at least two participants may be classified into m (the m is a natural number greater than 2) groups, and in the transmission step, the audio data of multiple participants mixed according to the groups may be transmitted through the m channels formed for the m groups.

[0014] The present invention provides an audio control method, which is an audio control method for a computer device including at least one processor, the audio control method comprising: a step of establishing an interactive media session with an asymmetric channel structure between a mixer included in a server for providing a voice conference and a first participant using the at least one processor; a step of transmitting audio data of the first participant to the mixer through a media session with a first channel structure in the interactive media session using the at least one processor; a step of receiving audio data of the at least two participants mixed into multiple channels through a media session with a second channel structure in the interactive media session from the mixer that mixes audio data of at least two participants among the multiple participants except the first participant into multiple channels using the at least one processor; and a step of individually controlling audio data of a channel selected from the multiple channels.

[0015] According to one embodiment, in the step of individually controlling, the volume of the audio data of the selected channel may be controlled, noise of the audio data of the selected channel may be eliminated, or the audio data of the selected channel may be modulated.

[0016] The present invention provides a computer program stored in a computer-readable recording medium, so as to be combined with a computer device to execute the above method in the computer device.

[0017] The present invention provides a computer-readable recording medium having recorded thereon a computer program for executing the above method in a computer device.

[0018] The present invention provides a computer device, characterized in that it includes at least one processor implemented to execute computer-readable instructions, and using the at least one processor, an interactive media session with an asymmetric channel structure is established between a first participant among multiple participants participating in a voice conference and a mixer, audio data is received from the first participant through a media session with a first channel structure in the interactive media session, audio data of at least two participants among the multiple participants except the first participant are mixed into multiple channels, and using the at least one processor, the audio data of the at least two participants mixed into the multiple channels are transmitted to the first participant through a media session with a second channel structure in the interactive media session.

[0019] The present invention provides a computer device, characterized in that it includes at least one processor implemented to execute computer-readable instructions, and using the at least one processor, an interactive media session with an asymmetric channel structure is established between a mixer included in a server for providing voice conferencing and a first participant, audio data of the first participant is transmitted to the mixer through a media session with a first channel structure in the interactive media session, audio data of the at least two participants among the multiple participants except the first participant are mixed into multiple channels from the mixer, audio data of the at least two participants mixed into the multiple channels through a media session with a second channel structure in the interactive media session are received, and audio data of a channel selected from the multiple channels is individually controlled.

[0020] The present invention can use audio mixing in a voice conference, and users in the audience position can control the audio of each audio data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A diagram showing an example of a network environment according to an embodiment of the present invention.

[0022] Figure 2 FIG. 1 is a block diagram showing an example of a computer device according to an embodiment of the present invention.

[0023] Figure 3 This is a diagram showing an example of a schematic state of an audio control system according to an embodiment of the present invention.

[0024] Figure 4 FIG. 1 is a diagram showing an example of an interactive media session of an asymmetric channel structure according to an embodiment of the present invention.

[0025] Figure 5 The figure shows an example of receiving audio data mixed through a media session of a single channel structure according to an embodiment of the present invention.

[0026] Figure 6 The diagram shows an example of receiving audio data mixed in a media session through a multi-channel structure according to an embodiment of the present invention.

[0027] Figure 7 FIG. 1 is a flowchart showing an example of an audio control method of a server according to an embodiment of the present invention.

[0028] Figure 8 The flowchart is a diagram showing an example of a method for controlling audio of a client according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0030] The audio control system of multiple embodiments of the present invention can be implemented by at least one computer device, and the audio control method of multiple embodiments of the present invention can be executed by at least one computer device included in the audio control system. In this case, a computer program of an embodiment of the present invention can be installed and driven in the computer device, and the computer device can execute the audio control method of multiple embodiments of the present invention according to the control of the driven computer program. The above-mentioned computer program can be stored in a computer-readable recording medium to be combined with the computer device to execute the audio control method in the computer.

[0031] Figure 1 A diagram showing an example of a network environment according to an embodiment of the present invention. Figure 1 The network environment shown in FIG. 1 includes multiple electronic devices 110, 120, 130, 140, multiple servers 150, 160, and a network 170. Figure 1 This is an example for explaining the invention. The number of electronic devices or the number of servers is not limited to Figure 1 As shown. And, Figure 1 The network environment is only used to illustrate an example of an environment applicable to this embodiment, and the environment applicable to this embodiment is not limited to Figure 1 network environment.

[0032] The plurality of electronic devices 110, 120, 130, 140 may be fixed terminals or mobile terminals implemented by a computer device. For example, the plurality of electronic devices 110, 120, 130, 140 may include a smart phone, a mobile phone, a navigation, a computer, a notebook computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet computer, etc. As an example, in Figure 1 In the figure, the shape of a smartphone is shown as an example of the electronic device 110, but in the embodiment of the present invention, in essence, the electronic device 110 can be one of a variety of physical computer devices that communicate with other multiple electronic devices 120, 130, 140 and / or servers 150, 160 through a network 170 using wireless communication or wired communication.

[0033] The communication method is not limited, and may include not only the communication method using the communication network (for example, mobile communication network, wired Internet, wireless Internet, broadcast network) that the network 170 may include, but may also include short-range wireless communication between multiple devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Furthermore, the network 170 may include one or more of a network topology such as a bus network, a star network, a ring network, a mesh network, a star bus network, a tree network, or a hierarchical network, but is not limited thereto.

[0034] Each server 150, 160 may be implemented by a computer device or multiple computer devices that communicate with multiple electronic devices 110, 120, 130, 140 through a network 170 to provide instructions, codes, files, content, services, etc. For example, the server 150 may be a system that provides services (for example, voice conferencing services, messaging services, email services, social networking services, map services, translation services, financial services, payment services, search services, content provision services, etc.) to multiple electronic devices 110, 120, 130, 140 connected through the network 170.

[0035] Figure 2 1 is a block diagram showing an example of a computer device according to an embodiment of the present invention. The plurality of electronic devices 110, 120, 130, 140 or the plurality of servers 150, 160 described above can be respectively Figure 2 The computer device 200 shown is implemented.

[0036] like Figure 2As shown, such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240. The memory 210 is a computer-readable recording medium, and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Among them, the read only memory and the permanent mass storage device such as a disk drive may also be included in the computer device 200 as additional permanent storage devices different from the memory 210. In addition, an operating system and at least one program code may be stored in the memory 210. Such multiple software structural elements may be loaded into the memory 210 from a computer-readable recording medium independent of the memory 210. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy disk drive, a hard disk, a magnetic tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, multiple software structural elements may be loaded into the memory 210 through the communication interface 230, rather than being loaded into the memory 210 through the computer-readable recording medium. For example, the plurality of software components may be loaded into the memory 210 of the computer device 200 based on a computer program installed from a plurality of files received via the network 170 .

[0037] The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input-output operations. Instructions may be provided to the processor 220 via the memory 210 or the communication interface 230. For example, the processor 220 may be configured to execute received instructions according to program code stored in a storage device such as the memory 210.

[0038] The communication interface 230 can provide a function for the computer device 200 to communicate with other devices (for example, the multiple storage devices described in the above content) through the network 170. As an example, the processor 220 of the computer device 200 can transmit requests or instructions, data, files, etc. generated according to program codes stored in storage devices such as the memory 210 to other multiple devices through the network 170 under the control of the communication interface 230. Conversely, the computer device 200 can receive signals or instructions, data, files, etc. from other devices through the network 170 through the communication interface 230 of the computer device 200. The signals or instructions, data, etc. received through the communication interface 230 can be transmitted to the processor 220 or the memory 210, and the files, etc. can be stored in the storage medium (the above-mentioned permanent large-capacity storage device) that can also be included in the computer device 200.

[0039] The input / output interface 240 may be a unit for coupling with the input / output device 250. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display, a speaker, etc. As another example, the input / output interface 240 may be a unit for coupling with a device such as a touch screen that integrates the functions for input and output. The input / output device 250 may be formed as one body with the computer device 200.

[0040] Furthermore, in other embodiments, the computer device 200 may include Figure 2 The computer device 200 may include fewer or more structural elements. However, it is not necessary to explicitly show most of the structural elements of the prior art. For example, the computer device 200 may be implemented to include at least a portion of the input / output device 250 described above or may also include other structural elements such as a transceiver, a database, etc.

[0041] Figure 3 A diagram showing an example of the general state of an audio control system according to an embodiment of the present invention. The audio control system of this embodiment may be composed of a server 310 providing a voice conference service and a plurality of client devices 320 for a plurality of participants. In this case, the server 310 may include a mixer 311 to mix the audio data of the other plurality of participants participating in the same voice conference, and transmit the mixed audio data to the first participant participating in the voice conference. For example, a situation in which a first client device 321, a second client device 322, and a third client device 323 among a plurality of client devices 320 participate in a voice conference instance may be considered. In this case, the server 310 may mix the first audio data received from the first client device 321 and the second audio data received from the second client device 322 through the mixer 311, and transmit the mixed audio data to the third client device 323.

[0042] As an example, the mixer 311 may be a software module that generates and relays audio data between multiple participants participating in the corresponding voice conference instance in the server 310 for a voice conference instance. At this time, the mixer 311 may establish an interactive media session with each of the multiple participants participating in the corresponding voice conference instance. In particular, the mixer 311 of this embodiment may establish an interactive media session with an asymmetric channel structure, so that multiple participants can control individual audio data from the mixed audio data.

[0043] Figure 4 FIG. 1 is a diagram showing an example of an interactive media session of an asymmetric channel structure according to an embodiment of the present invention. Figure 4An example of establishing an interactive media session with an asymmetric channel structure between the mixer 311 and the third client device 323 is described. The third client device 323 only needs to transmit its own audio data to the mixer 311, so the media session from the third client device 323 to the mixer 311 can be established as a single-channel structure. On the other hand, in the case of receiving only mixed audio data, the media session from the mixer 311 to the third client device 323 can also be established as a single-channel structure. However, in this case, it is impossible to distinguish the individual audio data of multiple participants in the mixed single-channel audio data, and therefore, it is impossible to control the individual audio data according to the participants. Therefore, in this embodiment, in order to control the individual audio data, the media session from the mixer 311 to the third client device 323 can be established as a multi-channel structure. This interactive media session with an asymmetric channel structure can be established for client devices that want to control individual audio data. In other words, multiple client devices that do not want to control individual audio data can establish only a single-channel interactive media session with the mixer 311.

[0044] Figure 5 This is a diagram showing an example of receiving audio data mixed through a media session of a single channel structure according to an embodiment of the present invention. Figure 6 The diagram shows an example of receiving audio data mixed in a media session through a multi-channel structure according to an embodiment of the present invention.

[0045] Figure 5 The following example is shown, that is, the media session from the mixer 311 to the third client device 323 is established as a single channel structure, and the mixer 311 mixes the audio data of the first client device 321 and the audio data of the second client device 322 through the media session of the single channel structure and transmits it to the third client device 323. This embodiment shows the following example, that is, the mixer 311 can generate a third data packet 531 for a single channel by mixing the first data packet 511 from the first client device 321 and the second data packet 521 from the second client device 322, and transmit the generated third data packet 531 to the third client device 323. In addition, the mixer 311 generates a sixth data packet 532 for a single channel by mixing the fourth data packet 512 from the first client device 321 and the fifth data packet 522 from the second client device 322, and transmits the generated sixth data packet 532 to the third client device 323. In this case, since the mixer 311 transmits the third data packet 531 and the sixth data packet 532 mixed for a single channel, the third client device 323 cannot independently control the audio data from the first client device 321 and the audio data from the second client device 322 .

[0046] on the other hand. Figure 6 The following example is shown, that is, the media session from the mixer 311 to the third client device 323 is established as a multi-channel structure, and the mixer 311 mixes the audio data of the first client device 321 and the audio data of the second client device 322 through the media session of the multi-channel structure, and transmits it to the third client device 323. This embodiment shows the following example, that is, the mixer 311 generates a seventh data packet 611 for multiple channels by mixing the first data packet 511 from the first client device 321 and the second data packet 521 from the second client device 322, and transmits the generated seventh data packet 611 to the third client device 323. In addition, the mixer 311 generates an eighth data packet 612 for multiple channels by mixing the fourth data packet 512 from the first client device 321 and the fifth data packet 522 from the second client device 322, and transmits the generated eighth data packet 612 to the third client device 323. In this case, the mixer 311 transmits the seventh data packet 611 and the eighth data packet 612 mixed for multi-channels, and thus the third client device 323 can control the audio data from the first client device 321 and the audio data from the second client device 322 separately.

[0047] Figure 7 2 is a flowchart showing an example of an audio control method of a server according to an embodiment of the present invention. The audio control method of the embodiment of the present invention can be executed by a computer device 200 that implements a server for voice conferencing services. As an example, the server can be a subject for providing voice conferencing services. In this case, the processor 220 of the computer device 200 can be implemented to execute control instructions based on the code of the operating system or at least one program code contained in the memory 210. The processor 220 can control the computer device 200 so that the computer device 200 executes according to the control instructions provided by the code stored in the computer device 200. Figure 7 The method includes multiple steps (step 710 to step 740).

[0048] In step 710, the computer device 200 may establish an interactive media session with an asymmetric channel structure between a first participant among multiple participants participating in the voice conference and a mixer. The voice conference may correspond to a voice conference instance, and the mixer may be a software module generated by the computer device 200. In addition, the multiple participants may be multiple clients participating in the same voice conference instance among multiple clients identified by an account of the voice conference service.

[0049] In this case, the computer device 200 may establish an interactive media session with an asymmetric channel structure for all participants, but according to an embodiment, an interactive media session with an asymmetric channel structure may be established only for participants who want to individually control audio data related to each participant.

[0050] In one embodiment, the asymmetric channel structure may include a first channel structure and a second channel structure, wherein the first channel structure includes a single channel structure and the second channel structure includes a multi-channel structure. In other words, an interactive media session of an asymmetric channel structure in which the media session from the first participant to the mixer and the media session from the mixer to the first participant have different channel structures may be established between the first participant and the mixer. In this case, since the first participant transmits his own audio data to the mixer, the media session from the first participant to the mixer may be established as a media session of the first channel structure. Furthermore, since the mixer may transmit audio data of multiple participants to the first participant, the media session from the mixer to the first participant may be established as a media session of the second channel structure.

[0051] In order to establish such an interactive media session with an asymmetric channel structure, the computer device 200 may determine at least one of the channel type, channel coding method and processing unit in each direction by exchanging media processing information with the first participant.

[0052] The channel type is used to define the type and number of channels, and the channel type may include a fixed channel and a variable channel. A fixed channel may mean a type with a fixed number of channels. And, a variable channel may mean a type in which the number of channels can be changed. As an example, a variable channel that can change up to 15 channels can be used in a real-time transport protocol. In other words, in step 710, the computer device 200 may determine a channel type and a channel number among the fixed channel and the variable channel. If a separate channel type is not determined, "Fixed 1" with one fixed channel may be determined as a default.

[0053] Furthermore, in the case where the channel coding method is multi-channel, the coding method of the multi-channel in the payload can be defined. In the case where the channel type is "fixed 1", it indicates a single channel, so it is meaningless to determine the channel coding method. This channel coding method may include codec coding of multi-channel coding using a codec and iterative coding expressing multi-channels by a processing unit. In other words, in step 710, in the case where the media session of the second channel structure is established as multi-channel, the computer device 200 may determine a channel coding method of codec coding and iterative coding, the codec coding using a codec supporting multi-channel coding, and the iterative coding expressing multi-channels by a processing unit of voice data.

[0054] As described above, in order to use iterative coding, the processing unit can be determined as a basic processing unit of voice data. In other words, when iterative coding is used as a channel coding method, the computer device 200 can determine the processing unit for iteratively encoded voice data.

[0055] In step 720, the computer device 200 may receive audio data from the first participant through the media session of the first channel structure in the interactive media session. As described above, the first channel structure may include a single channel structure, which means that the channel type is "fixed 1". As described above, when the channel type is "fixed 1", it is meaningless to determine the channel coding mode, and therefore, the processing unit does not need to determine it.

[0056] In step 730, the computer device 200 may mix the audio data of at least two participants other than the first participant among the multiple participants into multiple channels. If there are only two participants in the example of the voice conference, multiple channels are meaningless, but in the case of three or more participants, since the first participant can receive audio data from more than two participants, multiple channels can be used. In this case, the computer device 200 may mix the audio data received from more than two participants other than the first participant into multiple channels. At this time, the mixed audio data may be encoded by the channel encoding method determined in the above step 710.

[0057] In step 740, the computer device 200 may transmit the audio data of at least two participants mixed into multiple channels through the media session of the second channel structure in the interactive media session to the first participant. The audio data mixed into multiple channels may be transmitted to the first participant through the media session of the second channel structure, in other words, through the media session of the multi-channel structure. In this case, since the audio data of the participants can be identified according to the channel, the first participant can control the audio data of other participants according to the participants.

[0058] For example, the computer device 200 can transmit the audio data of n (n is a natural number greater than or equal to 2) participants who are at least two participants among the multiple participants except the first participant through n channels as a multi-channel. In this case, the first participant can control the audio data of the other multiple participants except the first participant respectively. As another example, the at least two participants mentioned above can be classified into m (m is a natural number greater than or equal to 2) groups. In this case, the computer device 200 can transmit the audio data of the multiple participants mixed by group through m channels formed for the m groups. At this time, the first participant can control the audio data mixed by group according to the group.

[0059] On the other hand, the first participant and the above-mentioned mixer can communicate according to the real-time transport protocol. At this time, when the media session of the second channel structure is established as a multi-channel, the order of the channels can be represented by the order of the synchronization sources appearing in the predefined action sources according to the real-time transport protocol, and the number of channels can be represented by the count of the action sources further defined according to the above-mentioned real-time transport protocol. The synchronization source is the source of audio data, in other words, it can mean multiple participants who want to transmit audio data to the first participant, and the action source can mean a list of such synchronization sources. That is, the order of the synchronization sources appearing in the action source can correspond to the order of the channels. In addition, the count of the action source represents the number of synchronization sources, and such a count of the action source can correspond to the number of channels.

[0060] Figure 8 1 is a flowchart showing an example of an audio control method of a client according to an embodiment of the present invention. The audio control method of this embodiment may be executed by a computer device 200 that implements a client for a voice conference service. As an example, a server may be a subject participating in a voice conference instance. In this case, the processor 220 of the computer device 200 may be implemented to execute control instructions based on an operating system code or at least one program code contained in the memory 210. The processor 220 may control the computer device 200 so that the computer device 200 executes the control instructions provided by the code stored in the computer device 200. Figure 8 The method includes multiple steps (step 810 to step 840).

[0061] In step 810, the computer device 200 may establish an interactive media session having an asymmetric channel structure between a mixer included in a server providing a voice conference and a first participant. Figure 7 Establishing an interactive media session with an asymmetric channel structure between the mixer and the first participant has been described in detail. The mixer and the first participant can determine at least one of a channel type, a channel coding method, and a processing unit by exchanging media processing information such as a codec.

[0062] In step 820, the computer device 200 may transmit the audio data of the first participant to the mixer through the media session of the first channel structure in the interactive media session. As described above, the first channel structure may include a single channel structure. The computer device 200 of this embodiment, which is the client device of the first participant, may transmit the audio data of the first participant to the mixer through the media session of the single channel structure.

[0063] In step 830, the computer device 200 may receive audio data of at least two participants mixed into multiple channels through a media session of a second channel structure in an interactive media session from a mixer that mixes audio data of at least two participants other than the first participant among multiple participants into multiple channels. The second channel structure may include a multi-channel structure. At this time, the computer device 200 may receive multi-channel audio data of audio data for more than two participants through such a media session of a multi-channel structure. For example, the computer device 200 may receive audio data of n (n is a natural number greater than or equal to 2) participants as at least two participants through n channels as multiple channels. As another example, the computer device 200 may also divide the more than two participants into more than two groups and receive audio data corresponding to each group through a media session of a multi-channel structure. For example, when considering the instance A of the voice conference in which the first participant, the second participant, the third participant, the fourth participant, the fifth participant, the sixth participant, the seventh participant, and the eighth participant participate, the first participant can set a group one including the second participant, the third participant, and the fourth participant and a group two including the fifth participant, the sixth participant, the seventh participant, and the eighth participant. In this case, the server establishes an association with the instance A of the voice conference to receive information related to the group one and information related to the group two from the first participant, and the mixer included in the server can establish a media session for the first participant by establishing an association with the instance A of the voice conference, and the above media session includes a first channel for mixing and transmitting audio data of multiple participants in group one (the second participant, the third participant, and the fourth participant) and a second channel for mixing and transmitting audio data of multiple participants in group two (the fifth participant, the sixth participant, the seventh participant, and the eighth participant). In this case, the first participant can control the audio data of group one and the audio data of group two separately in the instance A of the voice conference. The above description can easily understand the following embodiment, that is, setting other multiple participants into three or more groups and controlling the audio data of each group through three or more multi-channels.

[0064] In step 840, the computer device 200 may individually control the audio data of the channel selected from the multi-channel. For example, the computer device 200 may control the volume of the audio data of the selected channel, eliminate the noise of the audio data of the selected channel, or modulate the audio data of the selected channel. To this end, the computer device 200 may provide a user interface to the user (the first participant), and the user interface includes at least one of the functions of controlling the volume of the audio data of the selected channel, eliminating the noise of the audio data of the selected channel, and modulating the audio data of the selected channel.

[0065] As described above, according to various embodiments of the present invention, audio mixing can be used in a voice conference, and users in the audience position can control the audio of each audio data source.

[0066] The system or device described above can be implemented by a combination of hardware structural elements or hardware structural elements and software structural elements. For example, the device and structural elements described in the embodiment can be implemented using one or more general-purpose computers or special-purpose computers (such as processors, controllers, arithmetic logic units (arithmetic logic units, ALU), digital signal processors (digital signal processors), microcomputers, field programmable gate arrays (field programmable gate arrays, FPGA), programmable logic units (programmable logic units, PLU), microprocessors or any device that executes and responds to instructions). The processing device can execute an operating system (OS) and one or more applications executed on the above operating system. In addition, the processing device can also access, store, operate, process and generate data in response to the execution of software. For ease of understanding, sometimes the case of using a processing device is described, as long as the technicians in this technical field can understand that the processing device may include multiple processing elements (processing element) and / or multiple types of processing elements. For example, the processing device may include multiple processors or a processor and a controller. In addition, other processing configurations (processing configurations) such as parallel processors (parallel processors) may also be included.

[0067] Software may include a computer program, code, instructions, or a combination of more than one of them, which can be used to configure a processing device in a manner that allows it to operate in a desired manner or to deliver instructions to a processing device independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device in order to be parsed by or provide instructions or data to a processing device. Software may be distributed across computer systems connected via a network to be stored or run in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

[0068] The method of the embodiment can be implemented in the form of program instructions that can be executed by various computer units and recorded on a computer-readable medium. The above-mentioned computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium can continue to store programs that can be executed by a computer, or can temporarily store programs that can be executed by a computer for execution or download. In addition, the medium can be a variety of recording units or storage units in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a certain computer system, and can be dispersed on the network. As an example of a medium, there can be a medium composed of a magnetic medium such as a hard disk, a floppy disk, and a tape, an optical recording medium such as a CD-ROM, a DVD, a magneto-optical medium such as a floptical disk, and a read-only memory (ROM), a random access memory (RAM), a flash memory, etc., which is composed of a medium in a manner of storing program instructions. In addition, as an example of other media, an application store that circulates application programs, or a recording medium or storage medium managed by a website, a server, etc. that supplies or circulates other various software can be listed. The program instructions include not only machine language codes formed by a compiler but also high-level language codes executable by a computer using an interpreter or the like.

[0069] As described above, the embodiments are described by limited embodiments and drawings, and those skilled in the art can make various modifications and variations through the above descriptions. For example, the described techniques can be performed in a different order from the described methods and / or the structural elements of the described systems, structures, devices, circuits, etc. can be combined or combined in a different form from the described methods, or even if they are replaced or substituted by other structural elements or equivalents, appropriate results can be achieved.

[0070] Therefore, other examples, other embodiments and solutions within the scope equivalent to the scope of protection claimed by the invention also fall within the scope of protection claimed by the invention.

Claims

1. An audio control method for a computer device comprising at least one processor, characterized in that: include: The step of establishing, by means of the at least one processor, an interactive media session having an asymmetric channel structure between a first participant among a plurality of participants in the voice conference and a mixer; The step of utilizing the at least one processor to receive audio data from the first participant via a media session of a first channel structure in the interactive media session; A step of mixing audio data of at least two participants other than the first participant among the plurality of participants into multiple channels using the at least one processor; and The step of transmitting, using the at least one processor, the audio data of the at least two participants mixed into the multi-channel to the first participant via the media session of the second channel structure in the interactive media session, In the step of establishing the interactive media session, a channel type and a channel number of a fixed channel and a variable channel are determined, and The first channel structure corresponds to the fixed channel, and the second channel structure corresponds to the variable channel.

2. The audio control method according to claim 1, wherein: The first channel structure includes a single-channel structure, and the second channel structure includes a multi-channel structure.

3. The audio control method according to claim 1, wherein: In the step of establishing the interactive media session, at least one of the channel type, channel coding mode, and processing unit in each direction is determined by exchanging media processing information with the first participant.

4. The audio control method according to claim 1, wherein: In the step of establishing the above-mentioned interactive media session, when the media session of the above-mentioned second channel structure is established as multi-channel, a channel coding method is determined, one of codec coding and iterative coding, the above-mentioned codec coding uses a codec supporting multi-channel coding, and the above-mentioned iterative coding expresses multiple channels through a processing unit of voice data.

5. The audio control method according to claim 1, wherein: In the step of establishing the interactive media session, when iterative coding is used as the channel coding method, a processing unit for the iteratively coded voice data is determined.

6. The audio control method according to claim 1, characterized in that: The first participant communicates with the mixer via a real-time transport protocol. In the case where the media session of the second channel structure is established as multiple channels, the order of the channels is indicated by the order of the synchronization sources appearing in the active sources, and the number of channels is indicated by the count of the active sources. The order of the synchronization sources appearing in the active sources and the count of the active sources are defined according to the real-time transport protocol.

7. The audio control method according to claim 1, wherein: In the transmitting step, the audio data of n participants, which are the at least two participants, are transmitted through n channels, which are the multi-channels, where n is a natural number greater than or equal to 2.

8. The audio control method according to claim 1, wherein: The at least two participants are divided into m groups, where m is a natural number greater than 2. In the transmitting step, the audio data of the plurality of participants mixed in groups are transmitted through the m channels formed for the m groups.

9. An audio control method for a computer device comprising at least one processor, characterized in that: include: The step of establishing, by means of the at least one processor, an interactive media session having an asymmetric channel structure between a mixer included in a server for providing a voice conference and a first participant among a plurality of participants participating in the conference; The step of utilizing the at least one processor to transmit the audio data of the first participant to the mixer via the media session of the first channel structure in the interactive media session; The at least one processor receives, from the mixer for mixing the audio data of at least two participants among the plurality of participants except the first participant into multiple channels, the audio data of the at least two participants mixed into the multiple channels through the media session of the second channel structure in the interactive media session; as well as using the at least one processor to individually control the audio data of a channel selected from the plurality of channels; In the step of establishing the interactive media session, a channel type and a channel number of a fixed channel and a variable channel are determined, and The first channel structure corresponds to the fixed channel, and the second channel structure corresponds to the variable channel.

10. The audio control method according to claim 9, characterized in that: The first channel structure includes a single channel structure. The above-mentioned second channel structure includes a multi-channel structure.

11. The audio control method according to claim 9, characterized in that: In the step of establishing the interactive media session, at least one of the channel type, channel coding mode and processing unit in each direction is determined by exchanging media processing information with the mixer.

12. The audio control method according to claim 9, characterized in that: In the step of receiving audio data, audio data of n participants, which are the at least two participants, are received through n channels, which are the multi-channels, where n is a natural number greater than or equal to 2.

13. The audio control method according to claim 9, characterized in that: The at least two participants are divided into m groups, where m is a natural number greater than 2. In the step of receiving audio data, audio data of the at least two participants mixed in groups are received through the m channels formed for the m groups.

14. The audio control method according to claim 9, characterized in that: The method further includes providing a user interface, wherein the user interface includes at least one of a function of controlling the volume of the audio data of the selected channel, a function of eliminating noise of the audio data of the selected channel, and a function of modulating the audio data of the selected channel.

15. The audio control method according to claim 9, characterized in that: In the individual controlling step, the volume of the audio data of the selected channel is controlled, noise of the audio data of the selected channel is eliminated, or the audio data of the selected channel is modulated.

16. A computer-readable recording medium, characterized in that A computer program for executing the method according to any one of claims 1 to 15 in a computer device is recorded.

17. A computer device, characterized in that: comprising at least one processor implemented to execute computer-readable instructions, Using the at least one processor, an interactive media session having an asymmetric channel structure is established between a first participant among multiple participants in the voice conference and a mixer. Audio data is received from the first participant via a media session having a first channel structure in the interactive media session. Audio data of at least two participants other than the first participant among the multiple participants are mixed into multiple channels. The audio data of the at least two participants mixed into the multiple channels is transmitted to the first participant via a media session having a second channel structure in the interactive media session. In the step of establishing the interactive media session, a channel type and a channel number of a fixed channel and a variable channel are determined, and The first channel structure corresponds to the fixed channel, and the second channel structure corresponds to the variable channel.

Images