Conference system, conference processing device, and program

Abstract

We provide a conferencing system that allows the recipient to select the target audio source on their communication terminal. [Solution] In a web conferencing system, the speaker's communication terminal 100 includes a sound collection means for collecting sounds emitted from multiple sound sources, a sound information generation means for decomposing input sound information consisting of input sounds from the sound collection means based on feature quantities and generating each sound piece, an identification information adding means for adding identification information to each of the sound pieces that makes the multiple sound pieces uniquely identifiable, and a transmission means for transmitting synthesized sound information, which is a combination of the sound pieces to which the identification information has been added, to a second communication terminal. The receiver's communication terminal 200 includes a receiving means for receiving synthesized sound information from the communication terminal 100, a display control means for displaying a sound selection image associated with the identification information added to each sound piece included in the synthesized sound information on a display unit, a sound output means for outputting a sound based on a selection instruction for the sound selection image, and a control means for changing the sound output from the sound output means based on the selection instruction.

Description

TECHNICAL FIELD

[0001] The present invention relates to a conference system, a conference processing device, and a program. BACKGROUND ART

[0002] In conference systems such as audio conferences, video conferences, and web conferences, there is a known technique that removes unnecessary audio from the audio transmitted and received between a plurality of communication terminals, and transmits target audio (audio desired to be heard) from the transmitting side (speaker) to the receiving side (listener) (for example, see Patent Document 1). SUMMARY OF THE INVENTION PROBLEM TO BE SOLVED BY THE INVENTION

[0003] However, the conventional technique has a problem that it is not possible to select audio other than the target audio from the audio received by the listener side, so as to achieve a mode that does not output unnecessary audio and unnecessary sounds.

[0004] An object of the present invention is to provide a conference system that allows a communication terminal on the receiving side to select target audio (audio desired to be heard). MEANS FOR SOLVING THE PROBLEM

[0005] To solve the above problems, one aspect of the present invention is a conference system that enables a conference to be held by connecting multiple information processing terminals in a communicative manner, wherein, among the participants of the conference, the terminal on the speaker's side is designated as the first communication terminal and the terminal on the receiver's side is designated as the second communication terminal, the first communication terminal comprises: a sound collection means for collecting sounds emitted from multiple sound sources; a sound information generation means for decomposing the input sound information consisting of the sounds input from the sound collection means based on the characteristic quantities of each sound and generating sound information related to each sound; and identification information that enables unique identification of the multiple sound pieces of information, attached to each of the sound pieces of information. The device comprises an identification information adding means and a transmission means for transmitting synthesized sound information obtained by synthesizing each sound piece to which the identification information has been added to the second communication terminal, wherein the second communication terminal comprises a receiving means for receiving the synthesized sound information from the first communication terminal, a display control means for displaying a sound selection image associated with the identification information added to each sound piece included in the synthesized sound information on a display unit, a sound output means for outputting the sound based on a selection instruction for the sound selection image, and a control means for changing the sound output from the sound output means based on the selection instruction. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a conferencing system that allows the user to select the target audio (the audio they want to hear) on the receiving end of the communication terminal. [Brief explanation of the drawing]

[0007] [Figure 1] A schematic diagram showing an example of a web conferencing system according to an embodiment of the present invention. [Figure 2] A schematic diagram showing an example of audio data processing in a web conferencing system according to an embodiment of the present invention. [Figure 3] A block diagram showing an example of the hardware configuration of a communication terminal. [Figure 4] A functional block diagram showing an example of a web conferencing system according to an embodiment of the present invention. [Figure 5] An explanatory diagram showing an example of how the selected image will be displayed. [Figure 6]A sequence diagram showing the process from the receiver's communication terminal to the output of the target audio. [Figure 7] A functional block diagram showing another example of a web conferencing system according to an embodiment of the present invention. [Figure 8] Sequence diagram based on the configuration in Figure 7. [Figure 9] An explanatory diagram showing another example of a sound selection image. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. In the following embodiments, a web conferencing system 700 will be used as an example of a conferencing system.

[0009] <Overall Overview of Web Conferencing System 700> Figure 1 is an overall schematic diagram showing an example of a web conferencing system 700 according to an embodiment of the present invention.

[0010] Figure 1 shows a remote conference being held using the web conferencing system 700 between user A, who is at location P1, and user B, who is at location P2, which is far from location P1. For convenience, in the following explanation, user A will be referred to as "speaker A" and user B as "receiver B". Speaker A uses communication terminal 100 as the first communication terminal, and receiver B uses communication terminal 200 as the second communication terminal. Figure 1 illustrates how speaker A and receiver B participate in a web conference using their respective communication terminals. In other words, Figure 1 illustrates how multiple information processing terminals can be connected in a communicative manner to enable the holding of a conference.

[0011] Communication terminals 100 and 200 are, for example, notebook PCs (personal computers), desktop PCs, mobile phones, smartphones, tablet devices, wearable PCs, etc.

[0012] Communication terminal 100 and communication terminal 200 can communicate via a communication network 1000, such as the Internet. The communication network 1000 may be one or more LANs (Local Area Networks) inside a firewall, or it may include the Internet outside the firewall in addition to the LANs. It may also include VPNs (Virtual Private Networks) or Wide Area Ethernet (registered trademark). The communication network 1000 may be constructed using either wired or wireless connections. Furthermore, the communication network 1000 may be a combination of wired and wireless connections. In addition, if connected via a mobile phone network configured with standards such as LTE (Long Term Evolution), 4G, or 5G, a LAN is not required.

[0013] <Overview of audio data processing for Web Conferencing System 700> Figure 2 is a schematic diagram showing an example of sound data processing that can be performed in a web conferencing system 700 according to an embodiment of the present invention.

[0014] In general terms, the present invention provides a way to transmit sound (including the voices of speaker A and people around speaker A) from speaker A's communication terminal 100 to receiver B's communication terminal 200, where only speaker A's voice remains from the transmitted sound mixed with speaker A's voice, and other unwanted sounds are removed before output. Specifically, receiver B, using the communication terminal 200, can select which unwanted sounds to remove, and the system will remove the unwanted sounds selected by receiver B, allowing the system to output only the sound that receiver B wants to hear (the target sound). Alternatively, only the target sound may be output, or the output level of the target sound may be increased while the level of the selected unwanted sounds is lowered before output. In other words, the system is characterized by the fact that receiver B can customize the system so that the target sound (the sound they want to hear) is closer to their desired output by selecting it from the display screen 2050 of the communication terminal 200.

[0015] For example, as shown in Figure 2, when the voice of speaker A is transmitted from communication terminal 100 to communication terminal 200, the sound collection means, such as the microphone built into communication terminal 100 or an external microphone connected to communication terminal 100, will also pick up sounds other than speaker A's voice. "Sounds other than speaker A's voice" include, for example, the voice of participant C who is participating in the meeting with speaker A at base P1, or sounds from outside base P1 Z (outdoor sounds of base P1, vehicle sounds, work sounds, etc.). It is difficult for speaker A to know how much of these "sounds other than speaker A's voice" are reaching receiver B, and to what extent they are being output.

[0016] In this embodiment, the input sound S0 is sound collected by the sound collection means of the communication terminal 100 and emitted from multiple sound sources, for example, the voice of speaker A, the voice of participant C, and the sound of external Z. The communication terminal 100 analyzes the input sound S0 to identify the characteristics of each sound (e.g., sound pressure, frequency characteristics, etc.), and based on the quantities of the identified characteristics of each sound (feature quantities), it decomposes the input sound S0 and generates multiple sound data (voice data S1 of speaker A, voice data S2 of participant C, and voice data S3 of external Z). The input sound S0 is an example of input sound information.

[0017] Then, identification data is added to each of the multiple audio data generated in the communication terminal 100 (speaker A's audio data S1, participant C's audio data S2, and external Z's audio data S3) as identification information to identify each of them. The identification data includes an identification code; for example, as shown in Figure 2, the identification code "D001" is added to speaker A's audio data S1, the identification code "D002" is added to participant C's audio data S2, and the identification code "D003" is added to external Z's audio data S3. This allows each of the audio data S1, S2, and S3 to be managed in association with the identification data.

[0018] As described above, the communication terminal 100 generates audio data S1, S2, S3 from the input sound S0, and adds identification data (such as identification codes D001 to D003) to the audio data S1, S2, S3. Then, synthesized sound information obtained by synthesizing the audio data S1 to S3 and the identification data is transmitted from the communication terminal 100 to the communication terminal 200. When transmitting the audio data S1 to S3 and the identification data from the communication terminal 100 to the communication terminal 200, these pieces of data are re-synthesized as shown in FIG. 2, and re-synthesized data S0' as the synthesized sound information is transmitted to the communication terminal 200.

[0019] The analysis of the input sound S0 executed in the communication terminal 100 uses, for example, spectrum analysis that decomposes the frequency components of the sound and analyzes the level (volume, sound pressure) of each frequency component.

[0020] Upon receiving the sound data and the identification data from the communication terminal 100, the communication terminal 200 displays a sound selection image created based on the sound data and the identification data. FIG. 5 is an example of a display screen 2050 including a sound selection image 2053 as the sound selection image. The display screen 2050 is an example of a screen displayed on the display unit of the communication terminal 200 which serves as another information processing terminal. As shown in FIG. 5, the display screen 2050 includes and displays a sound selection image 2053 serving as a sound selection button or a sound selection switch, which is used when the listener B using the communication terminal 200 selects target sound (or selects unnecessary sound).

[0021] As illustrated in Figure 5, the display screen 2050 of the communication terminal 200 displays an image of a sound selection button as a sound selection means associated with identification data. When the listener B operates the sound selection button, the communication terminal 200 controls itself so as not to output the sound data associated with that sound selection button. Alternatively, the communication terminal 200 may control itself so as to output only the sound related to the sound data associated with that sound selection button. In other words, the communication terminal 200 can be made to selectively output only the target sound. Note that instead of outputting only the target sound, the output level of the target sound may be increased so that the listener B can easily distinguish the target sound from other sounds and hear it. Alternatively, the level of sounds other than the target sound may be lowered so that the listener B can easily distinguish the target sound from other sounds and hear it. Furthermore, the output level of the target sound may be increased and the level of sounds other than the target sound may be lowered so that the listener B can easily distinguish the target sound from other sounds and hear it. The above listening environment can be realized by the recipient B making a selection regarding the communication terminal 200.

[0022] <Hardware configuration of communication terminals 100 and 200> Figure 3 is a block diagram showing an example of the hardware configuration of communication terminal 100 and communication terminal 200. Since the hardware configurations of communication terminal 100 and communication terminal 200 are the same, the components and parts related to communication terminal 100 are assigned numbers in the 100s, and the components and parts related to communication terminal 200 are assigned numbers in the 200s, with the 100s numbers indicated in parentheses below.

[0023] Communication terminals 100 and 200 are implemented by a computer with the configuration shown in the diagram. Communication terminal 100 (200) includes a CPU (Central Processing Unit) 101 (201) as a control means, ROM (Read Only Memory) 102 (202), RAM (Random Access Memory) 103 (203), HDD (Hard Disk Drive) 104 (204), display device 105 (205), input device 106 (206), sound input / output device 107 (207), communication I / F (Interface) 108 (208), external I / F 109 (209), and bus line 110 (210).

[0024] The CPU 101(201) controls the overall operation of each component in the communication terminal 100(200). The CPU 101(201) reads programs and data stored in the ROM 102(202) and HDD 104(204) from the ROM 102(202) and HDD 104(204) into the RAM 103(203) and executes the processing.

[0025] ROM102(202) is a non-volatile semiconductor memory that stores programs and data (BIOS, OS settings, network settings, etc.).

[0026] RAM103(203) is a volatile semiconductor memory that temporarily stores programs and data.

[0027] HDD104 (204) is a non-volatile storage device that stores programs and data (OS, applications, etc.).

[0028] The display device 105 (205) consists of a display, etc., and displays various information and processing results such as a cursor, menu, window, text or images.

[0029] The input device 106 (206) consists of a keyboard, mouse, touch panel, etc., and accepts input from the user.

[0030] The audio input / output device 107 (207) consists of a microphone, speaker, headset, etc. The microphone is a type of sound collection means that inputs sound signals according to the control of the CPU 101 (201). The speaker is a type of playback means that outputs sound signals according to the control of the CPU 101 (201).

[0031] Communication I / F108(208) is an interface for data communication using communication network 1000.

[0032] External I / F 109 (209) is an interface for connecting to external devices. External devices include cameras and other imaging devices, recording devices, and recording media.

[0033] The recording medium 111 (211) is a flexible disk, CD, DVD, SD memory card, USB memory, etc.

[0034] Bus lines 110 (210) are address buses, data buses, etc., for electrically connecting the above-mentioned components, and transmit address signals, data signals, and various control signals.

[0035] <Functional Configuration> Figure 4 is a functional block diagram showing an example of a web conferencing system 700 according to an embodiment of the present invention.

[0036] The communication terminal 100 is an example of a first communication terminal and corresponds to the communication terminal used by speaker A in this embodiment. The communication terminal 100 includes a transmitting / receiving unit 151, an input receiving unit 152, an audio input / output unit 153, an audio information generation unit 154, an identification information addition unit 155, a display control unit 156, a determination unit 157, and a storage / reading unit 158. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 3 operating according to instructions from the CPU 101 in accordance with a program deployed on the RAM 103.

[0037] Furthermore, the communication terminal 100 includes a storage unit 159 constructed from a ROM 102 or HDD 104, as shown in Figure 3. The storage unit 159 mainly stores information used by the sound information generation unit 154 and the identification information addition unit 155.

[0038] The information used by the sound information generation unit 154 is auxiliary information that assists the analysis process for the sound information generation unit 154. For example, if the voice data of speaker A is registered (recorded) in advance, that registered (recorded) voice data corresponds to auxiliary information. In addition, voice data other than that of speaker A, which relates to ambient sounds that are easily picked up at the location where speaker A uses the communication terminal 100 (voice data relating to other people's voices or sound data corresponding to noise in the external environment), also corresponds to auxiliary information. When such auxiliary information is stored in the memory unit 159, the sound information generation unit 154 can extract and output the characteristic quantities of speaker A's voice while comparing them with the auxiliary information, thereby improving the accuracy of decomposition (separation) of speaker A's voice data.

[0039] Furthermore, when the sound information generation unit 154 performs the process of decomposing a sound into multiple sounds, it also performs the spectrum analysis process described later.

[0040] The information used in the identification information addition unit 155 is the information used when adding identification data to each of multiple sound data, and includes, for example, information related to voiceprint authentication, sound pressure (dB), frequency characteristics (Hz), etc.

[0041] The transmitting / receiving unit 151 is mainly implemented by the processing of the communication interface 108 by the CPU 101, and enables communication with the communication terminal 200 via the communication network 1000.

[0042] The transmitting / receiving unit 151 transmits information, such as requests from speaker A received by the input receiving unit 152, to the communication terminal 200, and receives information transmitted from the communication terminal 200. Thus, the transmitting / receiving unit 151 is an example of a transmitting means, and the transmitting / receiving unit 151 is also an example of a receiving means.

[0043] The input receiving unit 152, which is an input receiving means that receives input operations from the user (speaker A) of the communication terminal 100 to the display screen of the communication terminal 100, is mainly realized by the processing of the CPU 101 on the input device 106 and receives input from speaker A to the input device 106.

[0044] The sound input / output unit 153 is primarily realized by the processing performed by the CPU 101 on the sound input / output device 107. The sound collection means, such as a microphone, which constitutes the sound input / output device 107, captures the voice of speaker A speaking using the microphone, as well as sounds from its surroundings, and inputs the sound signal to the communication terminal 100 according to the control of the CPU 101. The playback means, such as a speaker, which constitutes the sound input / output device 107, outputs the sound signal according to the control of the CPU 101. Thus, the sound input / output unit 153 is an example of a sound input means, and the sound input / output unit 153 is also an example of a sound output means. The control processing of sound signal input and output performed using the sound input / output unit 153 corresponds to sound output processing.

[0045] The sound information generation unit 154, as a means for generating sound information, is mainly realized by the processing of the CPU 101. For example, the sound information generation unit 154 performs spectral analysis on the sound signal converted by the sound input / output unit 153 from the sound captured by the microphone of the sound input / output device 107. Through spectral analysis, the sound signal is decomposed (separated) into specific frequency groups. The sound information generation unit 154 then decomposes (separates) the frequency group into a frequency group corresponding to the voice of speaker A and a frequency group corresponding to sounds other than the voice of speaker A. In this way, the sound information generation unit 154 performs analysis processing to separate and decompose the specific sound captured by the microphone of the sound input / output device 107 from other sounds. The spectral analysis processing performed by the sound information generation unit 154 may be configured to use pre-registered sound data as auxiliary information if there is sound data pre-registered in the storage unit 159. As a result, multiple sound data decomposed (separated) based on feature quantities (frequency in the above case) are generated.

[0046] The identification information addition unit 155, which serves as a means for adding identification information, is mainly implemented by the processing of the CPU 101. The identification information addition unit 155 processes each sound generated by the sound information generation unit 154, for example. The data is associated with information stored in the memory unit 159, such as voiceprint authentication, sound pressure (dB), and frequency characteristics (Hz), and an identification code is added. When the identification information addition unit 155 handles information related to frequency characteristics, it performs spectrum analysis on the sound data generated by the sound information generation unit 154, divides the sound data based on the level (volume, sound pressure) of the sound components included in a predetermined frequency domain (frequency band), and adds identification information to each sound data.

[0047] The display control unit 156 is mainly implemented by the processing performed by the CPU 101 on the display device 105, and causes the display device 105 to display various display screens.

[0048] The decision unit 157 is mainly implemented by the CPU 101 and performs various decisions.

[0049] The storage and reading unit 158 ​​is mainly implemented by the CPU 101 and performs operations such as storing various data in the storage unit 159 and reading various data from the storage unit 159.

[0050] Since the functional configuration of the communication terminal 200 is the same as that of the communication terminal 100, we will explain the differences between them.

[0051] The communication terminal 200 is an example of a second communication terminal, and in this embodiment, it is the communication terminal used by the receiver B. The communication terminal 200 comprises a transmitting / receiving unit 251, an input receiving unit 252, an audio input / output unit 253, a display control unit 256, a determination unit 257, and a storage / reading unit 258. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 3 operating according to instructions from the CPU 201 in accordance with a program deployed on the RAM 203.

[0052] The transmitting / receiving unit 251 is mainly implemented by the processing of the communication interface 208 by the CPU 201, and enables communication with the communication terminal 100 via the communication network 1000.

[0053] The transmitting / receiving unit 251 transmits information, such as requests from the receiver B received by the input receiving unit 252, to the communication terminal 100, and receives information transmitted from the communication terminal 100. Thus, the transmitting / receiving unit 251 is an example of a transmission means, and the transmitting / receiving unit 251 is also an example of a reception means. The information reception process performed by the transmitting / receiving unit 251 corresponds to the reception process.

[0054] The audio input / output unit 253 is primarily implemented by the processing performed by the CPU 201 on the audio input / output device 207. The playback means, such as speakers, that constitute the audio input / output device 207 output sound signals according to the control of the CPU 201. The audio input / output unit 253 is both an example of an audio input means and an example of an audio output means.

[0055] The display control unit 256 is primarily implemented by the CPU 201's processing of the display device 205, and causes the display device 205 to display various display screens, such as the selection screen described later. The display control unit 256 is an example of a display control means. Furthermore, the processing related to displaying the selection screen and other screens on the display device 205, which is performed using the display control unit 256, corresponds to display control processing.

[0056] <Audio Selection Image> The sound selection image 2053 displayed on the communication terminal 200 will be explained using Figure 5. Figure 5 is an explanatory diagram showing an example of the sound selection image 2053 display.

[0057] This example shows a display screen 2050 that is displayed on the display device 205 of the communication terminal 200 of the recipient B during a web conference. The display screen 2050 has a first frame 2051 as a first display area that displays information received from the communication terminal 100, and a second frame 2052 as a second display area that displays information transmitted from the communication terminal 200.

[0058] When the display control unit 256 of the communication terminal 200 receives sound data and identification data from the communication terminal 100 via the communication network 1000, it causes the display device 205 to display a sound selection image 2053, which includes a voice selection means corresponding to the received sound data and identification data. Specifically, the sound selection image 2053 is displayed in the first frame 2051 of the display screen 2050.

[0059] The illustrated sound selection image 2053 shows the case where the sound data is decomposed (separated) into three parts. In the sound selection image 2053, for example, sound selection button SW1, as a sound selection means, is associated with the voice of speaker A in Figure 2 (voice data S1, identification code D001), sound selection button SW2 is associated with the voice of participant C in Figure 2 (voice data S2, identification code D002), and sound selection button SW3 is associated with the voice of external Z in Figure 2 (voice data S3, identification code D003). These sound selection buttons SW1 to SW3 can be operated on the communication terminal 200.

[0060] For example, if receiver B selects (turns on) sound selection button SW1 by using the cursor on the display screen 2050, the voice of sender A associated with it will be excluded (not output), and the voices of participant C and external Z will be output from the speaker of the communication terminal 200 (or another speaker connected to the communication terminal 200). If the target voice (the voice to be heard) is "speaker A's voice," receiver B can select (turns on) sound selection buttons SW2 and SW3, and deselect (turns off) sound selection button SW1 to output speaker A's voice from the speaker of the communication terminal 200.

[0061] The order in which the sound selection buttons SW1 to SW3 are arranged is not particularly limited, but from the viewpoint of ease of operation when the listener B narrows down the target sound, it is preferable that the sound selection buttons SW1 to SW3 are arranged in order of the characteristic quantities of the sound, for example, in order of the sound pressure of multiple sound information (multiple sound data).

[0062] The relationship between the ON / OFF states of the aforementioned sound selection buttons SW1-SW3 and the audio output prioritizes the idea of ​​"selecting the sound you want to mute." On the other hand, there is also the idea of ​​"selecting the sound you want to hear," in which case the sound associated with the selected (ON) sound selection button may be output from the speaker.

[0063] <Sequence of events from when the recipient B's communication terminal 200 outputs the target audio> Figure 6 is a sequence diagram showing the process from when the recipient B's communication terminal 200 outputs the target audio.

[0064] The input receiving unit 152 of the communication terminal 100 receives an instruction to start the web conferencing system 700 based on a predetermined input operation to the input device 106 of the speaker A (S601). When the input receiving unit 152 receives an instruction to start the web conferencing system 700, the CPU 101 starts the web conferencing system 700, and the display control unit 156 displays the web conferencing screen on the display device 105.

[0065] Similarly, the input receiving unit 252 of the communication terminal 200 receives an instruction to start the web conferencing system 700 based on a predetermined input operation to the input device 206 of the receiver B (S602). When the input receiving unit 252 receives the instruction to start the web conferencing system 700, the CPU 201 starts the web conferencing system 700, and the display control unit 256 displays the web conferencing screen on the display device 205. Steps S601 and S602 do not necessarily have to be performed in this order. For example, the web conferencing system 700 on the communication terminal 200 may be started first.

[0066] In the communication terminal 100, after the web conferencing system 700 is started, the input sound is analyzed as shown in Figure 2, that is, the input sound S0 collected by the sound input / output unit 153 is subjected to "decomposition" by the sound information generation unit 154 and "identification data addition" by the identification information addition unit 155 (S603). The input sound analysis in step S603 is performed for a predetermined time (tens of seconds to several minutes) from the start of the web conferencing system 700, and as a result, audio data and identification data are created. Here, the "decomposition" and other processes performed by the sound information generation unit 154 are an example of audio information generation processing, and the "identification data addition" and other processes performed by the identification information addition unit 155 are an example of identification information addition processing.

[0067] Once the analysis of the input sound in step S603 is complete, the audio data and identification data created in step S603 are transmitted from the transmitting / receiving unit 151 of the communication terminal 100 to the communication terminal 200 via the communication network 1000 (S604), and the transmitting / receiving unit 251 of the communication terminal 200 receives (acquires) the audio data and identification data transmitted from the communication terminal 100 (S605). Here, the "transmission" process performed by the transmitting / receiving unit 151 is an example of a transmission process.

[0068] Then, once voice data and identification data are received (acquired) from the communication terminal 100, the display control unit 256 of the communication terminal 200 displays a sound selection image 2053, as shown in Figure 5, on the display screen 2050 of the display device 205 (S606).

[0069] The input receiving unit 252 of the communication terminal 200 receives a sound selection instruction based on the selection operation of the sound selection buttons SW1 to SW3 of the sound selection image 2053 by the listener B (S607). When the input receiving unit 252 receives a sound selection instruction, the CPU 201 corrects the sound output from the speaker, etc. of the sound input / output unit 253 (S608). For example, the CPU 201 performs a correction that excludes the sound for which a sound selection instruction was given (does not output from the speaker, etc.).

[0070] <Modified version of Web Conferencing System 700> The web conferencing system 700 described above was explained as having a configuration in which the sound information generation unit 154 and the identification information addition unit 155 are mounted on the communication terminal 100 of the speaker A. However, the functions of the sound information generation unit 154 and the identification information addition unit 155 may be installed in other information processing devices. For example, a sound information management server 500 as shown in Figure 7 may be provided as another information processing device, and the functions of the sound information generation unit 154 and the identification information addition unit 155 may be transferred to the sound information management server 500.

[0071] Figure 7 is a functional block diagram showing another example of a web conferencing system 700 according to an embodiment of the present invention. The hardware configuration and functional configuration of the sound information management server 500 are the same as those of the communication terminals 100 and 200 described above, with each component and function corresponding to a code in the 500 series.

[0072] In this modified configuration, communication terminals 100 and 200 can communicate with the sound information management server 500 via the communication network 1000. The sound information management server 500 is a conference processing device that functions as a web server (or HTTP server) that stores and manages data such as audio data to be transmitted to communication terminals 100 and 200. The sound information management server 500 may also support cloud computing.

[0073] Figure 8 is a sequence diagram based on the configuration in Figure 7. Steps S801 and S802 in Figure 8 are the same as steps S601 and S602 in Figure 6, so their explanation is omitted.

[0074] When the web conferencing system 700 is started, the audio input / output unit 153 of the communication terminal 100 starts collecting sound (sound collection) (S803), and after a predetermined time, the CPU 101 sends an instruction to the sound information management server 500 requesting analysis of the input sound along with the collected input sound data (S804).

[0075] When the sound information management server 500 receives instructions from the communication terminal 100 to acquire input sound data and to analyze the input sound (S805), it performs analysis of the input sound, that is, "decomposition" by the sound information generation unit 554 and "addition of identification data" by the identification information addition unit 555 on the input sound data collected by the sound input / output unit 153 (S806). As a result, sound data and identification data are created. Here, the "decomposition" and other processes performed by the sound information generation unit 554 are an example of sound information generation processing, and the "addition of identification data" and other processes performed by the identification information addition unit 555 are an example of identification information addition processing.

[0076] Once the analysis of the input sound in step S806 is complete, the sound data and identification data created in step S806 are transmitted from the transmitting / receiving unit 551 of the sound information management server 500 to the communication terminal 200 via the communication network 1000 (S807), and the transmitting / receiving unit 251 of the communication terminal 200 receives (acquires) the sound data and identification data transmitted from the sound information management server 500 (S808). Here, the "transmission" process performed by the transmitting / receiving unit 551 is an example of a transmission process. The following steps S809 to S811 are the same as steps S606 to S608 in Figure 6, so their explanation is omitted.

[0077] Figure 9 is an explanatory diagram showing another example of sound selection image 2053.

[0078] The sound selection image 2053 displayed on the display screen 2050 of the communication terminal 200 is not limited to the one shown in Figure 5. For example, the sound selection image 2053 may be displayed as a checklist type image as shown in Figure 9(a), or as a toggle switch type image as shown in Figure 9(b). Also, the layout of the sound selection buttons SW1 to SW3 is not limited to the arrangement of the three sound selection buttons SW1 to SW3 arranged vertically as shown in Figure 5, but may also be arranged horizontally. Furthermore, the number of sound selection buttons displayed on the sound selection image 2053 is not limited to three, but may be two or fewer, or four or more.

[0079] The sequence diagrams in Figures 6 and 8 can be constructed as a program. This program may be distributed as an installable or executable file, recorded on a computer-readable storage medium. Examples of storage media include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray® Disc, SD card, or USB memory. Furthermore, the storage medium can be provided domestically or internationally as a program product.

[0080] The present invention is not limited to the embodiments described above, and can be modified in any way that a person skilled in the art can conceive of, including other embodiments, additions, changes, and deletions. Furthermore, any embodiment that achieves the function and effects of the present invention is included within the scope of the present invention. [Explanation of symbols]

[0081] 100, 200: Communication terminals 153: Audio Input / Output Section 154:Sound information generation section 155: Identification information addition section 205:Display device 253: Audio Input / Output Section 256: Display Control Unit 700: Web conferencing system 1000: Communication Network 2050 :Display screen 2051: First frame 2052: Second frame 2053: Audio Selection Image SW1, SW2, SW3: Voice selection buttons [Prior art documents] [Patent Documents]

[0082] [Patent Document 1] Japanese Patent Publication No. 2019-021248

Claims

1. A conference system that enables the holding of meetings by connecting multiple information processing terminals in a communicative manner, If, among the participants of the aforementioned meeting, the sender's terminal is designated as the first communication terminal and the receiver's terminal as the second communication terminal, The first communication terminal is, A sound collection method that collects sounds emitted from multiple sound sources, A sound information generation means decomposes the input sound information consisting of the sounds input from the sound collection means based on the characteristic quantities of each sound, and generates sound information related to each sound, An identification information adding means for adding identification information to each of the aforementioned multiple sound pieces of sound that makes them uniquely identifiable, A transmission means for transmitting synthesized sound information, which is obtained by synthesizing each sound information to which the aforementioned identification information has been added, to the second communication terminal, It has, The aforementioned second communication terminal is Receiving means for receiving the synthesized sound information from the first communication terminal, A display control means for displaying a sound selection image associated with identification information attached to each sound piece included in the synthesized sound information on the display unit, Sound output means that outputs the sound based on the selection instruction for the sound selection image, A control means for changing the sound output from the sound output means based on the selection instruction, A conference system characterized by having the following features.

2. The control means modifies the sound based on the selection instruction so that it is not output from the sound output means. The conference system according to claim 1.

3. The sound information generation means analyzes the frequency characteristics of the input sound information and decomposes it into individual sounds based on the levels of components included in a predetermined frequency range. The conference system according to claim 1 or 2.

4. Sound information generation means that takes sounds emitted from multiple sound sources as input sound information, decomposes the input sound information based on the characteristic quantities of each sound, and generates multiple sound information, An identification information adding means for adding identification information to each of the aforementioned plurality of sound information, which makes each of the sound information uniquely identifiable, A transmission means for transmitting the plurality of sound information and the identification information to the other conference processing device, as information to be displayed on the display unit of another information processing device, a sound selection image including sound selection means associated with the identification information, Equipped with, The display unit of the other conference processing device displays a sound selection image associated with each sound piece to which the identification information has been added. A conference processing device characterized by the following features.

5. A receiving means that receives synthesized sound information, which is obtained by decomposing input sound information related to sounds generated from multiple sound sources transmitted from another conference processing device based on the characteristic quantities of each sound, and adding identification information that makes each sound uniquely identifiable. A display control means for displaying a sound selection image associated with identification information attached to each sound information included in the synthesized sound information on the display unit, Sound output means that outputs the sound based on the selection instruction for the sound selection image, A control means for changing the sound output from the sound output means based on the selection instruction, Equipped with, A conference processing device characterized by the following features.

6. On the computer, Sound information generation process that takes sounds emitted from multiple sound sources as input sound information, decomposes the input sound information based on the characteristic quantities of each sound, and generates multiple sound information, An identification information addition process is performed to add identification information to each of the aforementioned multiple sound pieces of information, which makes each of the sound pieces of information uniquely identifiable. A transmission process that transmits the plurality of sound information and the identification information to another conference processing device, A program characterized by causing the execution of a specific action.

Images