Communication system, communication method, and program

Abstract

To reduce an influence on a user's audio acquired by a microphone and temporal changes of the influence when echo audio is removed, in a communication device that transmits and receives content data including audio to / from another communication device.SOLUTION: A communication device transmits and receives content data including audio to / from another communication device. The communication device includes: an audio acquisition unit that acquires audio; a first generating unit that generates first audio data in which the audio is represented by audio frequencies and volume; a second generating unit that generates second audio data in which the frequencies of the first audio data are shifted by a predetermined frequency; and a third generating unit that generates third audio data in which the volume of the second audio data is reduced or deleted in a comb-like manner with respect to the frequencies, based on the predetermined frequency.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention 、 The present invention relates to a communication system, a communication method, and a program. [Background technology]

[0002] 2. Description of the Related Art In a communication system in which a plurality of communication devices transmit and receive content including audio, an echo canceller that cancels audio echo is known.

[0003] For example, an echo canceller is known that detects a clock mismatch between the speaker and microphone, and shifts the frequency of the frequency signal on the speaker side or the frequency signal on the microphone side based on that mismatch, thereby effectively suppressing echo (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] For example, a conventional echo canceller such as that shown in Patent Document 1 has a problem in that when removing the echo sound output by the speaker from the input sound acquired by the microphone, the echo canceller affects the user's voice acquired by the microphone, and the effect changes over time.

[0005] One embodiment of the present invention has been made in consideration of the above-mentioned problems, and in a communication device that transmits and receives content data including audio to and from other communication devices, when removing echo audio, reduces the impact on the user's audio captured by a microphone and the changes in the impact over time. [Means for solving the problem]

[0006] In order to solve the above problems, a communication device according to an embodiment of the present invention is provided. system teeth, A communication system in which a plurality of communication devices transmit and receive content data including audio via a communication server, the communication server including a communication unit that communicates with the plurality of communication devices and a communication unit that receives a first content data including audio from one of the plurality of communication devices. Audio 、 First audio data represented by audio frequency and volume Convert to a first generator for generating a frequency of the first audio data at a predetermined frequency; onlya second generator that generates shifted second audio data; When the second audio data is divided into a plurality of frequency bands for each of the predetermined frequencies and arranged in order of frequency, the volume of the odd-numbered frequency bands or the even-numbered frequency bands is lowered or muted. a third generation unit that generates third audio data; a data generating unit that generates second content data including the third audio data; and a relay unit that transmits the second content data to other communication devices among the plurality of communication devices that participate in the same session as the one communication device; It has. [Effects of the Invention]

[0007] According to one embodiment of the present invention, in a communication device that transmits and receives content data including audio with other communication devices, when removing echo audio, the impact on the user's audio captured by a microphone and the change in impact over time can be reduced. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a communication system according to a first embodiment. [Figure 2] 2 is a diagram illustrating an example of a hardware configuration of a communication device according to the first embodiment. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a communication device according to the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating first audio data according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating second audio data according to the first embodiment. [Figure 6] FIG. 1 is a diagram (1) for explaining third audio data according to the first embodiment. [Figure 7] FIG. 10 is a diagram (2) for explaining the third audio data according to the first embodiment. [Figure 8] FIG. 3 is a diagram (3) for explaining the third audio data according to the first embodiment. [Figure 9] 5 is a flowchart illustrating an example of processing performed by the communication device according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a functional configuration of a communication device according to a second embodiment. [Figure 11] 10 is a flowchart illustrating an example of processing performed by a communication device according to the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a system configuration of a communication system according to a third embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a hardware configuration of a computer according to a third embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of a functional configuration of a communication server according to a third embodiment. [Figure 15] FIG. 11 is a sequence diagram illustrating an example of processing in a communication system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0010] [First embodiment] <System configuration> 1 is a diagram showing an example of the system configuration of a communication system according to a first embodiment. The communication system 1 is a system in which a communication device 100a transmits and receives content data including audio to and from another communication device 100b via a communication line 10. The content data may include, in addition to audio, data such as moving images, still images, or text. However, for ease of explanation, the following description will be given assuming that the content data is audio data.

[0011] It is assumed that the communication device 100a and the other communication device 100b are the same communication device. In the following description, the term "communication device 100" will be used to refer to any one of the communication devices 100a and 100b.

[0012] In FIG. 1, communication device 100a converts voice (e.g., the voice of a user using communication device 100a) acquired by microphone 101a provided in communication device 100a into voice data and transmits it to another communication device 100b via communication line 10.

[0013] The other communication device 100b outputs a voice A from a speaker 102b provided in the other communication device 100b based on the voice data received from the communication device 100a via the communication line 10. Similarly to the communication device 100a, the other communication device 100b converts a voice B acquired by a microphone 101b provided in the other communication device 100b into voice data and transmits the voice data to the communication device 100a via the communication line 10.

[0014] At this time, the input voice acquired by the microphone 101b includes voice B of a user using another communication device 100b, as well as voice A output from the speaker 102b, which is voice 103 acquired by the microphone 101b. If this voice 103 is sent to the communication device 100a as is, a voice echo occurs in which voice 103, which is a delayed version of voice A, is added to voice B output by the communication device 100a. This voice echo not only causes discomfort to the user, but may also cause feedback.

[0015] For this reason, for example, in the conventional technology disclosed in Patent Document 1, an echo canceller is installed that removes speech 103 from input speech acquired by microphone 101b so that only speech B can be transmitted to communication device 100a. However, when removing speech 103 output by speaker 102b from input speech acquired by microphone 101b, conventional echo cancellers have a problem in that they affect speech B of the user acquired by microphone 101b and the effect of this effect changes over time. Therefore, with conventional echo cancellers, speech B output by communication device 100a sounds unnatural and the unnaturalness changes over time, causing discomfort to the user.

[0016] Therefore, the communication device 100 according to this embodiment has a function of removing echo sounds and also of reducing the influence of the echo sounds on the user's voice picked up by the microphone and the change in the influence over time when the echo sounds are removed.

[0017] <Hardware configuration> Fig. 2 is a diagram illustrating an example of a hardware configuration of a communication device according to the first embodiment. As an example, the communication device 100 includes an ADC (Analog to Digital Converter) 201, a signal processing unit 202, a codec 203, a communication circuit 204, a DAC (Digital to Analog Converter) 205, a microphone 101, a speaker 102, etc., as illustrated in Fig. 2.

[0018] The ADC 201 is a circuit or device that converts an audio signal (analog signal) output from the microphone 101 into a digital signal. The ADC 201 may be included in the signal processing unit 202.

[0019] The signal processing unit 202 is a circuit, device, or computer that processes digital audio signals. As an example, the signal processing unit 202 is realized by a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. As another example, instead of (or in addition to) the DSP, ASIC, or FPGA, the signal processing unit 202 may be realized by a computer configured with a CPU (Central Processing Unit), a memory, a storage device, and the like, and a program executed by the computer.

[0020] The codec 203 is a device including an encoder 211 that encodes the audio data processed by the signal processing unit 202 and converts it into data for communication, and a decoder 212 that decodes the data for communication and converts it into audio data, or a program executed by a computer. The codec may be included in the signal processing unit 202.

[0021] The communication circuit 204 is a circuit, device, module, or the like for communicating with other communication devices 100 via the communication line 10. In this embodiment, the communication circuit 204 may perform wireless communication or may perform wired communication. Furthermore, the communication method by which the communication circuit 204 communicates with other communication devices 100 may be any communication method.

[0022] The hardware configuration of the communication device 100 shown in FIG. 2 is an example, and the communication device 100 may further include a computer for overall control, a camera, a display device, an input device, and the like.

[0023] <Functional configuration> 3 is a diagram showing an example of the functional configuration of a communication device according to the first embodiment. The communication device 100 according to this embodiment has functional configurations such as a voice acquisition unit 301, a first generation unit 302, a second generation unit 303, a third generation unit 304, a transmission unit 305, a reception unit 306, and a voice output unit 307. Each of the above functional configurations is realized by, for example, hardware, a program executed by a computer, or a combination of hardware and a program executed by a computer.

[0024] 2 and acquires sound (such as a user's voice) around the communication device 100. For example, the sound acquisition unit 301 converts an analog sound signal output by the microphone 101 into a digital sound signal using the ADC 201.

[0025] The first generation unit 302 represents the sound acquired by the sound acquisition unit 301 in terms of sound frequency and volume, and generates, for example, first sound data 401 as shown in Fig. 4. In Fig. 4, the horizontal axis represents frequency, and the vertical axis represents volume (or sound pressure level, amplitude, etc.), and the first sound data 401 represents the volume of the sound at each frequency.

[0026] The second generating unit 303 generates, for example, second audio data 501 as shown in Fig. 5 by shifting the frequency of the first audio data 401 generated by the first generating unit 302 by a predetermined frequency. The example in Fig. 5 shows second audio data 501 obtained by shifting the frequency of the first audio data 401 shown in Fig. 4 up by a predetermined frequency f. Note that the second generating unit 303 may also generate second audio data 501 by shifting the frequency of the first audio data 401 down by the predetermined frequency f.

[0027] The third generation unit 304 reduces or deletes the volume of the second audio data 501 in a comb-tooth pattern based on the above-mentioned predetermined frequency f, thereby generating, for example, third audio data 601 as shown in FIG. 6. This third audio data 601 is obtained by, for example, dividing the second audio data 501 into a plurality of frequency bands 701, each of which corresponds to a predetermined frequency f, as shown in FIG. 7, and muting the volumes of even-numbered frequency bands (2, 4, 6, . . .) when the second audio data 501 is arranged in frequency order, for example, as 1, 2, 3, . . . ). Note that the third audio data 601 is not limited to this, and the volumes of odd-numbered frequency bands (1, 3, 5, . . . ) may be muted. Furthermore, the third audio data 601 may be obtained by muting the volumes of even-numbered frequency bands or odd-numbered frequency bands, rather than muting them, to a predetermined volume or lower that is sufficiently low.

[0028] Such third audio data 601 can be generated, for example, by lowering (for example, rewriting to 0) the volume value of a predetermined frequency band (for example, an even-numbered frequency band) in the signal processing unit 202 of FIG. 2.

[0029] For example, in FIG. 1, the sound A output from the speaker 102b of the other communication device 100b based on this third sound data 601 does not include sounds in the even-numbered frequency bands.

[0030] Furthermore, the voice A output from the speaker 102b and captured by the microphone 101b is converted by the second generator 303 of the other communication device 100b into second voice data whose frequency is shifted higher by a predetermined frequency f. For example, as shown in Fig. 8(A), this second voice data becomes second voice data 801 in which volume data of odd-numbered frequency bands included in the third voice data 601 is shifted to even-numbered frequency bands.

[0031] Since the volume of the odd-numbered frequency bands of this second audio data 801 is 0 (silent), the third generation unit 304 included in the other communication device 100b reduces the volume of the even-numbered frequency bands to 0, thereby turning the data into silent data as shown in Fig. 8(B) In this way, the communication device 100 can reduce audio echo by generating the third audio data 601 and transmitting it to the other communication device 100.

[0032] Note that the audio data shown in Figures 4 to 8 are merely illustrative images and do not represent actual audio data. For example, it is desirable to determine an optimal value for the predetermined frequency f described in Figure 7 through experiments, for example, in the range of several Hz to several tens of Hz so that audio data can be reduced and there is no sense of incongruity.

[0033] 7, the frequency width to be reduced is frequency f or a frequency width with a margin added to frequency f. It is desirable to determine an optimal value for this margin by, for example, experiment or calculation so that an error in the clock signal does not cause echo sound to be insufficiently reduced at the boundary between the even-numbered frequency band and the odd-numbered frequency band, resulting in an unnatural sound.

[0034] 3, the functional configuration of the communication device 100 will be further described. The transmission unit 305 converts the third audio data 601 generated by the third generation unit 304 into data for communication, and transmits the converted data for communication (hereinafter referred to as first content data) to the other communication device 100. For example, the transmission unit 305 encodes the third audio data 601 using the encoder 211 in FIG. 2, and transmits the encoded first content data to the other communication device 100 via the communication circuit 204.

[0035] The receiving unit 306 receives communication data (hereinafter referred to as second content data) obtained by converting third audio data generated by the other communication device 100 into communication data from the other communication device 100. For example, the receiving unit 306 decodes the second content data received by the communication circuit 204 in FIG. 2 using the decoder 212, and outputs the decoded data to the audio output unit 307.

[0036] The audio output unit 307 outputs audio based on the second content received by the receiving unit 306. For example, the audio output unit 307 outputs a comb-like audio in which the volume of a predetermined frequency band is reduced, such as the third audio data 601 in Fig. 6, which is included in the second content.

[0037] <Processing flow> Next, the processing flow of the communication method according to the first embodiment will be described.

[0038] FIG. 9 is a flowchart illustrating an example of processing performed by the communication device according to the first embodiment.

[0039] (Transmission process) 9(A) shows an example of a transmission process in which communication device 100 transmits content data to another communication device. At the start of the process shown in FIG. 9(A), communication device 100 establishes communication with another communication device 100 and is in a state in which they can transmit and receive content data to and from each other.

[0040] In step S901 , the voice acquisition unit 301 acquires voice around the communication device 100 using the microphone 101 .

[0041] In step S902, the first generation unit 302 generates the first audio data 401, for example, as described with reference to FIG. 4, which represents the audio acquired by the audio acquisition unit 301 in terms of audio frequency and volume.

[0042] In step S903, the second generation unit 303 generates the second audio data 501, for example, as described in FIG. 5, by shifting the frequency of the first audio data 401 generated by the first generation unit 302 by a predetermined frequency f.

[0043] In step S904, the third generation unit 304 generates the third audio data 601, for example, as described in Figures 6 and 7, by lowering the volume of a predetermined frequency band of the second audio data 501 generated by the second generation unit 303.

[0044] In step S905, the transmitting unit 305 transmits content data (first content data) obtained by converting the third audio data 601 generated by the third generating unit 304 into data for communication to the other communication device 100 with which it is communicating.

[0045] (receiving process) 9(B) shows an example of a receiving process in which the communication device 100 receives content data from another communication device. At the start of the process shown in FIG. 9(B), the communication device 100 establishes communication with the other communication device 100 and is in a state in which the communication device 100 can transmit and receive content data to and from the other communication device 100.

[0046] In step S911, the receiving unit 306 receives content data (second content data) from another communication device 100 with which the receiving unit 306 is in communication.

[0047] In step S912, the audio output unit 307 reproduces the audio data included in the content data received by the receiving unit 306 and outputs the audio.

[0048] For example, the communication devices 100a and 100b shown in Fig. 1 transmit and receive audio by repeatedly executing the transmission process shown in Fig. 8(A) and the reception process shown in Fig. 8(B). As a result, echo audio included in the received audio is reduced, as described with reference to Figs. 8(A) and (B). In addition, in this embodiment, the frequency of the received audio is shifted by a predetermined frequency f. However, as described above, the predetermined frequency f is in the range of several Hz to several tens of Hz and does not change over time, which reduces discomfort to the user.

[0049] As described above, according to the first embodiment, in a communication device that transmits and receives content data including audio to and from another communication device 100, when removing echo audio, it is possible to reduce the impact on the user's audio captured by a microphone and the change in the impact over time.

[0050] [Second embodiment] 3 is an example, and various modifications and applications are possible. For example, the communication device 100 according to the first embodiment transmits and receives content data including the third audio data 601, but the communication device 100 may transmit and receive content data including the second audio data 501.

[0051] <Functional configuration> Fig. 10 is a diagram showing an example of the functional configuration of a communication device according to the second embodiment. Similar to the first embodiment described in Fig. 3, the communication device 100 according to the second embodiment includes a voice acquisition unit 301, a first generation unit 302, a second generation unit 303, a third generation unit 304, a transmission unit 305, a reception unit 306, and a voice output unit 307. In addition to the above functional configurations, the communication device 100 according to the second embodiment also includes a fourth generation unit 1001, a data conversion unit 1002, and the like.

[0052] In the second embodiment, the transmitting unit 305 transmits content data (first content data) obtained by converting the second audio data 501 generated by the second generating unit 303 into data for communication to another communication device 100 with which the transmitting unit 305 is communicating. Also, the receiving unit 306 receives content data (second content data) including the second audio data 501 transmitted by the other communication device 100 with which the transmitting unit 305 is communicating.

[0053] 2, and converts the audio included in the content data received by the receiving unit 306 into fourth audio data represented by frequency and volume. For example, the fourth generating unit 1001 converts the audio (second audio data 501) included in the content data received by the receiving unit 306 into fourth audio data represented by frequency and volume, in the same manner as the first generating unit 302.

[0054] Furthermore, the third generation unit 304 according to this embodiment generates, for example, third audio data 601 as shown in Fig. 6, in which the volume of a predetermined frequency band of the fourth audio data is lowered based on a predetermined frequency f. Note that the method by which the third generation unit 304 generates the third audio data may be the same as that in the first embodiment.

[0055] 2, and converts the third audio data 601 generated by the third generation unit 304 into audio data that can be played back by the audio output unit 307. Note that if the audio output unit 307 can play back the third audio data, the communication device 100 does not need to include the data conversion unit 1002.

[0056] <Processing flow> Next, a description will be given of the flow of processing in the communication method according to the second embodiment. Fig. 11 is a flowchart showing an example of processing in the communication device according to the second embodiment.

[0057] (Transmission process) Fig. 11(A) shows an example of a transmission process in which a communication device 100 transmits content data to another communication device. At the start of the process shown in Fig. 11(A), the communication device 100 establishes communication with the other communication device 100 and is in a state in which they can transmit and receive content data to and from each other. Further, detailed description of the process that is the same as that of the first embodiment will be omitted here.

[0058] In step S1101 , the voice acquisition unit 301 acquires voice around the communication device 100 using the microphone 101 .

[0059] In step S1102, the first generation unit 302 generates first audio data 401 that represents the audio acquired by the audio acquisition unit 301 in terms of audio frequency and volume.

[0060] In step S1103, the second generating unit 303 generates second audio data 501 by shifting the frequency of the first audio data 401 generated by the first generating unit 302 by a predetermined frequency f.

[0061] In step S1104, the transmitting unit 305 transmits content data (first content data) obtained by converting the second audio data 501 generated by the second generating unit 303 into data for communication to the other communication device 100 with which it is communicating.

[0062] (receiving process) Fig. 11(B) shows an example of a receiving process in which the communication device 100 receives content data from another communication device. At the start of the process shown in Fig. 11(B), the communication device 100 establishes communication with the other communication device 100 and is in a state in which they can transmit and receive content data to and from each other. Further, detailed description of the process similar to that of the first embodiment will be omitted here.

[0063] In step S1111, the receiving unit 306 receives content data (second content data) from another communication device 100 with which the receiving unit 306 is in communication.

[0064] In step S1112, the fourth generating unit 1001 generates fourth audio data that represents the audio included in the content data received by the receiving unit 306 in terms of frequency and volume.

[0065] In step S1113, the third generation unit 304 generates third audio data 601, for example, as shown in FIG. 6, by lowering the volume of a predetermined frequency band of the fourth audio data generated by the fourth generation unit 1001.

[0066] In step S1114, the audio output unit 307 outputs audio based on the third audio data 601 generated by the third generation unit 304. For example, the audio output unit 307 converts the third audio data 601 into reproducible audio data using the data conversion unit 1002 as necessary, and outputs audio by playing back the converted audio data.

[0067] Through the above processing, the audio output unit 307 according to the second embodiment outputs an audio output signal similar to that of the audio output unit 307 according to the first embodiment to the speaker 102. Therefore, also in the second embodiment, in a communication device that transmits and receives content data including audio to and from another communication device 100, when removing echo audio, it is possible to reduce the influence on the user's audio picked up by the microphone and the change in the influence over time.

[0068] [Third embodiment] In the third embodiment, an application example of the communication system 1 according to the present embodiment will be described. FIG. 12 is a diagram showing an example of the system configuration of the communication system according to the third embodiment. The communication system 1 according to the present embodiment may be, for example, a conference system in which a plurality of communication devices 100c, 100d, 100e, etc. are connected to a communication server 1200 via a communication network 1201 to hold a video conference, a web conference, or the like. Furthermore, the communication device 100 according to the present embodiment may be, for example, a video conference device, an IWB (Interactive White Board: an electronic whiteboard with a blackboard function that allows mutual communication), a notebook PC (Personal Computer), or any of various other conference terminals.

[0069] 14, the communication server 1200 may include a first generating unit 302, a second generating unit 303, and a third generating unit 304. This allows existing conference terminals to be used as the communication devices 100c, 100d, 100e, and so on.

[0070] <Hardware configuration> The communication server 1200 has, for example, the hardware configuration of a computer 1300 as shown in Fig. 13. Alternatively, the communication server 1200 may be configured by a plurality of computers 1300.

[0071] 13 is a diagram showing an example of the hardware configuration of a computer according to the third embodiment. The computer 1300 includes, for example, a CPU 1301, a ROM (Read Only Memory) 1302, a RAM (Random Access Memory) 1303, a HD (Hard Disk) 1304, an HDD (Hard Disk Drive) controller 1305, a display 1306, an external device connection I / F (Interface) 1307, a network I / F 1308, a keyboard 1309, a pointing device 1310, a DVD-RW (Digital Versatile Disk Rewritable) drive 1312, a media I / F 1314, and a bus line 1315.

[0072] Of these, the CPU 1301 is an arithmetic unit that controls the overall operation of the computer 1300. The ROM 1302 is a non-volatile memory that stores programs used to start up the CPU 1301. The RAM 1303 is a volatile memory that is used as a work area for the CPU 1301. The HD 1304 is a large-capacity storage device that stores programs such as an OS (Operating System) and applications, various types of data, etc. The HDD controller 1305 controls the reading and writing of various types of data from and to the HD 1304 under the control of the CPU 1301.

[0073] The display 1306 is a display device that displays various types of information such as a cursor, menus, windows, characters, or images. The external device connection I / F 1307 is an interface for connecting various types of external devices. The network I / F 1308 is a communication interface for performing data communication using a communication network. The keyboard 1309 is a type of input means that has multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 1310 is a type of input means that selects and executes various instructions, selects a processing target, moves the cursor, etc.

[0074] The DVD-RW drive 1312 controls reading and writing (storing) of data from and to the DVD-RW 1311. Note that the DVD-RW 1311 is an example of a storage medium, and other storage media may be used. The media I / F 1314 controls reading and writing (storing) of data from and to a medium 1313 such as a flash memory. The bus line 1315 includes an address bus, a data bus, various control signals, and the like for electrically connecting the various components such as the CPU 1301 shown in FIG. 13.

[0075] In addition to the hardware configuration of the computer 1300, the communication server 1200 may further include a signal processing device for executing audio signal processing, or the signal processing unit 202, etc.

[0076] <Functional configuration> 14 is a diagram illustrating an example of the functional configuration of a communication server according to the third embodiment. The communication server 1200 realizes a communication unit 1401, a communication management unit 1402, a relay unit 1403, and the like, for example, by the CPU 1301 executing a predetermined program.

[0077] The communication unit 1401 connects the communication server 1200 to the communication network 1201 using the network I / F 1308 included in the communication server 1200, and executes communication processing for communicating with the communication devices 100c, 100d, 100e, and the like.

[0078] The communication management unit 1402 receives connection requests from the communication devices 100c, 100d, and 100e, and executes communication management processing to allow the communication devices 100c, 100d, and 100e requesting connection to the same conference to participate in the same conference (session).

[0079] The relay unit 1403 executes a relay process to relay content data transmitted and received between the communication devices 100c, 100d, and 100e participating in the same conference. For example, the relay unit 1403 receives content data transmitted by the communication device 100c and transfers the received content data to the other communication devices 100d and 100e participating in the same conference. Similarly, the relay unit 1403 transfers content data received from the communication device 100d to the communication devices 100c and 100e, and transfers data received from the communication device e to the communication devices 100c and 100d.

[0080] Furthermore, the relay unit 1403 according to this embodiment includes, for example, a first generating unit 302, a second generating unit 303, a third generating unit 304, and a data generating unit 1404. Note that the basic processing contents of the first generating unit 302, the second generating unit 303, and the third generating unit 304 are similar to those of the first generating unit 302, the second generating unit 303, and the third generating unit 304 according to the first embodiment, and therefore detailed description thereof will be omitted here.

[0081] The first generation unit 302 converts the audio contained in the content data received from one of the communication devices 100 (e.g., communication device 100c) participating in the conference into, for example, first audio data 401 as shown in FIG. 4, which represents the audio in terms of audio frequency and volume.

[0082] The second generating unit 303 generates second audio data 501 as shown in FIG. 5, for example, by shifting the frequency of the first audio data 401 generated by the first generating unit 302 by a predetermined amount.

[0083] The third generation unit 304 generates, for example, third audio data 601 as shown in FIG. 6 by lowering the volume of a predetermined frequency band of the second audio data 501 generated by the second generation unit 303 based on a predetermined frequency f.

[0084] The data generating unit 1404 generates content data for communication, including the third audio data 601 generated by the third generating unit 304. This content data includes, for example, image data included in the content received from one communication device 100, and the third audio data 601.

[0085] The relay unit 1403 transfers the content data generated by the data generation unit 1404 to other communication devices 100 (for example, communication devices 100d and 100e) participating in the same conference as the one communication device 100.

[0086] In this embodiment, it is assumed that the communication devices 100c, 100d, and 100e are existing conference devices, and therefore a description of the hardware configuration and functional configuration of the communication device 100 will be omitted.

[0087] <Processing flow> Fig. 15 is a sequence diagram showing an example of processing in the communication system according to the third embodiment. This diagram shows an example of processing in which the communication server 1200 transfers content data received from the communication device 100c to other communication devices 100d and 100e participating in the same conference in the communication system 1 shown in Fig. 12. It is assumed that, at the start of the processing shown in Fig. 15, the communication devices 100c, 100d, and 100e are connected to the communication server 1200 via the communication network 1201 and are participating in the same conference.

[0088] In step S1501, the communication unit 1401 of the communication server 1200 receives content data from the communication device 100c.

[0089] In step S1502, the first generating unit 302 of the communication server 1200 converts the audio included in the content data received from the communication device 100c into first audio data 401 that represents the audio in terms of audio frequency and volume.

[0090] In step S1503, the second generating unit 303 of the communication server 1200 generates second audio data 501 by shifting the frequency of the first audio data 401 generated by the first generating unit 302 by a predetermined amount.

[0091] In step S1504, the third generation unit 304 of the communication server 1200 generates third audio data 601, for example, as shown in FIG. 6, by lowering the volume of a predetermined frequency band of the second audio data 501 generated by the second generation unit 303 based on a predetermined frequency f.

[0092] In step S1505, the data generating unit 1404 of the communication server 1200 generates content data for communication, including the third audio data 601 generated by the third generating unit 304.

[0093] In steps S1506 and S1507, the relay unit 1403 of the communication server 1200 transmits the content data generated by the data generation unit 1404 to the other communication devices 100d and 100e participating in the same conference as the communication device 100c.

[0094] In steps S1508 and S1509, the communication devices 100d and 100e play back the received content data, thereby outputting audio with the volume of a predetermined frequency band lowered, as shown in FIG.

[0095] By the processing of FIG. 15, the communication system 1 according to the third embodiment can reduce the influence on the user's voice picked up by the microphone and the change in the influence over time when removing echo voice using an existing conference terminal.

[0096] In this embodiment, if the communication devices 100c, 100d, and 100e have an echo canceller, it is desirable to set the echo canceller function to off.

[0097] [Other embodiments] For example, in a communication system 1 having a system configuration as shown in Fig. 12, the communication terminals 100c, 100d, and 100e may be conference terminals including the functional configurations as shown in Fig. 3 and 10. In this case, a conference system can be constructed using the communication terminals 100c, 100d, and 100e and an existing communication server 1200, and the same effects as those of the third embodiment can be obtained.

[0098] As described above, according to each embodiment of the present invention, in a communication device 100 that transmits and receives content data including audio to and from other communication devices 100, when removing echo audio, it is possible to reduce the impact on the user's audio captured by the microphone and the change in the impact over time.

[0099] <Supplementary information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by electronic circuits, and devices such as ASIC, DSP, FPGA, or conventional circuit modules designed to perform each function described above.

[0100] Additionally, the devices described in each of the above embodiments are merely illustrative of one of multiple computing environments for implementing the embodiments disclosed herein. In one embodiment, communication server 1200 includes multiple computing devices, such as a server cluster, configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0101] Furthermore, the communication server 1200 can be configured to share the disclosed processing steps, for example, Fig. 15, in various combinations. For example, at least one of the first generating unit 302, the second generating unit 303, and the third generating unit 304 included in the communication server 1200 may be included in the communication device 100. Furthermore, the functional configurations included in the communication server 1200 may be integrated into one server device or may be distributed across multiple devices. [Explanation of symbols]

[0102] 1. Communication system (conference system) 100, 100a, 100b Communication equipment 100c~100e Communication equipment (conference equipment) 301 Audio Acquisition Unit 302 First generation unit 302 303 Second generation unit 303 304 Third generation unit 304 305 Transmission Unit 306 Receiving Unit 307 Audio output section 401 First audio data 501 Second Audio Data 601 Third Audio Data 701 frequency band 1001 Fourth Generation 1200 Communication Server f given frequency [Prior art documents] [Patent documents]

[0103] [Patent Document 1] International Publication No. 2006 / 087813

Claims

1. A communication system in which a plurality of communication devices transmit and receive content data including audio via a communication server, The communication server a communication unit that communicates with the plurality of communication devices; a first generation unit that converts audio included in first content data received from one of the plurality of communication devices into first audio data represented by an audio frequency and a volume; a second generator configured to generate second audio data by shifting the frequency of the first audio data by a predetermined frequency; a third generation unit that generates third audio data by lowering or muting the volume of odd-numbered frequency bands or even-numbered frequency bands when the second audio data is divided into a plurality of frequency bands for each of the predetermined frequencies and arranged in order of frequency; a data generating unit that generates second content data including the third audio data; a relay unit that transmits the second content data to other communication devices among the plurality of communication devices that participate in the same session as the one communication device; A communication system comprising:

2. 2. The communication system according to claim 1, wherein the communication system is a conference system in which the plurality of communication devices transmit and receive content data including images and audio to and from each other.

3. A communication system in which a plurality of communication devices transmit and receive content data including audio via a communication server, The communication server communicating with the plurality of communication devices; converting audio included in first content data received from one of the plurality of communication devices into first audio data represented by an audio frequency and a volume; generating second audio data by shifting the frequency of the first audio data by a predetermined frequency; generating third audio data by lowering or muting the volume of odd-numbered or even-numbered frequency bands when the second audio data is divided into a plurality of frequency bands for each of the predetermined frequencies and arranged in order of frequency; a process of generating second content data including the third audio data; transmitting the second content data to other communication devices among the plurality of communication devices that participate in the same session as the one communication device; A communication method that performs the following.

4. A communication system in which a plurality of communication devices transmit and receive content data including audio via a communication server, The communication server, communicating with the plurality of communication devices; converting audio included in first content data received from one of the plurality of communication devices into first audio data represented by an audio frequency and a volume; generating second audio data by shifting the frequency of the first audio data by a predetermined frequency; generating third audio data by lowering or muting the volume of odd-numbered or even-numbered frequency bands when the second audio data is divided into a plurality of frequency bands for each of the predetermined frequencies and arranged in order of frequency; a process of generating second content data including the third audio data; transmitting the second content data to other communication devices among the plurality of communication devices that participate in the same session as the one communication device; A program that executes.

Images