Server, terminal device and program for online conference

Abstract

To provide a server, a terminal device, and a program for an on-line conference that can check a situation in which a speaker's voice is not normally output in a reception side terminal device.SOLUTION: According to an embodiment, the server has a communication interface, a memory, and a processor. The communication interface communicates with a first terminal device that sends voice data generated from an input voice, and a second terminal device that outputs a voice based on the voice data received from the first terminal device. The memory stores a result of voice recognition performed by the first terminal device on the input voice input to the first terminal device, and a result of voice recognition performed by the second terminal device on voice data of the input voice that the second terminal device receives from the first terminal device. The processor determines the difference between the input voice input to the first terminal device and the voice output by the second terminal device based on the voice data of the input voice received from the first terminal device, on the basis of the comparison between the result of voice recognition performed by the first terminal device and the result of voice recognition performed by the second terminal device.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to a server, a terminal device, and a program for an online conference. [Background technology]

[0002] Conventionally, there is a technology called online conferencing, in which multiple terminal devices connected via a network transmit and receive audio to conduct a conversation between multiple people, such as a conference. The multiple terminal devices participating in an online conference are often in different communication environments. In a terminal device with a poor communication environment, some of the audio input by other terminal devices is interrupted or is not output as accurate audio.

[0003] Conventionally, a technique for measuring communication quality between terminal devices in an online conference involves sending a small amount of test data back and forth and calculating the throughput (transfer speed) from the time difference. Although such conventional techniques are simple, they often do not reflect the human experience of an online conference. For example, voice may be audible even if the throughput is temporarily low, or voice may be interrupted even with a stable throughput measurement value. For this reason, there is a demand for a technique that can reliably detect whether the speaker's voice is accurately reaching the listener during an online conference. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-228114 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to solve the above-mentioned problems, there are provided a server, a terminal device, and a program for an online conference that are capable of checking whether the voice of a speaker is not being normally outputted on a receiving terminal device. [Means for solving the problem]

[0006] According to an embodiment, a server has a communication interface, a memory, and a processor. The communication interface communicates with a first terminal device that transmits voice data generated from an input voice and a second terminal device that outputs a voice based on the voice data received from the first terminal device. The memory stores a voice recognition result by the first terminal device for an input voice input to the first terminal device and a voice recognition result by the second terminal device for voice data of the input voice received by the second terminal device from the first terminal device. The processor performs a comparison between the voice recognition result by the first terminal device and the voice recognition result by the second terminal device. If the difference exceeds a first threshold, a first warning is output, and if the difference exceeds a second threshold, a second warning is output. . [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating a configuration example of an online conference system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a control system in a server used in the online conference system according to the embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating an example of the configuration of a control system in a terminal device used in the online conference system according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a speech recognition result by a plurality of terminal devices in the online conference system according to the embodiment. [Diagram 5] FIG. 5 is a flowchart illustrating an example of the operation of a server used in the online conference system according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of the operation of a server used in the online conference system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a diagram for explaining an outline of an online conference system 1 according to an embodiment. As shown in FIG. 1, an online conference system 1 according to an embodiment includes a server 10 and a plurality of terminal devices 20 (21, 22, 23, . . . ) that are connected via a network. The server 10 is a management device that manages the quality of voice calls in each terminal device 20. The server 10 determines how voice input to a certain terminal device (first terminal device) 21 is output from other terminal devices (second terminal devices) 22, 23 connected via a network. In the example shown in Fig. 1, the first terminal device is the terminal device 21 to which a speaker inputs voice, and the second terminal devices are the terminal devices 22, 23 of listeners other than the speaker.

[0009] The server 10 obtains from the terminal device 21 a speech recognition result of a speech input by a speaker to the terminal device (first terminal device) 21. The server 10 also obtains from the terminal devices 22 and 23 a speech recognition result of a speech (speech output by the second terminal device) received from the terminal device 21 by terminal devices (second terminal devices) 22 and 23 of persons other than the speaker (listeners) via the network.

[0010] The server 10 compares the speech recognition result of the speech input to the speaker's terminal device 21 with the speech recognition result of the speech output from the listener's terminal devices 22, 23. If the speech recognition result at the terminal device 21 matches with the speech recognition result at the terminal devices 22, 23, the server 10 determines that the speech input to the terminal device 21 is accurately output from the terminal devices 22, 23. If the speech recognition result at the terminal devices 22, 23 differs from the speech recognition result at the terminal device 21, the server 10 determines that the speech input to the terminal device 21 is not accurately output from the terminal devices 22, 23. If the speech recognition result at the terminal devices 22, 23 differs from the speech recognition result at the terminal device 21 by more than a preset value (threshold value), the server 10 transmits a warning to the terminal devices 22, 23.

[0011] The multiple terminal devices 20 (21, 22, 23, ...) are information processing devices equipped with a microphone and a speaker. The microphone inputs (collects) sounds including the voice of a person. The speaker outputs sounds based on voice data. The information processing device as the terminal device 20 is, for example, a personal computer, a smartphone, or a tablet terminal. The terminal device 20 may also be configured such that either or both of the microphone 2 and the speaker 3 are connected to an information processing device such as a computer.

[0012] The terminal device 20 collects the voice (audio) of the speaker using a microphone and transmits the collected audio data (audio data) to the other terminal devices 20 participating in the online conference. The terminal device 20 also receives audio data such as the speaker's voice from the other terminal devices 20 via the network and outputs the received audio data as sound from a speaker.

[0013] The terminal device 20 transmits voice data of the sound collected by the microphone to another terminal device, and outputs a sound based on the voice data received from the other terminal device from a speaker. The terminal device 20 also performs voice recognition processing. When the terminal device 20 collects the voice of a speaker with the microphone 2, the terminal device 20 performs voice recognition processing on the collected voice. When the terminal device 20 receives voice data from another terminal device, the terminal device 20 performs voice recognition processing on the voice to be output based on the received voice data. Furthermore, the terminal device 20 uploads the voice recognition result obtained by the voice recognition processing to the server 10.

[0014] 1, a terminal device 21 is a first terminal device used by a speaker, and terminal devices 22 and 23 are second terminal devices used by listeners. In the example shown in FIG. 1, terminal device 21 as the first terminal device collects the voice of the speaker with a microphone and transmits data of the collected voice (voice data) to other terminal devices 22 and 23. Terminal devices 22 and 23 as the second terminal devices receive the voice data from terminal device 21 via a network and output sound based on the received voice data from a speaker.

[0015] Furthermore, when the terminal device 21 as the first terminal device detects the voice of a speaker from the sound collected by the microphone 2, it performs voice recognition processing on the voice collected by the microphone 2. The terminal device 21 transmits to the server 10 the voice recognition result obtained by the voice recognition processing on the voice collected by the microphone 2. Furthermore, when the terminal devices 22 and 23 as the second terminal devices receive voice data from the terminal device 21 as the first terminal device, they perform voice recognition processing on the sound based on the received voice data. The terminal devices 22 and 23 transmit to the server 10 the voice recognition result obtained by the voice recognition processing on the sound based on the voice data received from the terminal device 21.

[0016] Next, the configuration of the server 10 according to the embodiment will be described. FIG. 2 is a block diagram showing an example of the configuration of the server 10 according to the embodiment. As shown in FIG. 2, the server 10 includes a processor 101, a main storage device 102, an auxiliary storage device (memory) 103, and a communication interface 104. The processor 101 controls the entire server 10. The processor 101 is, for example, a CPU. The processor 101 executes programs to perform various processes described below. For example, the processor 101 executes processes such as comparing the voice recognition results of the terminal devices and outputting a warning in accordance with the comparison result of the voice recognition results.

[0017] The main storage device 102 is a main memory that stores data. The main storage device 102 is configured, for example, with a RAM (Random Access Memory) or the like. The main storage device 102 temporarily stores data being processed by the processor 101. For example, the main storage device 102 stores data required for executing a program and the results of the program execution. The main storage device 102 also operates as a buffer memory for temporarily holding data.

[0018] The auxiliary storage device 103 is a storage that stores data. The auxiliary storage device 103 includes a non-rewritable non-volatile memory such as a ROM (Read Only Memory) and a rewritable non-volatile memory. The rewritable non-volatile memory is, for example, a HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (registered trademark), or a flash ROM.

[0019] The auxiliary storage device 103 stores various programs and control data executed by the processor 101. For example, the auxiliary storage device 103 stores a program for comparing the voice recognition results of the terminal devices 20 in the online conference system. The auxiliary storage device 103 also stores a program for outputting a warning according to the comparison result of the voice recognition results of the terminal devices 20.

[0020] 2, the auxiliary storage device 103 has a storage area 113 for storing the voice recognition results by each terminal device 20. The storage area 113 stores the voice recognition results for the voice input to the terminal device 21 and the voice recognition results for the voice received (output) by the terminal devices 22 and 23 from the terminal device 21.

[0021] The communication interface 104 is an interface for communicating with each terminal device 20 in the online conference system. The communication interface may include an interface for communicating via a wired line, or may include an interface for communicating wirelessly. For example, the processor 101 acquires a voice recognition result from each terminal device 20 participating in the online conference system via the communication interface 104. The processor 101 also transmits a warning to a specific terminal device 20 via the communication interface 104 according to a comparison result of the voice recognition results by each terminal device 20.

[0022] Next, the configuration of the terminal device 20 according to the embodiment will be described. FIG. 3 is a block diagram showing an example of the configuration of the terminal device 20 according to the embodiment. In the exemplary configuration shown in FIG. 3, the terminal device 20 includes a processor 201, a main memory device 202, an auxiliary memory device (memory) 203, a communication interface 204, a voice processing circuit 205, a microphone 206, a speaker 207, a display device (announcement device) 208, and an operation device 209.

[0023] The processor 201 controls the entire terminal device 20. The processor 201 is, for example, a CPU. The processor 201 executes programs to perform various processes described below. For example, the processor 201 performs processes such as generating voice data of an input sound, transmitting the voice data, recognizing the input sound, transmitting the voice recognition result to the server 10, and outputting a warning. The processor 201 also receives voice data, outputs voice based on the voice data, recognizes the voice of the received (output) voice, and transmits the voice recognition result to the server 10.

[0024] The main storage device 202 is a main memory that stores data. The main storage device 202 is configured, for example, with a RAM (Random Memory) or the like. The main storage device 202 temporarily stores data being processed by the processor 201. For example, the main storage device 202 may store data necessary for executing a program and the execution results of the program. The main storage device 202 also operates as a buffer memory for temporarily storing data. For example, the main storage device 202 stores voice data obtained by processing sounds collected by each microphone 206 with the voice processing circuit 205. The main storage device 202 also stores received voice data.

[0025] The auxiliary storage device 203 is a storage device that stores data. The auxiliary storage device 203 includes a non-rewritable non-volatile memory such as a ROM (read-only memory), and a rewritable non-volatile memory. The rewritable non-volatile memory is, for example, a HDD (hard disk drive), an SSD (solid state drive), an EEPROM (registered trademark), or a flash ROM.

[0026] The auxiliary storage device 203 stores programs and control data executed by the processor 201. The auxiliary storage device 203 stores programs for performing various processes as described above. For example, the auxiliary storage device 203 stores a voice recognition program for performing voice recognition on an input voice or received voice data. The auxiliary storage device 203 also stores a program for transmitting a voice recognition result to the server 10, a program for outputting a warning in response to a notification from the server 10, and the like. Furthermore, in the example shown in FIG. 3, the auxiliary storage device 203 has a storage area 213 for holding a voice recognition result.

[0027] The communication interface 204 is an interface for communicating with the other terminal devices 20 and the server 10 participating in the online conference system. The communication interface 204 may include an interface for communicating via a wired line, or may include an interface for communicating wirelessly. For example, the processor 201 transmits and receives voice data to and from the other terminal devices 20 participating in the online conference system via the communication interface 204. The processor 201 also transmits the result of voice recognition on the input voice or the received voice data to the server 10. Furthermore, when the processor 201 receives a notice of a warning via the communication interface 204, the processor 201 performs a process of notifying the warning using a speaker or a display device.

[0028] The microphone 206 collects (acquires) sound. The microphone 206 inputs the collected sound as an analog signal (analog waveform) and outputs the analog signal of the input sound to the audio processing circuit 205, for example. The audio processing circuit 205 receives an analog signal of a sound collected by a microphone 206, and outputs the input analog signal as audio data in the form of digital data. The audio processing circuit 205 includes an AD converter that digitizes an analog waveform. The microphone 206 may be an external device connected to the terminal device 20. When the microphone 206 is an external device, the audio processing circuit 205 may include an interface for audio input to which the microphone 206 is connected.

[0029] The speaker 207 outputs sound. The speaker 207 emits a sound based on a response waveform as a response sound supplied from the processor 201. In addition, the speaker 207 may be configured to output, as a notification device, a warning content in response to a warning received from the server 10 (described later) by sound. The speaker 207 may be an external device connected to the terminal device 20. When the speaker 207 is an external device, the terminal device 20 may include an interface that outputs a signal indicating a waveform of a sound to be output to the speaker 207.

[0030] The display device 208 displays an image. The display device 208 operates as a notification device. For example, the display device 208 displays a warning screen for notifying a warning in response to a warning received from the server 10 described later. The operation device 209 accepts an operation instruction from a user. For example, the display device 208 and the operation device 209 are configured by a display with a touch panel. The operation device 209 may also include a numeric keypad, a keyboard, or a pointing device.

[0031] Next, the voice recognition results that the server 10 according to the embodiment collects from each terminal device 20 will be described. FIG. 4 is a diagram showing an example of the speech recognition results by each terminal device 20 stored in the storage area 213 of the auxiliary storage device 203 in the server 10. As shown in FIG. The server 10 collects the voice recognition results from each terminal device 20. The server 10 stores the voice recognition results collected from each terminal device in the storage area 113 of the auxiliary storage device 103. In the example shown in Fig. 4, the server 10 stores the voice recognition results for the voice data of the input voice received by the other terminal devices in association with the voice recognition results for the input voice. In the example shown in Fig. 4, the speaker's terminal device (first terminal device) 21 is terminal A, and the listener's terminal devices (second terminal devices) 22 and 23 are terminals B and C.

[0032] Terminal A inputs the voice uttered by the speaker through microphone 206 and performs voice recognition on the input voice (input voice). Terminal A supplies the voice recognition result for the input voice to server 10 in association with information indicating a time (time information). Here, terminal A may also transmit information indicating that the voice recognition result is for the voice uttered by the speaker (input voice) together with the voice recognition result and the time information.

[0033] Furthermore, terminals B and C each receive voice data of the input voice from terminal A and perform voice recognition on the received voice data. Terminals B and C supply the voice recognition result for the received voice data to server 10 in association with time information. Here, terminals B and C may transmit information indicating that the voice recognition result is for voice data received via the network together with the voice recognition result and the time information. Furthermore, terminals B and C may transmit information indicating that the voice recognition result is for voice data from terminal A together with the voice recognition result and the time information.

[0034] The server 10 stores the voice recognition results of the terminals A, B, and C in association with time information. It is assumed that the difference between the time when the terminal A inputs the input voice and the time when the other terminals B and C receive the voice data of the input voice of the terminal A is short. In this case, the voice recognition result for the input voice and the voice recognition result for the voice data of the input voice received by the other terminals are stored in the memory area 213 in association with each other, as shown in FIG.

[0035] The difference between the speech recognition result for the input voice by the speaker's terminal A and the speech recognition result for the voice data of the input voice by terminal B indicates the communication quality between terminal A and terminal B. The speech recognition result for the input voice by the speaker's terminal A is not affected by the communication environment such as the network. In contrast, the speech recognition result for the voice data of the input voice by the listener's terminals B and C is affected by the communication environment (communication quality) between terminal A and terminal A. For example, if the communication quality between terminal B and terminal A is poor, the speech recognition result by terminal B will differ greatly from the speech recognition result by terminal A.

[0036] That is, the greater the difference between the speech recognition result of terminal A for the input speech and the speech recognition result of terminal B for the speech data of the input speech, the worse the communication conditions between terminal A and terminal B can be determined. If the speech recognition result of terminal A for the input speech and the speech recognition result of terminal B for the speech data of the input speech match, the communication conditions between terminal A and terminal B can be determined to be good. Similarly, the communication conditions between terminal A and terminal C can be determined from the difference between the speech recognition result of terminal A for the input speech and the speech recognition result of terminal C for the speech data of the input speech.

[0037] In the example shown in FIG. 4, the speech recognition result for the input speech input to terminal A at time "00:01" matches the speech recognition result for that input speech in terminals B and C. The speech recognition result for the input speech at time "00:12" matches the speech recognition result for that input speech in terminal B. However, the speech recognition result for the input speech at time "00:12" partially mismatches the speech recognition result for that input speech in the terminal. As a result, it can be determined that at time "00:12," the communication quality between terminals A and B is good, but the communication quality between terminals A and C has deteriorated slightly.

[0038] 4, the speech recognition result for the input speech at time "00:23" does not match the speech recognition result for the input speech at terminals B and C. In addition, the speech recognition result for the input speech at time "00:34" does not match the speech recognition result for the input speech at terminals B and C. As a result, it can be determined that the input speech is not being output normally at times "00:23" and "0034" due to poor communication quality with terminal A.

[0039] In this embodiment, the server 10 acquires information as shown in Fig. 4 by collecting voice recognition results from each terminal device participating in the online conference. The server 10 compares the voice recognition result for the input voice with the voice recognition results for the voice data of the input voice received by the other terminal devices. The server 10 determines the difference between the input voice of terminal A and the output voice of terminal B or C corresponding to the input voice by calculating the difference between the corresponding voice recognition results.

[0040] The server 10 judges whether the magnitude of the difference between the voice recognition result by terminal A and the voice recognition result by terminal B or terminal C exceeds a predetermined threshold (default value). If the magnitude of the difference exceeds the predetermined threshold, the server 10 warns terminal A that voice is not being output normally. For example, if the difference between the voice recognition result by terminal A and the voice recognition result by terminal B exceeds the threshold, the server 10 warns terminal A that the speaker's voice is not being output normally at terminal B. Terminal A notifies the warning from the server 10 via the display device 208. This allows the speaker using terminal A to know which terminal is not outputting voice normally.

[0041] Next, the operation of the terminal device 20 in the online conference system 1 according to the embodiment will be described. FIG. 5 is a flowchart for explaining an example of the operation of the terminal device 20 in the online conference system 1 according to the embodiment. The processor 201 of the terminal device 20 participating in the online conference system accepts input of voice collected by the microphone 206 or input of voice (voice data) received from another terminal device 20 (ACT11). The processor 201 may be able to switch between an operation mode in which the voice input from the microphone 206 is enabled and an operation mode in which the voice input is disabled. For example, the processor 201 enables or disables the voice input from the microphone 206 in response to an instruction input by the user using the operation device 209.

[0042] When the voice input from the microphone 206 is invalid, the processor 201 inputs (receives) voice data from the other terminal device 20 without acquiring the input voice (ACT11, YES). When the processor 201 receives the voice data from the other terminal device 20, it outputs voice based on the voice data from the speaker 207. As a result, the terminal device 20 (terminal devices 22 and 23 as second terminal devices) outputs the input voice input by the other terminal device 20 (terminal device 21 as first terminal device) from the speaker 207.

[0043] When the voice input from the microphone 206 is valid, the processor 201 acquires the sound collected by the microphone 206 as the input voice via the voice processing circuit 205 (ACT11, YES). The processor 201 transmits (distributes) voice data generated from the acquired input voice to the other terminal device 20. This allows the processor 201 of the terminal device 20 (for example, the terminal device 21 as the first terminal device) to transmit (distribute) the voice (input voice) of the speaker collected by the microphone 206 as voice data to the other terminal device 20 (for example, the terminal devices 22 and 23 as the second terminal device). Note that when the voice input from the microphone 206 is valid, the processor 201 also executes a process of outputting a voice based on the voice data received from the other terminal device 20 from the speaker 207 in parallel with the process of distributing the input voice to the other terminal device 20.

[0044] When an input voice collected by the microphone 206 is acquired via the voice processing circuit 205 (ACT11, YES), the processor 201 performs a voice recognition process on the input voice (ACT12). The processor 201 stores a voice recognition result for the input voice in the memory area 213 of the auxiliary storage device 203 (ACT13). For example, the processor 201 stores the voice recognition result in the memory area 213 in association with time information indicating the time when the input voice was input. Furthermore, the processor 201 also stores information indicating that the voice recognition result is a voice recognition result for the input voice collected by the microphone 206.

[0045] Furthermore, when voice data from another terminal device 20 is received by the communication I / F 204 (ACT11, YES), the processor 201 performs voice recognition processing on the received voice data (ACT12). The processor 201 stores the voice recognition result for the voice data received from the other terminal device 20 in the memory area 213 of the auxiliary storage device 203 (ACT13). For example, the processor 201 stores the voice recognition result in the memory area 213 in association with time information indicating the time when the voice data was input. Furthermore, the processor 201 also stores information indicating that the voice recognition result is the voice recognition result for the voice data received from the other terminal device.

[0046] Here, the voice recognition process for the input voice and the voice recognition process for the received voice data are executed by the same voice recognition program. Also, the voice recognition process executed by each terminal device 20 is executed by a voice recognition program configured with the same algorithm. However, the voice recognition programs executed by each terminal device 20 may be different programs as long as there is no difference in the recognition results for the same voice that is greater than or equal to a threshold value.

[0047] Furthermore, the processor 201 determines whether or not to transmit the voice recognition result stored in the storage area 213 to the server 10 (ACT14). The processor 201 transmits the voice recognition result stored in the storage area 213 to the server 10 based on a preset condition. For example, the processor 201 transmits the voice recognition result at predetermined time intervals. The processor 201 may also transmit the voice recognition result to the server 10 every time a series of sentences is stored as a voice recognition result. The processor 201 may also transmit the voice recognition result to the server 10 every time the amount of data of the untransmitted voice recognition result stored in the storage area 213 reaches a predetermined amount.

[0048] When it is determined that the voice recognition result is to be transmitted to the server 10, the processor 201 transmits the untransmitted voice recognition result stored in the storage area 213 to the server 10 via the communication I / F 204 (ACT15). For example, the processor 201 transmits to the server 10 the voice recognition result in which additional information such as time information is associated with each series of sentences (text) obtained by voice recognition.

[0049] Furthermore, the processor 201 accepts a warning from the server 10 during the online conference (ACT16). When the processor 201 receives a notification indicating a warning from the server 10, it issues a warning according to the contents of the notification (ACT17). For example, it is assumed that after the terminal A distributes the input voice (speaker's speech) input to the microphone 206 to the terminal B, the terminal A receives a warning from the server 10 that the input voice is not being output normally at the terminal B. In this case, the processor 201 of the terminal A displays a warning on the display device 208 that the input voice (speaker's speech) is not being output normally at the terminal B.

[0050] This allows the terminal device of the speaker (first terminal device) to report the terminal device (second terminal device) from which the speaker's speech is not being output normally. As a result, the speaker using the first terminal device can recognize the terminal device from which the speaker's speech is not being output normally without interrupting the online conference.

[0051] Next, the operation of the server 10 in the online conference system 1 according to the embodiment will be described. FIG. 6 is a flowchart for explaining an example of the operation of the server 10 in the online conference system 1 according to the embodiment. The processor 101 of the server 10 communicates with each terminal device 20 participating in the online conference by the online conference system 1. The processor 101 receives a speech recognition result from each terminal device 20 via the communication I / F 104 (ACT31).

[0052] When a voice recognition result is received from a certain terminal device 20 (ACT31, YES), the processor 201 stores the received voice recognition result in the auxiliary storage device 103 (ACT32). For example, the processor 201 stores the voice recognition results received from each terminal device 20 in the storage area 113 of the auxiliary storage device 103 in association with each other by time. Also, as shown in FIG. 4, the processor 201 may store the voice recognition result (voice recognition result for voice data of input voice received via a network) by the listener's terminal device (second terminal device) 20 in association with the voice recognition result (voice recognition result for input voice) by the speaker's terminal device (first terminal device) 20 in the storage area 113.

[0053] When the speech recognition results received from the terminal devices 20 are stored, the processor 201 compares the stored speech recognition results (ACT33). The processor 201 associates the speech recognition result for the input speech input by the terminal device 20 of the speaker with the speech recognition result for the speech data of the input speech received by the terminal device 20 of the listener. The processor 201 calculates the difference between the speech recognition result for the input speech and the speech recognition result for the speech data received by the other terminal device 20. For example, the processor 201 quantifies the difference between the two corresponding speech recognition results using the Levenshtein distance.

[0054] Here, it is assumed that the processor 201 of each terminal device 20 uses the same voice recognition program for voice recognition. When voice data of an input voice output from a certain terminal device (first terminal device) is accurately transmitted to another terminal device (second terminal device), the input voice and the output voice based on the voice data of the input voice match. In this case, the voice recognition result for the input voice by the first terminal device and the voice recognition result for the voice data of the input voice by the second terminal device also match. On the other hand, when the voice data of the input voice output from the first terminal device is not accurately transmitted to the second terminal device, the input voice and the output voice based on the voice data of the input voice do not match. In this case, the voice recognition result for the input voice by the first terminal device and the voice recognition result for the voice data of the input voice by the second terminal device do not match.

[0055] An input voice input to a first terminal device is converted into text based on the voice recognition result of the input voice by the first terminal device. An output voice based on the voice data of the input voice received by the second terminal device from the first terminal device is converted into text based on the voice recognition result of the voice data (output voice) of the input voice received by the second terminal device. Therefore, the difference between the voice recognition result by the first terminal device and the voice recognition result by the second terminal device is a value indicating the degree to which the input voice input by the first terminal device is accurately output by the second terminal device. For example, the more unstable the communication path from the first terminal device to the second terminal device, the greater the difference between the voice recognition result by the first terminal device and the voice recognition result by the second terminal device.

[0056] The processor 201 determines whether to issue a warning based on the difference between the voice recognition result for the input voice (the voice recognition result by the first terminal device) and the voice recognition result for the voice data received by the other terminal device 20 (the voice recognition result by the second terminal device) (ACT34). For example, the processor 201 determines whether the difference between the voice recognition result for the input voice and the voice recognition result for the voice data (output voice) received by the other terminal device 20 exceeds a predetermined threshold. The predetermined threshold is set to a level at which the user can recognize the input voice and the output voice as the same content.

[0057] If the difference between the speech recognition result for the input speech and the speech recognition result for the output speech exceeds a predetermined threshold, the processor 201 determines to issue a warning. If the difference between the speech recognition result for the input speech and the speech recognition result for the output speech is equal to or smaller than a predetermined threshold, the processor 201 determines that it is not necessary to issue a warning.

[0058] The processor 201 may compare the difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device with a plurality of thresholds. For example, a first threshold and a second threshold smaller than the first threshold may be set as the plurality of thresholds. The processor 201 may issue a first warning when the first threshold is exceeded, and issue a second warning when the first threshold is not exceeded and exceeds the second threshold. This enables the server 10 to issue a warning according to the difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device.

[0059] The processor 201 may also store the difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device in chronological order. In this case, the processor 201 may issue a warning according to a change in the difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device in chronological order. For example, the processor 201 may issue a warning when the difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device tends to become larger.

[0060] When it is determined that a warning is necessary (ACT34, YES), the processor 201 notifies the terminal device (first terminal device) 20 that input the input voice of a warning (ACT35). The processor 201 identifies the terminal device 20 that performed voice recognition on the input voice as the first terminal device. For example, the processor 201 identifies the terminal device 20 that is the transmission source of the voice recognition result for the input voice as the first terminal device. When the processor 201 identifies the terminal device (first terminal device) that input the input voice, it transmits a warning to the first terminal device that is the transmission source of the input voice, notifying that the input voice is not being normally sent by another terminal device.

[0061] The processor 201 may also identify a second terminal device that is a transmission source of a speech recognition result of an output speech in which the difference between the input speech and the speech recognition result exceeds a threshold value. When the processor 201 identifies the second terminal device, the processor 201 transmits a warning to the first terminal device that is the transmission source of the input speech, notifying that the input speech is not normally transmitted to the identified second terminal device.

[0062] The processor 201 may issue a warning to multiple terminal devices or a preset terminal device without specifying the terminal device (first terminal device) 20 that input the input voice. For example, the processor 201 may issue a warning to all terminal devices 20 participating in the online conference (or all terminal devices that have transmitted the voice recognition results). The processor 201 may also issue a warning to a preset terminal device such as a terminal device used by the organizer.

[0063] The processor 201 of the server 10 repeatedly executes the above-mentioned processes of ACT31-35 while the online conference continues (ACT36, NO). The processor 201 may also end the processes of ACT31-35 when it receives an instruction to stop the process of notifying the speaker of a warning.

[0064] The above-mentioned processing of the server 10 may be executed by any one of the terminal devices 20. That is, the online conference system 1 may be configured by having any one of the terminal devices 20 execute the above-mentioned processing of the server 10. For example, the terminal device 20 can execute the above-mentioned processing by installing a program that executes the above-mentioned processing of the server 10. This makes it possible to configure an online conference system consisting of a plurality of terminal devices 20 without providing a server 10.

[0065] According to the above process, the server of the online conference system according to the embodiment acquires a speech recognition result for an input speech from a first terminal device. The server acquires, from a second terminal device, a speech recognition result for speech data of the input speech received by the second terminal device from the first terminal device. The server determines a difference between the speech recognition result for the input speech acquired from the first terminal device and the speech recognition result for the speech data of the input speech acquired from the second terminal device.

[0066] In this way, the server according to the embodiment can evaluate whether the input voice inputted in the first terminal device is normally outputted in the second terminal device, and as a result, it is also possible to evaluate the communication status between the first terminal device and the second terminal device.

[0067] Furthermore, the server issues a warning when a difference between a speech recognition result for the input speech and a speech recognition result for the speech data of the input speech received by the second terminal device exceeds a threshold, thereby making it possible to notify that the input speech inputted at the first terminal device is not being normally outputted at the second terminal device.

[0068] Furthermore, the server issues a warning to the first terminal device when the difference between the speech recognition result for the input speech and the speech recognition result for the speech data of the input speech received by the second terminal device exceeds a threshold. This makes it possible to notify the speaker, who is the user of the first terminal device, that the input speech input at the first terminal device is not being output normally at the second terminal device. As a result, the speaker can recognize during the online conference that his or her remarks are not being output normally at the terminal devices of the listeners.

[0069] In the above-mentioned embodiment, the case where the program executed by the processor is stored in advance in the memory in the device has been described. However, the program executed by the processor may be downloaded to the device from a network, or may be installed in the device from a storage medium. The storage medium may be any storage medium capable of storing a program, such as a CD-ROM, and readable by the device. Furthermore, the function obtained by pre-installation or downloading may be realized in cooperation with an OS (operating system) in the device.

[0070] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. The following is an appended summary of the contents of the claims as originally filed. [1] a communication interface for communicating with a first terminal device that transmits voice data generated from an input voice and a second terminal device that outputs a voice based on the voice data received from the first terminal device; a memory that stores a speech recognition result by the first terminal device for an input speech input to the first terminal device, and a speech recognition result by the second terminal device for speech data of the input speech received by the second terminal device from the first terminal device; a processor that determines a difference between an input voice input to the first terminal device and a voice that the second terminal device outputs based on voice data of the input voice received from the first terminal device, based on a comparison between a voice recognition result by the first terminal device and a voice recognition result by the second terminal device; A server having: [2] When a difference between the speech recognition result by the first terminal device and the speech recognition result by the second terminal device exceeds a threshold, the processor outputs a warning indicating that an input speech input to the first terminal device does not match a speech output by the second terminal device based on speech data of the input speech received from the first terminal device. [1] The server described in the above. [3] When a difference between a speech recognition result by the first terminal device and a speech recognition result by the second terminal device exceeds a threshold, the processor transmits a warning to the first terminal device indicating that the input speech is not normally output by the second terminal device. [1] The server described in the above. [4] A communication interface for communicating with a server and other terminal devices; Transmitting voice data of an input voice collected by a microphone to another terminal device and transmitting a voice recognition result for the input voice to the server; outputting from a speaker a voice based on voice data received from another terminal device via the communication interface, and transmitting a voice recognition result for the voice data to the server; a processor that issues a warning using an alarm device when a notification is received from the server that an input voice does not match a voice output based on voice data of the input voice received by another terminal device; A terminal device having the above configuration. [5] A memory for storing a speech recognition result is provided. the processor stores in the memory a speech recognition result for the input speech and a speech recognition result for the speech data, and transmits the speech recognition result to the server each time an amount of data of the untransmitted speech recognition result stored in the memory reaches a predetermined amount. [4] A terminal device according to the present invention. [6] A server having a communication interface for communicating with a plurality of terminal devices participating in an online conference, a first terminal device that transmits voice data generated from an input voice to another terminal device; storing in a memory a voice recognition result by the first terminal device for the input voice received from the first terminal device via a communication interface; a second terminal device that outputs a voice based on the voice data received from the first terminal device, and stores in a memory a voice recognition result by the second terminal device for the voice data of the input voice received via a communication interface from the second terminal device; determining a difference between an input voice input to the first terminal device and a voice output by the second terminal device based on voice data of the input voice received from the first terminal device, based on a comparison between a voice recognition result by the first terminal device and a voice recognition result by the second terminal device; A program for online meetings that runs [Explanation of symbols]

[0071] 10...server, 20 (21, 22, 23)...terminal device, 101...processor, 103...auxiliary storage device (memory), 104...communication interface, 201...processor, 203...auxiliary storage device (memory), 204...communication interface, 205...audio processing circuit, 206...microphone, 207...speaker.

Claims

1. a communication interface for communicating with a first terminal device that transmits voice data generated from an input voice and a second terminal device that outputs a voice based on the voice data received from the first terminal device; a memory for storing a speech recognition result by the first terminal device for an input speech input to the first terminal device, and a speech recognition result by the second terminal device for speech data of the input speech received by the second terminal device from the first terminal device; a processor that outputs a first warning when a difference between a speech recognition result by the first terminal device and a speech recognition result by the second terminal device exceeds a first threshold, and outputs a second warning when the difference exceeds a second threshold; A server having:

2. the processor, when there are a plurality of second terminal devices outputting a voice based on the voice data of the input voice, identifies a second terminal device in which a difference between a voice recognition result by the first terminal device and a voice recognition result by the second terminal device for the voice data of the input voice exceeds the first or second threshold, and transmits the first or second warning to the first terminal device together with information indicating the identified second terminal device; The server of claim 1 .

3. A memory; A communication interface for communicating with a server and other terminal devices; Transmitting voice data of an input voice collected by a microphone to another terminal device and storing a voice recognition result for the input voice in the memory; outputting from a speaker a voice based on voice data received from another terminal device via the communication interface, and storing a voice recognition result for the voice data in the memory; transmitting the untransmitted speech recognition results to the server each time the amount of data of the untransmitted speech recognition results stored in the memory reaches a predetermined amount; a processor that issues a warning using an alarm device when a notification is received from the server that an input voice does not match a voice output based on voice data of the input voice received by another terminal device; A terminal device having the above configuration.

4. The processor stores in the memory a voice recognition result for the input voice in association with time information indicating the time the input voice was collected and information indicating that the voice recognition result is for the input voice, and stores in the memory a voice recognition result for the voice data in association with time information indicating the time the voice data was input and information indicating that the voice recognition result is for voice data received from another terminal device. The terminal device according to claim 3.

5. A server having a communication interface for communicating with a plurality of terminal devices participating in an online conference, a first terminal device that transmits voice data generated from an input voice to another terminal device; storing in a memory a voice recognition result by the first terminal device for the input voice received from the first terminal device via a communication interface; a second terminal device that outputs a voice based on the voice data received from the first terminal device; storing in a memory a voice recognition result by the second terminal device for the voice data of the input voice received via a communication interface from the second terminal device; outputting a first warning when a difference between a speech recognition result by the first terminal device and a speech recognition result by the second terminal device exceeds a first threshold, and outputting a second warning different from the first warning when the difference exceeds a second threshold different from the first threshold; A program for online meetings that runs

Images