Voice playback control system, voice playback control method, and voice playback control program

Abstract

To provide a speech reproduction control system that reproduces a plurality of speeches in parallel in a distinguishable manner.SOLUTION: A speech reproduction control system includes: a speech reproducer 17 that reproduces speech data in speech; a character reproducer 19 that converts the speech data into characters for reproduction; and a parallel reproduction controller 15 that reproduces first speech data and second speech data in parallel. Therein, the parallel reproduction controller 15 reproduces the first speech data by using the character reproducer 19, turns off reproduction by the speech reproducer 17 or performs the reproduction at a volume lower than a volume for reproduction of the second speech data, and reproduces the second speech data by using the speech reproducer 17.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an audio reproduction control system, an audio reproduction control method, and an audio reproduction control program, and particularly to an audio reproduction control system suitable for reproducing a plurality of types of audio, an audio reproduction control method executed by the audio reproduction control system, and an audio reproduction control program for causing a computer to execute the audio reproduction control method.

Background Art

[0002] In recent years, voice call systems in which a plurality of users located at remote locations communicate via a computer, such as for meetings, have become widespread. For example, Japanese Patent Application Laid-Open No. 2021-184189 discloses an online meeting system connected to a plurality of terminals via a network for realizing an online meeting between the terminals, having a distribution control unit that mutually distributes audio data and image data between the plurality of terminals, and when the distribution control unit receives a limited conversation instruction to specify a part of the plurality of terminals including the terminal as a specific terminal and conduct a conversation only between the specific terminals during the online meeting, performs limited distribution that prohibits distributing the audio data exchanged between the specific terminals in a state where it can be reproduced by other terminals is disclosed.

[0003] However, in the online meeting system described in Japanese Patent Application Laid-Open No. 2021-184189, a participant who has instructed a limited conversation must listen to the conversation in the online meeting by voice and must identify by voice which of the limited conversation and the online meeting conversation it is. Therefore, when it is difficult to identify the voice, a participant who has instructed a limited conversation may have difficulty distinguishing between the limited conversation and the online meeting conversation.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] One of the objectives of this invention is to provide an audio playback control system that can play multiple audios in parallel while maintaining their identifiable characteristics.

[0006] Another object of this invention is to provide an audio playback control method that enables the simultaneous playback of multiple audio signals in a distinguishable manner.

[0007] Another object of this invention is to provide an audio playback control program that can play multiple audios in parallel in a distinguishable manner. [Means for solving the problem]

[0008] According to one aspect of this invention, the audio playback control system comprises an audio playback means for playing audio data as audio, a text playback means for converting audio data into text and playing it back, and a parallel playback control means for playing first audio data and second audio data in parallel, wherein the parallel playback control means plays the first audio data by the text playback means and plays it back by the audio playback means either OFF or at a lower volume than the second audio data, and plays the second audio data by the audio playback means. From this state, the system switches to a state where the second audio data is played back by the text playback means, and playback by the audio playback means is turned OFF or played back at a lower volume than the first audio data, and the first audio data is played back by the audio playback means. .

[0009] According to another aspect of this invention, the audio playback control method causes the audio playback control device to perform an audio playback step of playing audio data as audio, a text playback step of converting audio data into text and playing it back, and a parallel playback control step of playing first audio data and second audio data in parallel, wherein the parallel playback control step plays the first audio data in the text playback step and does not play it back in the audio playback step or plays it back at a lower volume than the second audio data, and plays the second audio data back in the audio playback step. From this state, the system switches to a state where the second audio data is played in the text playback step, but not played in the audio playback step, or played at a lower volume than the first audio data, and the first audio data is played in the audio playback step. .

[0010] According to yet another aspect of this invention, the audio playback control program causes the computer to perform an audio playback step of playing audio data as audio, a text playback step of converting the audio data into text and playing it back, and a parallel playback control step of playing first audio data and second audio data in parallel, wherein the parallel playback control step plays the first audio data in the text playback step but does not play it in the audio playback step or plays it at a lower volume than the second audio data, and plays the second audio data in the audio playback step. From this state, the system switches to a state where the second audio data is played in the text playback step, but not played in the audio playback step, or played at a lower volume than the first audio data, and the first audio data is played in the audio playback step. . [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of the system configuration of a remote conferencing system in one embodiment of the present invention. [Figure 2] A block diagram showing an example of a server hardware configuration. [Figure 3] A block diagram showing an example of a PC hardware configuration. [Figure 4] This figure shows an example of the functions of the CPU provided by the server in this embodiment. [Figure 5] This figure shows an example of the functions of the CPU in the PC in this embodiment. [Figure 6] Figure 1 shows an example of the operation screen. [Figure 7] This figure shows an example of an invitation instruction screen. [Figure 8] Figure 1 shows an example of an invitation acceptance screen. [Figure 9] Figure 1 shows an example of a parallel playback screen. [Figure 10] The second figure shows an example of a parallel playback screen. [Figure 11] The third figure shows an example of a parallel playback screen. [Figure 12] Figure 4 shows an example of a parallel playback screen. [Figure 13] The second figure shows an example of the operation screen. [Figure 14] It is the fifth figure showing an example of a parallel playback screen. [Figure 15] It is the second figure showing an example of an invitation reception screen. [Figure 16] It is a flowchart showing an example of the flow of audio playback control processing. [Figure 17] It is a flowchart showing an example of the flow of the first parallel call control processing. [Figure 18] It is a flowchart showing an example of the flow of the second parallel call control processing. [Figure 19] It is a figure showing an example of the functions of the CPU included in the server in the modification example. [Figure 20] It is a figure showing an example of the functions of the CPU included in the PC in the modification example.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0013] FIG. 1 is a diagram showing an example of the system configuration of a remote conference system in one embodiment of the present invention. The remote conference system 1 is an example of an audio playback control system. Referring to FIG. 1, the remote conference system 1 includes a server 100 and personal computers (hereinafter referred to as "PCs") 200-1, 200-2, 200-3, 200-4 to 200-N. However, N is a positive integer, and here it is 5 or more. Each of the server 100 and the PCs 200-1 to N is connected to the Internet 5 and can communicate with each other.

[0014] Each of the PCs 200-1 to N includes a camera, a microphone for collecting sound, and a speaker for outputting sound. Each of the PCs 200-1 to N is a general computer, and its main hardware configuration and functions are the same.

[0015] In addition, any device equipped with a camera, microphone, speaker, and communication functions may be used instead of PC200-1~N, such as a PDA (Personal Digital Assistance) or smartphone. Furthermore, the network is not limited to the Internet 5; any other network is acceptable as long as Server 100 and PC200-1~N can communicate with each other. The network may be, for example, a local area network (LAN) or a wide area network (WAN).

[0016] In remote conferencing system 1, participants join a meeting by operating one of the PCs from PC200-1 to PC200-N. Hereafter, any one of PC200-1 to PC200-N will be referred to as PC200.

[0017] Each of the PCs from PC200-1 to PC200-N has a program installed for participating in the conference, and the conference is held when each of the PCs from PC200-1 to PC200-N communicates with server 100. The program installed on each of the PCs from PC200-1 to PC200-N may be a dedicated program for communicating with server 100, or a general browser program if server 100 provides a web service.

[0018] Server 100 enables the remote conferencing system by executing an audio playback control program. Server 100 communicates with PCs 200-1 to N and transmits the data it receives from each of PCs 200-1 to N to the other PCs 200-1 to N.

[0019] The data transmitted and received between each of the PCs 200-1 to N and the server 100 includes audio data representing sound, image data representing images, and application data. Images include still images and moving images. The data transmitted and received between each of the PCs 200-1 to N and the server 100 may be compressed or uncompressed.

[0020] Server 100 manages the data to be sent to each of the PCs 200-1 through N. For example, Server 100 can send audio data received from each of the PCs 200-1 through N to all of them. Furthermore, regarding image data, Server 100 can aggregate the image data received from each of the PCs 200-1 through N and send it to each of them.

[0021] Server 100 determines and transmits the audio data, image data, and application data to be sent in response to requests from each of the PCs 200-1 to N. Therefore, the images displayed on each of the PCs 200-1 to N may be the same or different, and the audio output from each of the PCs 200-1 to N may be the same or different. Furthermore, each of the PCs 200-1 to N may process multiple audio data received from Server 100 and output it. In this case, the processing of audio data on Server 100 becomes unnecessary, thus reducing the load. Also, each of the PCs 200-1 to N may process multiple image data received from Server 100 and display it. In this case, the processing of multiple image data on Server 100 becomes unnecessary, thus reducing the load.

[0022] Figure 2 is a block diagram showing an example of a server hardware configuration. Referring to Figure 2, the server 100 is a computer that performs arithmetic processing and includes a central processing unit (CPU) 101 for controlling the entire server 100, each connected to a bus 113; a ROM (Read Only Memory) 102 for storing programs to be executed by the CPU 101; a RAM (Random Access Memory) 103 used as a workspace for the CPU 101; an HDD 104 for non-volatile data storage; a communication unit 105 for connecting the CPU 101 to the internet 5; a display unit 106 for displaying images; an operation unit 107 for receiving operation inputs; and an external storage device 110.

[0023] The communication unit 105 is an interface for connecting the server 100 to the internet 5. Therefore, the CPU 101 can communicate with PCs 200-1 to 100N connected to the internet 5 via the communication unit 105.

[0024] The external storage device 110 is fitted with a CD-ROM (Compact Disk Read Only Memory) 111. The CPU 101 controls the external storage device 110 to read the data stored on the CD-ROM 111.

[0025] In this embodiment, the CPU 101 executes a program stored in the ROM 102 or HDD 104. Alternatively, the CPU 101 may control the external storage device 110 to read a program for execution from the CD-ROM 111, store the read program in the RAM 103, and then execute it.

[0026] Furthermore, the CPU 101 downloads programs from computers connected to the Internet 5 and stores them in the HDD 104. Also, when a computer connected to the Internet 5 writes a program to the HDD 104, the program is stored in the HDD 104. The CPU 101 may also load the program stored in the HDD 104 into the RAM 103 and execute it.

[0027] Furthermore, the recording medium for storing the program to be executed by the CPU 101 is not limited to the CD-ROM 111, but may also be a flexible disk, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, mask ROM, EPROM (Erasable Programmable ROM), or other semiconductor memory. The program referred to here includes not only programs that can be directly executed by the CPU 101, but also source programs, compressed programs, encrypted programs, etc.

[0028] Figure 3 is a block diagram showing an example of the hardware configuration of a PC. Referring to Figure 3, the PC200 is a computer that performs arithmetic processing and includes a CPU201 for controlling the entire PC200, a ROM202 for storing programs to be executed by the CPU201, a RAM203 used as a workspace for the CPU201, an HDD204 for non-volatile data storage, a communication unit205 for connecting the CPU201 to the Internet 5, a display unit206 for displaying images, an operation unit207 for receiving input from participants who are users, a camera208 for capturing images of participants, a speaker209 for outputting sound, a microphone210 for collecting the operator's voice, and an external storage device211.

[0029] The external storage device 211 is fitted with a CD-ROM 211A. The CPU 201 controls the external storage device 211 to read the data stored on the CD-ROM 211A.

[0030] A module comprising at least two of the following components—a camera 208, a speaker 209, and a microphone 210—may be connected to the PC 200. The module may include, for example, a headset comprising the speaker 209 and microphone 210.

[0031] Figure 4 shows an example of the functions of the CPU in the server in this embodiment. The functions shown in Figure 4 are functions realized by the CPU 101 of the server 100 when the CPU 101 executes a terminal-side voice playback control program stored in ROM 102, HDD 104, or CD-ROM 111. The terminal-side voice playback control program is a part of the voice playback control program. Referring to Figure 4, the CPU 101 includes a call unit 11, a parallel call control unit 13, a parallel playback control unit 15, a voice playback unit 17, and a text playback unit 19.

[0032] The call unit 11 controls calls between multiple users. The call unit 11 enables calls between two or more users in a group consisting of two or more users. The call unit 11 transmits the voice of one user in a group to all other users in that group. The call unit 11 treats calls within each group as separate calls. In other words, the call unit 11 treats a call in one group as a separate call from calls in other groups. Therefore, a call in one group is not transmitted to users in other groups. Note that the same user may be included in multiple groups.

[0033] The call unit 11 determines the group of users making a call and determines which PC each of the multiple users belonging to the same group will operate. The call unit 11 transmits the data received from one PC 200 to one or more other PCs 200 belonging to the same group.

[0034] The following explanation uses the example of N users P-1 to N each operating PC200-1 to N. The data transmitted and received by the communication unit 105 between PC200-1 to N includes at least audio data. The data transmitted and received by the communication unit 105 between PC200-1 to N may also include image data, including still images or moving images.

[0035] For example, when users P-1 to M (where M is an integer smaller than N) participate in a conference, the call unit 11 determines that the PCs 200-1 to M operated by each user P-1 to M belong to the same group. The call unit 11 controls the communication unit 105 to receive voice data from one of the PCs 200-1 to M belonging to the same group, for example, PC 200-1, and transmits the received voice data to all PCs 200-2 to M except PC 200-1. Also, when user P-1 and user P-2 are in a call, the call unit 11 determines that the PCs 200-1 and 200-2 operated by users P-1 and P-2 belong to the same group. The call unit 11 transmits the data received from either PC 200-1 or PC 200-2 to the other.

[0036] The parallel call control unit 13 executes calls in the first group and calls in the second group in parallel. If there is a specific user belonging to both the first group and the second group, the parallel call control unit 13 executes calls in the first group and calls in the second group in parallel for that specific user. The parallel call control unit 13 outputs the audio data of the calls in the first group and the audio data of the calls in the second group to the parallel playback control unit 15.

[0037] The parallel call control unit 13 may prohibit calls in the second group. For example, if any of the users belonging to the first group specify that calls in the second group should be prohibited, calls in the second group will be prohibited. The parallel call control unit 13 will prohibit calls in the second group if it is determined that calls in the second group should be prohibited.

[0038] The parallel playback control unit 15 outputs the audio data of the calls in the first group and the audio data of the calls in the second group to the audio playback unit 17 and the text playback unit 19. The parallel playback control unit 15 controls the audio playback unit 17 and the text playback unit 19 to play back the audio data of the calls in the first group and the audio data of the calls in the second group in parallel.

[0039] The audio playback unit 17 plays back audio data as sound. The audio playback unit 17 outputs the played-back audio to the parallel call control unit 13.

[0040] The text playback unit 19 plays back the audio data by converting it into text. Specifically, the text playback unit 19 converts the audio data into text by performing speech recognition. The text playback unit 19 outputs text information consisting of the text converted from the audio data to the parallel call control unit 13. The text playback unit 19 may also identify the user who is speaking from the audio data and output text information to the parallel call control unit 13 with user identification information added to identify the identified user. Alternatively, the text playback unit 19 may translate the text information converted from the audio data into another language and output the translated text information to the parallel call control unit 13.

[0041] The parallel playback control unit 15 controls the text playback unit 19 to play back first audio data indicating a call in the first group. In parallel with the playback of the first audio data by the text playback unit 19, the parallel playback control unit 15 controls the audio playback unit 17 to play back second audio data for a call in the second group, and either does not play back the first audio data or plays back the first audio data at a lower volume than the second audio data. The audio playback unit 17 treats the voice of a specific user as a call in the second group, not a call in the first group. Therefore, the voice spoken by a specific user is not included in the first audio data, but is included in the second audio data.

[0042] The parallel call control unit 13 outputs user identification information for identifying a specific user, audio data of the audio played back by the audio playback unit 17, and text information played back by the text playback unit 19 to the call unit 11.

[0043] The communication unit 11 transmits voice data and text information to the PC 200 operated by a specific user. Alternatively, instead of transmitting text information, the text information may be superimposed on the image data transmitted to the PC 200 operated by the specific user.

[0044] The parallel playback control unit 15 executes the calls in the first group and the second group in parallel when a call in the second group is started while a call in the first group is continuing in the call unit 11. Alternatively, the parallel playback control unit 15 may execute the calls in the first group and the second group in parallel when it receives a start instruction from a specific user indicating the start of a parallel call after a call in the second group has started while a call in the first group is continuing in the call unit 11.

[0045] Furthermore, the parallel playback control unit 15 may switch between playing back the call in the first group and playing back the call in the second group if it receives a switching instruction from a specific user while simultaneously executing calls in the first group and the second group. Specifically, in response to receiving a switching instruction, the parallel playback control unit 15 controls the text playback unit 19 to play back the second audio data, controls the audio playback unit 17 to play back the first audio data, and either does not play back the second audio data or plays back the second audio data at a lower volume than the first audio data.

[0046] Figure 5 shows an example of the functions of the CPU in the PC in this embodiment. The functions shown in Figure 5 are functions realized by the CPU 201 of the PC 200 when it executes a playback program stored in the ROM 202, HDD 204, or CD-ROM 211A. The playback program is part of the audio playback control program. Referring to Figure 5, the CPU 201 of the PC 200 includes a terminal-side call unit 251, an audio input control unit 253, an audio output control unit 255, a display control unit 257, and an image input control unit 259.

[0047] The terminal-side call unit 251 controls calls between the user operating the PC200 and other users. There are one or more other users. The terminal-side call unit 251 enables calls between the user operating the PC200 and one or more other users in a group consisting of the user operating the PC200 and one or more other users.

[0048] The terminal-side call unit 251 controls the communication unit 205 to receive voice data and text information from the server 100. The terminal-side call unit 251 outputs the voice data received from the server 100 to the voice output control unit 255 and outputs the text information received from the server 100 to the display control unit 257. If image data is received from the server 100, the terminal-side call unit 251 outputs the image data to the display control unit 257.

[0049] The display control unit 257 displays an image of the character information on the display unit 206 in response to the input of character information from the terminal-side call unit 251. The display control unit 257 displays an image of the image data on the display unit 206 in response to the input of image data from the terminal-side call unit 251. In addition, the display control unit 257 displays an image of the character information and an image of the image data on the display unit 206 in response to the input of both character information and image data from the terminal-side call unit 251. The image containing the text information and the image data may be displayed separately, or the image containing the text information may be superimposed on the image data.

[0050] The audio output control unit 255 plays the audio data. Specifically, the audio output control unit 255 converts the digital audio data into an analog signal and outputs the analog audio data to the speaker 209. As a result, the audio data is output from the speaker 209.

[0051] The voice input control unit 253 receives analog audio data output from the microphone 210. The voice input control unit 253 converts the analog audio data into digital audio data and outputs the converted audio data to the terminal-side call unit 251. The voice input control unit 253 may also compress the audio data and output the compressed audio data to the terminal-side call unit 251. The audio data that the voice input control unit 253 outputs to the terminal-side call unit 251 corresponds to the voice spoken by the user operating the PC 200.

[0052] The image input control unit 259 receives analog image data output from the camera 208. The image input control unit 259 converts the analog image data into digital image data and outputs the converted image data to the terminal-side call unit 251.

[0053] The terminal-side call unit 251 controls the communication unit 205 to transmit voice data input from the voice input control unit 253 and image data input from the image input control unit 259 to the server 100.

[0054] The operation of the audio playback system will be explained below, using PC200-1, operated by user P-1, as an example.

[0055] Figure 6 is the first diagram showing an example of the operation screen. The operation screen 300 is the screen displayed on the display unit 206 of PC 200-1 when five users P-1 to P-5 are holding a conference. The operation screen 300 displays images 301-1 to P-5. Images 301-1 to P-5 are images captured by the cameras 208 equipped on each of PC 200-1 to P-5. Images 301-1 to P-5 correspond to users P-1 to P-5, respectively. In this state, the server 100 treats the conversation between the five users P-1 to P-5 as a conversation of the first group.

[0056] When the operation screen 300 is displayed on the display unit 206 of PC200-1, the playback mode is normal mode on both the server 100 and PCs 200-1 to 5. The voices spoken by each of the users P-2 to 5 are output as conference audio from the speaker 209 of PC200-1. In addition, the voice spoken by user P-1 is picked up by the microphone 210 of PC200-1 and output from the speakers 209 of each of the PCs 200-2 to 5.

[0057] A button 303 with the word "privacy" superimposed on the lower right of each of the images 301-2 to 5, excluding image 301-1 which corresponds to user P-1, is displayed. By using the button 303 with the word "privacy" superimposed on the lower right of each of the images 301-2 to 5, user P-1 can specify a call partner other than the one in the conference call they are currently on. Here, the state in which the button 303 displayed in the lower right of image 301-4 which corresponds to user P-4 is indicated is shown by hatching.

[0058] Figure 7 shows an example of an invitation instruction screen. Referring to Figure 7, the invitation instruction screen 305 is a pop-up screen and is displayed on the display unit 206 superimposed on the operation screen 300. The invitation instruction screen 305 is displayed when user P-1 instructs the button 303 displayed in the lower right corner of each of the images 301-2 to 305 on the operation screen 300. In Figure 7, the invitation instruction screen 305 is displayed when user P-1 instructs the button 303 displayed in the lower right corner of image 301-4 on the operation screen 300.

[0059] The invitation instruction screen 305 includes the message "Do you want to start a private conversation with Mr. D?", a button displaying the word "Yes", and a button displaying the word "No". When user P-1 presses the button displaying "Yes", the invitation acceptance screen is displayed on the display unit 206 of PC 200-4 operated by user P-4.

[0060] Figure 8 is the first diagram showing an example of an invitation acceptance screen. Here, the operation screen 300 and the invitation acceptance screen 307 displayed on the display unit 206 of PC200-4 operated by user P-4 are shown. Referring to Figure 8, the invitation acceptance screen 307 is a pop-up screen and is displayed on the display unit 206 superimposed on the operation screen 300. The invitation acceptance screen 307 includes the message "Person A has requested a private call," a button with the word "Allow," and a button with the word "Deny." When user P-4 presses the button with the word "Allow," a call between user P-1 and user P4 becomes possible. In this case, the playback mode switches to parallel playback mode on server 100, PC200-1, and PC200-4. In parallel playback mode, the call between user P-1 and user P-4 is treated as a call of the second group on server 100. If user P-4 presses the button displaying the word "Reject," the call between user P-1 and user P4 will not be established, and the playback mode on server 100, PC200-1, and PC200-4 will remain in normal mode and will not switch to parallel playback mode.

[0061] Figure 9 is the first diagram showing an example of a parallel playback screen. The parallel playback screen 310 shown in Figure 9 is displayed on the display unit 206 of the PC 200-1 operated by user P-1. The parallel playback screen 310 includes a first group display area 313, a button 311 that displays the string "Return to Meeting", and a button 312 that displays the string "Swap". In parallel playback mode, users P-1 to P-5 belong to the first group, and users P-1 and P-4 belong to the second group.

[0062] In the first group display area 313, the conversations of users P-2, 3, and 5, who belong to the first group but not to the second group, are displayed as text information. In Figure 9, the text information corresponding to the voices of users P-2, 3, and 5 is displayed. The text information is displayed from top to bottom in the order in which the voices were spoken. In addition, user identification information is displayed at the beginning of the text information to identify the user who spoke. Here, the user identification information for users P-2, 3, and 5 is B, C, and E, respectively.

[0063] The conversations of users P-1 and P-4, who belong to the second group, are played back as audio. In Figure 9, the audio is shown as text in speech bubbles. Note that the conversations of users P-1 and P-4, who belong to the second group, are not transmitted to users P-2, P-3, and P-5.

[0064] In this way, user P-1 can communicate with user P-4 by voice while recognizing the content of each user's utterances (P-2, P-3, and P-5) based on the character information displayed in the first group display area 313.

[0065] When user P-1 presses button 311, the call for the second group ends, and the system returns to having only the call for the first group. In this case, the operation screen 300 shown in Figure 6 is displayed on the display unit 206, and the calls of users P200-1 to P200-5 belonging to the first group are resumed by voice.

[0066] When user P-1 presses button 312, the playback of the first group and the playback of the second group are swapped.

[0067] Figure 10 is a second diagram showing an example of a parallel playback screen. The parallel playback screen 310 shown in Figure 10 is displayed on the display unit 206 of PC200-1 when user P-1 presses button 312 of the parallel playback screen 310 shown in Figure 9. The parallel playback screen 310 shown in Figure 10 includes a second group display area 314, a button 311 that displays the string "Return to Meeting", and a button 312 that displays the string "Swap". In parallel playback mode, users P200-1 to P200-5 belong to the first group, and users P200-1 and P200-4 belong to the second group.

[0068] The second group display area 314 is the area related to calls of the second group. In the second group display area 314, the calls of users P-1 and P-4 belonging to the second group are displayed as text information. In Figure 10, the text information corresponding to the voices of users P-1 and P-4 is displayed. The text information is displayed from top to bottom in the order in which the voices were spoken. In addition, user identification information is displayed at the beginning of the text information to identify the user who spoke. Here, user identification information A and D are used to identify users P-1 and P-4, respectively.

[0069] Calls from users P-2, 3, and 5, who belong to the first group but not to the second group, are played back as audio. In Figure 10, the audio is shown as text in speech bubbles. Calls from users P-1 and 4, who belong to the second group, are not transmitted to users P-2, 3, and 5.

[0070] In this way, user P-1 can recognize the content of the utterances of users P-1 and P-4 from the character information displayed in the second group display area 314, and can also confirm the utterances made by users P-2, P-3, and P-5 audibly.

[0071] When user P-1 presses button 311, the call for the second group ends, and the system returns to having only the call for the first group. In this case, the operation screen 300 shown in Figure 6 is displayed on the display unit 206, and the calls of users P-1 to P-5 belonging to the first group are resumed by voice.

[0072] When user P-1 presses button 312, the playback of the first group and the playback of the second group are swapped. In this case, the parallel playback screen 310 shown in Figure 9 is displayed on the display unit 206 of PC200-1.

[0073] Figure 11 is a third diagram showing an example of a parallel playback screen. The difference between the parallel playback screen 310 shown in Figure 11 and the parallel playback screen 310 shown in Figure 9 is the addition of a shared screen display area 315. Here, the screen shared by user P-2 in the first group is displayed in the shared screen display area 315. The screen displayed in the shared screen display area 315 is the screen that user P-2 has instructed to be shared in the first group by PC200-2, and is the screen displayed on the display unit 206 of PC200-2. The screen displayed in the shared screen display area 315 is transmitted from PC200-2 to PC2001,3~5 via server 100.

[0074] Furthermore, the conversations of users P-1 and P-4, who belong to the second group, are output as audio and are shown as text in speech bubbles in Figure 10.

[0075] Figure 12 is the fourth figure showing an example of a parallel playback screen. The parallel playback screen 310 shown in Figure 12 differs from the parallel playback screen 310 shown in Figure 9 in that a second group display area 314 has been added. Here, the first group display area 313 and the second group display area 314 are arranged side by side. The first group display area 313 is the area related to the call of the first group, while the second group display area 314 is the area related to the call of the second group. Here, the second group display area 314 displays an image of user P-4 belonging to the second group. Note that the second group display area 314 may be a screen shared by the second group.

[0076] Furthermore, the conversations of users P-1 and P-4, who belong to the second group, are output as audio and are shown as text in speech bubbles in Figure 11.

[0077] Figure 13 is a second diagram showing an example of the operation screen. The difference between the operation screen 300 shown in Figure 13 and the operation screen 300 shown in Figure 6 is the addition of a call list 320. The call list 320 is a table listing the user identification information of users that user P-1 can call. The user identification information included in the call list 320 includes users P-1 to P-5 belonging to the first group, as well as other users P-6 to P-N.

[0078] When user P-1 selects one or more user identification information displayed in the call list 320, the playback mode switches from normal mode to parallel playback mode. Here, we will explain using the case where user P-6 selects user identification information F as an example.

[0079] Figure 14 is the fifth figure showing an example of a parallel playback screen. The parallel playback screen 310 shown in Figure 14 is displayed on the display unit 206 of PC200-1 when user P-1 indicates the user identification information F displayed in the call list 320 of the operation screen 300 shown in Figure 13.

[0080] The parallel playback screen 310 shown in Figure 14 includes a first group display area 313, a button 311 that displays the string "Return to Meeting", and a button 312 that displays the string "Swap". In parallel playback mode, users P200-1 to P200-5 belong to the first group, and users P200-1 and P200-6 belong to the second group.

[0081] In the first group display area 313, the conversations of users P-2 to P-5, who belong to the first group but not to the second group, are displayed as text information. In Figure 14, the text information corresponding to the voices of each user P-2 to P-5 is displayed. The text information is displayed from top to bottom in the order in which the voices were spoken. In addition, user identification information is displayed at the beginning of the text information to identify the user who spoke. Here, the user identification information for users P-2 to P-5 is B, C, D, and E, respectively.

[0082] The conversations of users P-1 and P-6, who belong to the second group, are played back as audio. In Figure 14, the audio is shown as text in speech bubbles. Note that the conversations of users P-1 and P-6, who belong to the second group, are not transmitted to users P-2 through P-5.

[0083] In this way, user P-1 can communicate with user P-6 by voice while recognizing the content of each user's utterance from users P-2 to P-5 based on the character information displayed in the first group display area 313.

[0084] When user P-1 presses button 311, the call for the second group ends, and the system returns to having only the call for the first group. In this case, the operation screen 300 shown in Figure 13 is displayed on the display unit 206, and the calls of users P200-1 to P200-5 belonging to the first group are resumed by voice.

[0085] When user P-1 presses button 312, the playback of the first group and the playback of the second group are swapped.

[0086] Figure 15 is a second diagram showing an example of an invitation acceptance screen. Here, we show the invitation acceptance screen 308 that is displayed on the display unit 206 of PC200-1 when a call request is made to user P-1 by user P-6, who does not belong to the first group.

[0087] Referring to Figure 15, the invitation acceptance screen 308 is a pop-up screen and is displayed on the display unit 206 superimposed on the operation screen 300 shown in Figure 6. The invitation acceptance screen 308 includes the message "F has requested a call," a button with the word "Allow," and a button with the word "Deny." When user P-4 presses the button with the word "Allow," a call between user P-1 and user P-6 becomes possible, and the playback mode on PC200-1 switches to parallel playback mode. In parallel playback mode, the call between user P-1 and user P-6 is treated as a call for a second group on server 100. In this case, the parallel playback screen 310 shown in Figure 14 is displayed on the display unit 206 of PC200-1.

[0088] If user P-1 presses the button displaying the word "Reject," the call between user P-1 and user P6 will not be established, and PC200-1 will maintain the normal playback mode and will not switch to parallel playback mode.

[0089] Figure 16 is a flowchart illustrating an example of the audio playback control process. The audio playback control process is performed by the CPU 101 of the server 100, which executes an audio playback control program stored in the ROM 102, HDD 104, or CD-ROM 111. The server 100 controls the switching of audio data transmitted and received for each of the PCs 200-1 to N. The control for switching audio data for each of the PCs 200-1 to N is the same. Here, we will explain using the control of switching audio data transmitted and received between the server 100 and PC 200-1 as an example.

[0090] Referring to Figure 16, the CPU 101 of the server 100 determines whether or not the first call has started. It remains in a waiting state until the first call starts (NO in step S01), and if the first call has started (YES in step S01), the process proceeds to step S02. The first call is a call between user P-1 and one or more other users. Here, we will explain using the example of user P-1 making a call with four other users P-2 to P-5. The set of users P-1 to P-5 making a call in the first call constitutes the first group.

[0091] In step S02, the first call is established, and the process proceeds to step S03. PCs 200-1 to 5, operated by users P-1 to 5, are interconnected, enabling the transmission and reception of voice data between PCs 200-1 to 5. In addition to voice data, image data can also be transmitted and received between PCs 200-1 to 5.

[0092] In step S03, the first audio data is played, and processing proceeds to step S04. The first audio data is the voice of other users P-2 to P-5 belonging to the first group. Server 100 transmits the first audio data received from any of PC200-2 to P-5 to PC200-1. PC200-1 plays the first audio data received from Server 100 and outputs the audio of the first audio data from speaker 209.

[0093] In step S04, the first window is generated, and the process proceeds to step S05. The first window contains images related to the first call. Images related to the first call include, for example, video footage of other users P-2 to P-5 captured by camera 208, and images of data shared by the first group. In step S05, the first window is sent to PC 200-1, and the process proceeds to step S06. As a result, the first window is displayed on the display unit 206 on PC 200-1. Steps S03 and S04 are executed simultaneously.

[0094] In step S06, it is determined whether or not the instruction to start the second call has been received. The instruction to start the second call is received if user P-1 designates user P-4 as the recipient of the second call, or if any of the other users P-2 to N designate user P-1 as the recipient of the second call. The instruction to start the second call also includes an instruction to invite user P-1 to a call of a group other than the first group. If the instruction to start the second call is received, the process proceeds to step S07; otherwise, the process returns to step S03. However, if the second call is prohibited, the process returns to step S03 even if the instruction to start the second call is received. The second call is prohibited if any of the first groups in the first call have given an instruction to prohibit the second call. Here, we will explain using the example of user P-1 designating user P-4 as the recipient of the second call when the second call is not prohibited.

[0095] In step S07, the second call is established, and the process proceeds to step S08. PC200-1, operated by user P-1, and PC200-4, operated by user P-4, are connected, enabling the transmission and reception of audio data between PC200-1 and PC200-4. Server 100 transmits the second audio data received from PC200-2 to PC200-1. PC200-1 plays the second audio data received from Server 100 and outputs the audio of the second audio data from speaker 209. In addition to audio data, image data can also be transmitted and received between PC200-1 and PC200-4.

[0096] If user P-1 designates another user as the recipient of a second call, the second call may be connected if the designated user grants permission. Also, if user P-1 is designated as the recipient of a second call by any of the other users P-2 to N, the second call may be connected if user P-1 grants permission. Furthermore, if user P-1 is invited to a call of a group other than the first group, the second call may be connected if user P-1 grants permission.

[0097] In step S08, the first parallel call control process is executed, and the process proceeds to step S09. The details of the first parallel call control process will be described later, but it is a process that controls PC200-1 so that the second audio data of the second call is played back as audio, and the first audio data of the first call is played back in a different form than the playback of the second audio data.

[0098] In step S09, it is determined whether or not a switching instruction has been received. A switching instruction is an operation that user P-1 inputs to PC200-1, which instructs the system to switch between the first and second call modes. If the switching instruction has been received (YES in step S09), the process proceeds to step S10; otherwise, the process returns to step S08.

[0099] In step S10, the second parallel call control process is executed, and the process proceeds to step S11. The details of the second parallel call control process will be described later, but it is a process that controls PC200-1 so that the first audio data of the first call is played back as audio, and the second audio data of the second call is played back in a different form than the playback of the first audio data.

[0100] In step S11, it is determined whether or not a switching instruction has been received. A switching instruction is an operation that user P-1 inputs to PC200-1, which instructs the PC200-1 to switch the playback mode between the first call and the second call. If the switching instruction has been received (YES in step S11), the process returns to step S08; otherwise (NO in step S11), the process returns to step S10.

[0101] When the second call ends, the process returns to step S02, and the first call becomes connected. When the first call ends, the second call becomes connected. In this case, the second call replaces the first call, and the process returns to step S02. When both the first and second calls have ended, the process is complete.

[0102] Figure 17 is a flowchart showing an example of the flow of the first parallel call control process. The first parallel call control process is executed in step S08 of the voice playback control process. Referring to Figure 17, the CPU 101 of the server 100 converts the first voice data into text (step S21) and proceeds to step S22. The text information converted from the first voice data may be translated into another language. The text information after converting the first voice data into text includes text information in the same language as the first voice data and text information after translating that text information into another language. Furthermore, if the first voice data includes the voices of multiple users, the user who spoke the voice may be identified from the voice data, and user identification information to identify the user who spoke the voice may be added to the text information corresponding to the voice.

[0103] In step S22, a first window is generated, and the process proceeds to step S23. The first window includes images related to the first call. The first window includes at least images of the text information generated in step S21. The images related to the first call also include, for example, video footage of other users P-2 to P-5 captured by camera 208, and images of data shared by the first group.

[0104] In step S23, a second window is generated, and the process proceeds to step S24. The second window contains images related to the second call. Images related to the second call include, for example, video footage of another user P-4 captured by camera 208, and images of data shared by the second group.

[0105] In step S24, the first audio data and the second audio data are played. The first audio data is played audibly, and the second audio data is played audibly. In this case, the volume of the audio when the first audio data is played is lower than the volume of the audio when the second audio data is played. Alternatively, the second audio data may be played audibly without the first audio data being played audibly.

[0106] In step S25, the audio from the first audio data and the audio from the second audio data are combined to generate combined data. If the first audio data is not played aloud, the combined data will only contain the audio from the second audio data. In the next step S26, the audio data and image data are sent to PC200-1, and processing returns to the audio playback control process. The image data includes data showing images of the screen, including the first window generated in step S22 and the second window generated in step S23. The audio data includes compressed data of the combined data generated in step S25.

[0107] Figure 18 is a flowchart showing an example of the flow of the second parallel call control process. The second parallel call control process is executed in step S10 of the voice playback control process. Referring to Figure 18, the CPU 101 of the server 100 converts the second voice data into text (step S31) and proceeds to step S32. The text information converted from the second voice data may be translated into another language. The text information after converting the second voice data into text includes text information in the same language as the second voice data and text information after translating that text information into another language. Furthermore, if the second voice data includes the voices of multiple users, the user who spoke the voice may be identified from the voice data, and user identification information to identify the user who spoke the voice may be added to the text information corresponding to the voice.

[0108] In step S32, a second window is generated, and the process proceeds to step S33. The second window includes images related to the second call. The second window includes at least an image of the text information generated in step S31. The images related to the second call also include, for example, video footage of another user P-4 captured by camera 208, and images of data shared by the second group.

[0109] In step S33, the first window is generated, and the process proceeds to step S34. The first window contains images related to the first call. Images related to the first call include, for example, video footage of other users P-2 to P-5 captured by camera 208, and images of data shared by the first group.

[0110] In step S34, the first audio data and the second audio data are played. The first audio data is played audibly, and the second audio data is played audibly. In this case, the volume of the audio when the second audio data is played is lower than the volume of the audio when the first audio data is played. Alternatively, the first audio data may be played audibly without the second audio data being played audibly.

[0111] In step S35, the audio from the playback of the first audio data and the audio from the playback of the second audio data are combined to generate combined data. If the second audio data is not played aloud, the combined data contains only the audio from the first audio data. In the next step S36, the audio data and image data are sent to PC200-1, and processing returns to the audio playback control process. The image data includes data showing the screen, including the second window generated in step S32 and the first window generated in step S33. The audio data includes compressed data of the combined data generated in step S35.

[0112] <Variation> In the remote conferencing system 1 described above, server 100 generates the images and audio output to PCs 200-1 to N. In the modified remote conferencing system, PCs 200-1 to N perform some of the functions of server 100.

[0113] Figure 19 shows an example of the functions of the CPU in the modified server. The functions shown in Figure 19 are functions realized by the CPU 101 of the server 100 when the CPU 101 executes the modified audio playback control program stored in the ROM 102, HDD 104, or CD-ROM 111. Referring to Figure 19, the differences from the functions shown in Figure 4 are that the parallel playback control unit 15, audio playback unit 17, and text playback unit 19 have been removed, and the parallel call control unit 13 has been changed to a parallel call control unit 13A. Therefore, the CPU 101 includes a call unit 11 and a parallel call control unit 13A.

[0114] The parallel call control unit 13A executes calls in the first group and calls in the second group in parallel. If there is a specific user belonging to both the first group and the second group, the parallel call control unit 13A executes calls in the first group and calls in the second group in parallel for that specific user. The parallel call control unit 13A outputs a command to the call unit 11 so that the voice data of the call in the first group and the voice data of the call in the second group are sent to the PC 200 operated by the specific user.

[0115] Figure 20 shows an example of the functions of the CPU in the modified PC. The functions shown in Figure 20 are realized by the CPU 201 of the PC 200 when it executes the terminal-side playback control program in the modified PC, which is stored in ROM 202, HDD 204, or CD-ROM 211A. The terminal-side playback control program is part of the audio playback control program. Referring to Figure 20, the differences from the functions shown in Figure 5 are the addition of the parallel playback control unit 15, audio playback unit 17, and text playback unit 19, and the change from the terminal-side call unit 251 to the terminal-side call unit 251A. The other functions are the same as those shown in Figure 5, so they will not be explained again here.

[0116] The terminal-side call unit 251A controls calls between the user operating the PC200 and other users. There are one or more other users. The terminal-side call unit 251A enables calls between the user operating the PC200 and one or more other users in a group consisting of the user operating the PC200 and one or more other users. The terminal-side call unit 251A treats calls in multiple groups as separate calls. The terminal-side call unit 251A sends and receives information about calls in the first group to and from the server 100, and also sends and receives information about calls in the second group to and from the server 100.

[0117] The terminal-side call unit 251A controls the communication unit 205 to receive voice data from the server 100. The terminal-side call unit 251 outputs the voice data received from the server 100 to the parallel playback control unit 15. If image data is received from the server 100, the terminal-side call unit 251A outputs the image data to the parallel playback control unit 15.

[0118] The parallel playback control unit 15 outputs the audio data of the calls in the first group and the audio data of the calls in the second group to the audio playback unit 17 and the text playback unit 19. The parallel playback control unit 15 controls the audio playback unit 17 and the text playback unit 19 to play back the audio data of the calls in the first group and the audio data of the calls in the second group in parallel.

[0119] The audio playback unit 17 plays audio data as sound. The audio playback unit 17 outputs the played audio to the audio output control unit 255.

[0120] The text playback unit 19 plays back the audio data by converting it into text. Specifically, the text playback unit 19 converts the audio data into text by speech recognition. The text playback unit 19 outputs text information consisting of the text converted from the audio data to the display control unit 257. The text playback unit 19 may also identify the user who is speaking from the audio data and output text information with user identification information added to identify the identified user to the parallel call control unit 13. Alternatively, the text playback unit 19 may translate the text information converted from the audio data into another language and output the translated text information to the parallel call control unit 13.

[0121] The parallel playback control unit 15 controls the text playback unit 19 to play back the first audio data of the call in the first group. In parallel with the playback of the first audio data by the text playback unit 19, the parallel playback control unit 15 controls the audio playback unit 17 to play back the second audio data of the call in the second group, and either does not play back the first audio data or plays back the first audio data at a lower volume than the second audio data.

[0122] The parallel playback control unit 15 instructs the terminal-side call unit 251A to send the audio data output from the audio input control unit 253 to the server 100 as audio data for the second group, and not as audio data for the first group. As a result, the user's voice is transmitted to users in the second group, but not to users in the first group.

[0123] The parallel playback control unit 15 executes the calls in the first group and the second group in parallel when a call in the second group is started while a call in the first group is continuing in the call unit 11. Alternatively, the parallel playback control unit 15 may execute the calls in the first group and the second group in parallel when it receives a start instruction from a specific user indicating the start of a parallel call after a call in the second group has started while a call in the first group is continuing in the call unit 11.

[0124] Furthermore, the parallel playback control unit 15 may switch between playing back the call in the first group and playing back the call in the second group if it receives a switching instruction from a specific user while simultaneously executing calls in the first group and the second group. Specifically, in response to receiving a switching instruction, the parallel playback control unit 15 controls the text playback unit 19 to play back the second audio data, controls the audio playback unit 17 to play back the first audio data, and either does not play back the second audio data or plays back the second audio data at a lower volume than the first audio data.

[0125] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0126] <Summary of Embodiments> (Item 1) Audio playback means for playing audio data as audio, A text playback method that converts audio data into text and plays it back, It includes a parallel playback control means for playing back first audio data and second audio data in parallel, The parallel playback control means is an audio playback control system that plays the first audio data using the text playback means and plays it using the audio playback means either OFF or at a lower volume than the second audio data, and plays the second audio data using the audio playback means.

[0127] In this scenario, the second audio data is played aloud, while the first audio data is converted to text and played back, either without audibility or at a lower volume than the second audio data. Therefore, the user can confirm the second audio data aloud while simultaneously confirming the first audio data in text. Consequently, since multiple audios are output in different forms, the user can distinguish and recognize them. As a result, a voice playback control system capable of playing multiple audios in parallel while maintaining identifiability can be provided.

[0128] (Item 2) The audio playback control system according to Item 1, wherein when playback of the second audio data is started while the first audio data is being played back by the audio playback means, the parallel playback control means plays back the first audio data by the text playback means and plays it back either with the audio playback means turned OFF or at a volume lower than that of the second audio data, and plays back the second audio data by the audio playback means.

[0129] In this scenario, if playback of the second audio data begins while the first audio data is playing, the second audio data will be played aloud, while the first audio data will be converted to text and played back, either without audibility or at a lower volume than the second audio data. Therefore, the second audio data, which is played later, will have priority over the first audio data. Consequently, the user does not need to specify which audio data to prioritize, making operation easier.

[0130] (Item 3) The audio playback control system according to Item 2, wherein the parallel playback control means plays the first audio data with the text playback means and plays it with the audio playback means turned OFF or at a lower volume than the second audio data when playback of the second audio data is started while the first audio data is being played by the audio playback means.

[0131] In this scenario, if playback of the second audio data begins while the first audio data is playing, the second audio data will be played aloud, while the first audio data will be converted to text and played back, either without audibility or at a lower volume than the second audio data. Therefore, since the start of the second audio data triggers the playback of both the first and second audio data, no switching operation is required. Consequently, operation is simple.

[0132] (Item 4) The audio playback control system according to Item 2, wherein when playback of the second audio data is started while the first audio data is being played by the audio playback means, the parallel playback control means plays the first audio data by the text playback means and plays it off or at a lower volume than the second audio data in response to a predetermined instruction from the user, and plays the second audio data by the audio playback means.

[0133] In this scenario, if playback of the second audio data begins while the first audio data is playing, the second audio data will be played aloud according to the user's predetermined instructions, while the first audio data will be converted to text and played back, either without audibility or at a lower volume than the second audio data. Therefore, the user can specify the timing at which multiple audio sources are output in different forms, thus improving convenience.

[0134] (Item 5) A means of making calls between multiple users, The aforementioned communication means further comprises parallel call control means for executing a first call and a second call in parallel, The first audio data is the audio data of the first call, The second audio data is the audio data of the second call, The voice playback control system according to item 1, wherein the parallel call control means plays back voice data using the parallel playback control means.

[0135] In this scenario, the second call is played aloud, while the first call is converted to text and played back, either without audibility or at a lower volume than the second call. Therefore, the user can confirm the second call aloud while simultaneously confirming the first call in text. As a result, since multiple audio sources are output in different formats, the user can distinguish and recognize them. Consequently, a voice playback control system capable of playing multiple calls in parallel while maintaining identifiability can be provided.

[0136] (Item 6) The audio playback control system according to item 5, further comprising display control means capable of displaying a first window containing a first image related to the first call and a second window containing a second image related to the second call.

[0137] In this scenario, a first window containing the first image related to the first call and a second window containing the second image related to the second call are displayed. Therefore, the first and second images can be displayed separately.

[0138] (Clause 7) The audio playback control system according to Clause 6, wherein the display control means displays the character information reproduced by the character playback means in the first window.

[0139] Following this procedure, the audio and video related to the first call will be displayed in the first window, allowing for a consolidated display of information related to the first call.

[0140] (Clause 8) The first call is a call between the operator and one or more first users, The voice playback control system according to item 5, wherein the second call is a call between the operator and one or more second users.

[0141] In this scenario, a user can simultaneously make calls between two users and between two users.

[0142] (Item 9) The audio playback control system according to Item 1, wherein the parallel playback control means switches from a state in which the first audio data is played by the text playback means and playback by the audio playback means is turned OFF or played at a volume lower than that of the second audio data, and the second audio data is played by the audio playback means, to a state in which the second audio data is played by the text playback means and playback by the audio playback means is turned OFF or played at a volume lower than that of the first audio data, and the first audio data is played by the audio playback means.

[0143] In this scenario, the playback methods for the first audio data and the second audio data are different, and the playback methods can be switched. Therefore, the operator can select which of the first and second audio data to prioritize.

[0144] (Item 10) The audio playback control system according to Item 1, further comprising a prohibition means for prohibiting playback by the parallel playback control means.

[0145] In this scenario, the second audio data is played aloud, and the first audio data is converted to text and played back, while not being played aloud or playing it at a lower volume than the second audio data is prohibited. Therefore, priority can be given to playing the first audio data.

[0146] (Clause 11) The voice playback control system according to Clause 8, wherein the second user includes at least one of the one or more first users.

[0147] (Clause 12) The voice playback control system according to Clause 8, wherein the second user includes at least one third user other than the one or more first users.

[0148] (Item 13) An audio playback step that plays audio data as audio, A text playback step that converts audio data into text and plays it back, The audio playback control device is instructed to perform a parallel playback control step, which involves playing back the first audio data and the second audio data in parallel. The parallel playback control step is a voice playback control method in which the first voice data is played in the text playback step and not played in the voice playback step or played at a lower volume than the second voice data, and the second voice data is played in the voice playback step.

[0149] Following this approach, it is possible to provide an audio playback control method that allows for the simultaneous playback of multiple audio sources in a distinguishable manner.

[0150] (Item 14) An audio playback step that plays audio data as audio, A text playback step that converts audio data into text and plays it back, The computer is instructed to perform a parallel playback control step, which involves playing the first audio data and the second audio data in parallel. The parallel playback control step is a voice playback control program that plays the first voice data in the text playback step and does not play it in the voice playback step, or plays it at a lower volume than the second voice data, and plays the second voice data in the voice playback step.

[0151] Following this approach, it is possible to provide an audio playback control program that can play multiple audio sources simultaneously in a distinguishable manner. [Explanation of Symbols]

[0152] 1 Remote conferencing system, 5 Internet, 100 Server, 200, 200-1~N PC, 11 Call unit, 13, 13A Parallel call control unit, 15 Parallel playback control unit, 17 Audio playback unit, 19 Text playback unit, 251, 251A Terminal-side call unit, 253 Audio input control unit, 255 Audio output control unit, 257 Display control unit, 259 Image input control unit, 300, 301 Operation screen, 301-1~4 Images, 303 Button, 305 Invitation instruction screen, 307 Invitation acceptance screen, 308 Invitation acceptance screen, 310 Parallel playback screen, 311, 312 Button, 313 First group display area, 314 Second group display area, 315 Shared screen display area, 320 Call list.

Claims

1. A means of playing audio data as audio, A text playback method that converts audio data into text and plays it back, It includes a parallel playback control means for playing back first audio data and second audio data in parallel, The parallel playback control means switches from a state in which the first audio data is played back by the text playback means and playback by the audio playback means is turned OFF or played back at a lower volume than the second audio data, and the second audio data is played back by the audio playback means, to a state in which the second audio data is played back by the text playback means and playback by the audio playback means is turned OFF or played back at a lower volume than the first audio data, and the first audio data is played back by the audio playback means.

2. The audio playback control system according to claim 1, wherein the parallel playback control means, when playback of the second audio data is started while the first audio data is being played by the audio playback means, plays the first audio data by the text playback means and turns off playback by the audio playback means or plays it at a volume lower than that of the second audio data, and plays the second audio data by the audio playback means.

3. The audio playback control system according to claim 2, wherein the parallel playback control means, in response to the start of playback of the second audio data while the first audio data is being played by the audio playback means, plays the first audio data by the text playback means and turns off playback by the audio playback means or plays it at a volume lower than the second audio data, and plays the second audio data by the audio playback means.

4. The audio playback control system according to claim 2, wherein the parallel playback control means, when playback of the second audio data is started while the first audio data is being played by the audio playback means, plays the first audio data by the text playback means and turns off playback by the audio playback means or plays it at a lower volume than the second audio data, in accordance with a predetermined instruction from the user, and plays the second audio data by the audio playback means.

5. A means of making calls between multiple users, The aforementioned communication means further comprises parallel communication control means for executing a first call and a second call in parallel, The first audio data is the audio data of the first call, The second audio data is the audio data of the second call, The voice playback control system according to claim 1, wherein the parallel call control means plays back voice data using the parallel playback control means.

6. The audio playback control system according to claim 5, further comprising display control means capable of displaying a first window containing a first image related to the first call and a second window containing a second image related to the second call.

7. The audio playback control system according to claim 6, wherein the display control means displays the character information reproduced by the character playback means in the first window.

8. The first call is a call between the operator and one or more first users. The voice playback control system according to claim 5, wherein the second call is a call between the operator and one or more second users.

9. The audio playback control system according to claim 1, further comprising a prohibition means for prohibiting playback by the parallel playback control means.

10. The audio playback step plays audio data using sound, A text playback step that converts audio data into text and plays it back, The audio playback control device is instructed to perform a parallel playback control step, which involves playing back the first audio data and the second audio data in parallel. The parallel playback control step is a voice playback control method that switches from a state in which the first voice data is played in the text playback step and not played in the voice playback step or played at a lower volume than the second voice data, and the second voice data is played in the voice playback step, to a state in which the second voice data is played in the text playback step and not played in the voice playback step or played at a lower volume than the first voice data, and the first voice data is played in the voice playback step.

11. The audio playback step plays audio data using sound, A text playback step that converts audio data into text and plays it back, The computer is instructed to perform a parallel playback control step, which involves playing the first audio data and the second audio data in parallel. The parallel playback control step is an audio playback control program that switches from a state in which the first audio data is played in the text playback step and not played in the audio playback step or played at a lower volume than the second audio data, and the second audio data is played in the audio playback step, to a state in which the second audio data is played in the text playback step and not played in the audio playback step or played at a lower volume than the first audio data, and the first audio data is played in the audio playback step.

Images