Information processing device and program

Abstract

To provide a mechanism for enabling to make a conversation individually with a specific user without interrupting participation in a conversation being performed by all of a plurality of users in a service for exchanging at least voices among the plurality of users to perform the conversation with all of the plurality of users.SOLUTION: In the case that a service in which at least voices are exchanged among a plurality of users to perform conversation of all of the plurality of users, is used, a processor outputs, to devices of users each performing individual conversation with a specific user among the plurality of users, discriminately voices of individual conversation and voices of conversation of all of the plurality of users, and outputs, to devices of users who do not perform the individual conversation, the voices of all of the plurality of users without outputting the voices of the individual conversation.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] 2. Description of the Related Art In a service in which at least voice is exchanged between a plurality of users, such as an online conference system, an individual conversation may be held between a specific user among the plurality of users.

[0003] Patent Document 1 describes a system in which a plurality of participants are divided into groups in advance, and calls can be made within the groups and with the entire conference.

[0004] Patent Document 2 describes a device that plays back the voice of a specific participant with emphasis. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 6-164741 [Patent Document 2] Japanese Patent Application Laid-Open No. 2015-046822 Summary of the Invention [Problem to be solved by the invention]

[0006] However, there may be cases where a user wishes to have an individual conversation with a specific user without interrupting participation in a conversation being conducted by a group of multiple users.

[0007] The object of the present invention is to provide a system in which, in a service in which a conversation is conducted among a plurality of users by exchanging at least voice among the plurality of users, a specific user can have an individual conversation with the plurality of users without interrupting their participation in the conversation being conducted among the plurality of users. [Means for solving the problem]

[0008] The invention according to claim 1 has a processor, and when a service is used in which a plurality of users exchange at least voices and a conversation is conducted among the plurality of users, the processor outputs, to a device of a user who is having an individual conversation with a specific user among the plurality of users, voices of the individual conversation and voices of the entire conversation of the plurality of users, while distinguishing between the voices of the individual conversation and voices of the entire conversation of the plurality of users, and outputs, to a device of a user who is not having the individual conversation, voices of the entire conversation of the plurality of users without outputting the voices of the individual conversation. along with , The voice of the user having the individual conversation is output only to a device of a user having the individual conversation with the user having the individual conversation, and when the user having the individual conversation responds to the overall conversation of the multiple users, the individual conversation is interrupted and the voice of the user having the individual conversation is output to the devices of all of the multiple users, an image for responding to the overall conversation of the multiple users is displayed on a display of the device of the user having the individual conversation, and when the image is operated by the user having the individual conversation, the individual conversation is interrupted and the voice of the user having the individual conversation is output to the devices of all of the multiple users. , an information processing device.

[0009] The invention of claim 2 is an information processing device as described in claim 1, wherein, when the speakers used by the users having the individual conversations in the service are stereo speakers, the processor outputs the audio of the individual conversation and the audio of the overall conversation of the multiple users from separate channels on the stereo speakers.

[0010] The invention of claim 3 is the information processing device described in claim 1, wherein, when the speaker used by the user having the individual conversation in the service is a mono speaker, the processor increases the volume of the audio of the individual conversation more than the volume of the audio of the overall conversation of the multiple users, and outputs the audio of the overall conversation of the multiple users and the audio of the individual conversation from the mono speaker.

[0011] The invention of claim 4 is an information processing device described in any one of claims 1 to 3, wherein the service further transmits and receives images of each of the multiple users, and the processor processes an image of the face of a user who has an individual conversation in the service, and displays the processed image on the display of a device of a user who does not have an individual conversation.

[0014] Claim 5The invention according to claim 1 further includes a method for displaying an unedited image of the user who is conducting the individual conversation on the displays of devices of all of the users when the user who is conducting the individual conversation responds to the entire conversation of the multiple users. 1 The information processing device is described in

[0015] The invention of claim 6 relates to a computer, when a service is used in which a plurality of users exchange at least voices and a conversation is conducted among the plurality of users, the computer outputs the voice of the individual conversation and the voice of the overall conversation of the plurality of users to a device of a user who is having an individual conversation with a specific user among the plurality of users, while outputting the voice of the individual conversation and the voice of the overall conversation of the plurality of users to a device of a user who is not having an individual conversation, without outputting the voice of the individual conversation. along with , The voice of the user having the individual conversation is output only to a device of a user having the individual conversation with the user having the individual conversation, and when the user having the individual conversation responds to the overall conversation of the multiple users, the individual conversation is interrupted and the voice of the user having the individual conversation is output to the devices of all of the multiple users, an image for responding to the overall conversation of the multiple users is displayed on a display of the device of the user having the individual conversation, and when the image is operated by the user having the individual conversation, the individual conversation is interrupted and the voice of the user having the individual conversation is output to the devices of all of the multiple users. This is a program to make it work like this. [Effects of the Invention]

[0016] Claim 1, 6 According to the invention, in a service in which a conversation is held among a plurality of users by exchanging at least voice between the plurality of users, it is possible to have an individual conversation with a specific user without interrupting participation in the conversation being held among the plurality of users. Furthermore, even while an individual conversation is taking place, the entire conversation can be handled.

[0017] According to the inventions set forth in claims 2 and 3, it is possible to distinguish between individual conversations and the entire conversation.

[0018] Claim 4, 5 According to the invention, individual conversations and general conversations can be conducted without displaying an image of the actual face of a user who is having an individual conversation to users who are not having an individual conversation. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing system. [Figure 2] FIG. 1 is a block diagram showing the hardware configuration of an online conference system. [Figure 3] FIG. 2 is a block diagram showing the hardware configuration of the terminal device. [Figure 4] FIG. 1 is a block diagram showing the functions of an online conference system. [Figure 5] 10 is a flowchart showing the flow of processing by an individual conversation unit. [Figure 6] 10 is a flowchart showing a flow of processing by a communication control unit. [Figure 7] 10 is a flowchart showing a processing flow by a voice control function of a communication control unit. [Figure 8] 10 is a flowchart showing a flow of processing by a microphone control function of a communication control unit. [Figure 9] 10 is a flowchart showing a flow of processing by an image processing function of the communication control unit. [Figure 10] FIG. 2 is a diagram schematically illustrating the corners of the mouth of an individual conversation partner. [Figure 11] 10 is a table showing a list of conference participants. [Figure 12] FIG. 10 is a diagram showing a screen displayed while using the online conference service. [Figure 13] FIG. 10 is a diagram showing a screen displayed while using the online conference service. [Figure 14] 10 is a table showing a list of conference participants. [Figure 15] FIG. 10 is a diagram showing a screen displayed while using the online conference service. [Figure 16] FIG. 10 is a diagram showing a screen displayed while using the online conference service. DETAILED DESCRIPTION OF THE INVENTION

[0021] An information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 shows an example of the configuration of an information processing system according to an embodiment.

[0022] As an example, an information processing system according to the embodiment includes an online conference system 10 and N terminal devices (N is an integer equal to or greater than 1). In the example shown in Fig. 1, the information processing system includes terminal devices 12A, 12B, ..., 12N. Hereinafter, when there is no need to distinguish between the terminal devices 12A, 12B, ..., 12N, they will be referred to as "terminal devices 12."

[0023] The online conference system 10 and the terminal device 12 have a function to communicate with other devices. The communication may be wired communication using a cable, or may be wireless communication. Examples of wireless communication include short-range wireless communication and Wi-Fi (registered trademark). Examples of short-range wireless communication include Bluetooth (registered trademark) and RFID (Radio Frequency Identifier). Each device may communicate with other devices via a communication path such as a LAN (Local Area Network) or the Internet.

[0024] The online conference system 10 provides a service for exchanging information among a plurality of users.

[0025] The information exchanged in the service may be, for example, images, videos, sounds, characters, symbols or codes other than characters, files, or a combination of at least two of these. Of course, information other than these may also be exchanged. Exchanging information means sending information and receiving information.

[0026] The service in question is, for example, a service that provides online conferences. In online conferences, information such as sound, images, and videos is exchanged between multiple users. Online conferences are also called web conferences, remote conferences, video conferences, etc. The service in question may be a service that provides a social networking service (SNS). Hereinafter, the service provided by the online conference system 10 will be referred to as an "online conference service." For ease of explanation, the name of the service includes the word "conference," but the service may, of course, be used for purposes other than conferences. In such cases, information such as sound, images, and videos is exchanged between multiple users.

[0027] The terminal device 12 is, for example, a personal computer (hereinafter referred to as a "PC"), a tablet PC, a smartphone, or a mobile phone.

[0028] A user accesses the online conference system 10 using a terminal device 12 and uses the online conference service provided by the online conference system 10. For example, multiple users use their own terminal devices 12 to use the online conference service, thereby exchanging information among the multiple users.

[0029] In the online conference service, for example, information is exchanged between a plurality of terminal devices 12. A user account for using the online conference service may be created for each user, and information may be exchanged between the plurality of user accounts.

[0030] For example, an address (e.g., a URL) for accessing an online conference service and using the online conference service is generated by the online conference system 10. Each user can obtain the address and use their own terminal device 12 to access the address, thereby using the online conference service corresponding to the address.

[0031] For example, when a user who will be the host of an online conference uses his / her terminal device 12 to request the use of an online conference service from the online conference system 10, an address for accessing the online conference service is generated by the online conference system 10. In response to the user's request, the address is transmitted from the online conference system 10 to the terminal device 12. The user who has acquired the address may transmit the address to other users who will participate in the same online conference service. This allows each user to acquire the address and access the same online conference service to participate.

[0032] A channel may be created for the online conference service, and information may be exchanged between multiple users for each channel. For example, an address for accessing the online conference service corresponding to the channel and using the online conference service may be generated by the online conference system 10 for each channel. By accessing the address using the terminal device 12, a user can use the online conference service of the channel corresponding to the accessed address.

[0033] A service ID and a corresponding password for using the online conference service may be generated by the online conference system 10, and the online conference service corresponding to the service ID and password may be provided to the user. The user obtains the service ID and the corresponding password, accesses the online conference system 10 using the terminal device 12, and transmits the service ID and the password to the online conference system 10. As a result, the online conference service corresponding to the service ID and the password is provided to the user.

[0034] For each channel, a service ID for the channel and a password corresponding thereto may be generated by the online conference system 10, and the online conference service of the channel corresponding to the service ID and password may be provided to the user.

[0035] In the online conference service according to the embodiment, at least audio is exchanged between multiple users participating in the same online conference service, and a conversation is conducted among the multiple users as a whole. Also, individual conversations are conducted among specific users among the multiple users. For example, a conversation is conducted among all users participating in the online conference service on the same channel. Also, individual conversations are conducted among specific users among the multiple users participating in the online conference service on the same channel.

[0036] In the following, a conversation conducted among multiple users participating in the same online conference service will be referred to as a "general conversation," and a conversation conducted individually between specific users will be referred to as an "individual conversation."

[0037] In a group conversation, information is exchanged among all users participating in the same online conference service. For example, a voice spoken by a user in an online conference service is output to all other users participating in the online conference service. The voice is emitted from the speakers used by each user.

[0038] In an individual conversation, information (for example, voice) exchanged between specific users is output only to the specific users, and is not output to users who are not engaged in the individual conversation.

[0039] In an online conference service, multiple users exchange at least audio, and a general conversation is conducted among all of the multiple users. Audio of the individual conversation and audio of the general conversation are output separately to the terminal device 12 of a user who is having an individual conversation. Audio of the individual conversation is not output to the terminal device of a user who is not having an individual conversation, but audio of the entire group of the multiple users (i.e., audio exchanged in the general conversation) is output.

[0040] In the general conversation and the individual conversation, in addition to voice, images, videos, character strings, etc. may be exchanged, and the images, videos, character strings, etc. may be displayed on the display of the terminal device 12 of each user.

[0041] The online conference system 10 corresponds to an example of an information processing device. A part of the processing executed by the online conference system 10 may be executed by the terminal device 12.

[0042] The hardware configuration of the online conference system 10 will be described below with reference to Fig. 2. Fig. 2 shows an example of the hardware configuration of the online conference system 10.

[0043] The online conference system 10 includes, for example, a communication device 14, a UI 16, a memory 18, and a processor 20.

[0044] The communication device 14 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information from other devices. The communication device 14 may have a wireless communication function or a wired communication function.

[0045] The UI 16 is a user interface and includes a display and an operation device. The display is a liquid crystal display, an EL display, or the like. The operation device is a keyboard, a mouse, input keys, an operation panel, or the like. The UI 16 may be a UI such as a touch panel that combines a display and an operation device. The UI 16 may also include a microphone and a speaker.

[0046] The memory 18 is a device that configures one or more storage areas for storing data. The memory 18 is, for example, a hard disk drive (HDD), a solid state drive (SSD), various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more memories 18 are included in the online conference system 10.

[0047] The processor 20 is configured to control the operation of each part of the online conference system 10. The processor 20 may include a memory.

[0048] The hardware configuration of the terminal device 12 will be described below with reference to Fig. 3. Fig. 3 shows an example of the hardware configuration of the terminal device 12.

[0049] The terminal device 12 includes, for example, a communication device 22, a UI 24, a memory 26, and a processor 28.

[0050] The communication device 22 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information transmitted from other devices. The communication device 22 may have a wireless communication function or a wired communication function.

[0051] The UI 24 is a user interface and includes a display, an operating device, a microphone, a speaker, and a camera. The display is a liquid crystal display, an EL display, or the like. The operating device is a keyboard, a mouse, input keys, an operating panel, or the like. The UI 24 may be a UI such as a touch panel that combines a display and an operating device.

[0052] The memory 26 is a device that configures one or more storage areas for storing data. The memory 26 is, for example, a hard disk drive (HDD), a solid state drive (SSD), various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more memories 26 are included in the terminal device 12.

[0053] The processor 28 is configured to control the operation of each part of the terminal device 12. The processor 28 may include a memory.

[0054] Hereinafter, functions of the online conference system 10 relating to general conversation and individual conversation will be described with reference to Fig. 4. Fig. 4 is a block diagram showing functions of the online conference system 10 relating to general conversation and individual conversation.

[0055] Hereinafter, a user who participates in the overall conversation but not in an individual conversation will be referred to as a "conference participant," and a specific user who participates in an individual conversation will be referred to as an "individual conversation participant." An individual conversation is conducted by multiple individual conversation participants who are also participating in the overall conversation. In other words, an individual conversation participant is also participating in the overall conversation.

[0056] 4, conference participant 1 uses terminal device 12A, conference participant 2 uses terminal device 12B, conference participant 3 uses terminal device 12C, individual conversation participant 1 uses terminal device 12D, and individual conversation participant 2 uses terminal device 12E. Each participant uses their own terminal device 12 to participate in the same online conference service (e.g., an online conference service on the same channel). Individual conversation participants 1 and 2 are participating in the same individual conversation.

[0057] The online conference system 10 includes a system basic unit 30, an individual conversation unit 32, and a communication control unit 34. These are realized by a processor 20 of the online conference system 10.

[0058] The system basic unit 30 has a function for realizing an overall conversation. Specifically, the system basic unit 30 transmits and receives information such as audio, images, and videos between the terminal devices 12 of all users participating in the same online conference service (e.g., terminal devices 12A, 12B, 12C, 12D, and 12E) via a communication path. That is, the system basic unit 30 receives information such as audio, images, and videos from the terminal devices 12 of all users participating in the same online conference service via a communication path, and transmits the information such as audio, images, and videos to the terminal devices 12 of all users participating in the same online conference service. For example, the system basic unit 30 receives information (e.g., audio) transmitted from the terminal device 12A and transmits the information to the terminal devices 12B, 12C, 12D, and 12E. In this way, in an overall conversation, information such as audio, images, and videos is shared by all users participating in the overall conversation.

[0059] The individual conversation unit 32 has a function of realizing individual conversations. Specifically, the individual conversation unit 32 transmits and receives information such as audio, images, and videos between the terminal devices 12 of multiple specific users who are conducting individual conversations via a communication path. In other words, the individual conversation unit 32 receives information such as audio, images, and videos from the terminal device 12 of a specific user who is conducting an individual conversation via a communication path, and transmits the received information to the terminal device 12 of another specific user who is conducting the individual conversation. The individual conversation unit 32 also manages requests and acceptances for individual conversations.

[0060] For example, the individual conversation unit 32 receives information (e.g., voice) transmitted from the terminal device 12D of the individual conversation party 1, and transmits the information to the terminal device 12E of the individual conversation party 2. Similarly, the individual conversation unit 32 receives information transmitted from the terminal device 12E of the individual conversation party 2, and transmits the information to the terminal device 12D of the individual conversation party 1.

[0061] The communication control unit 34 has an audio control function, a microphone control function, and an image processing function, and controls the output of audio, the output of the microphone, the collection of sound by the microphone, and the processing of images generated by taking pictures with the camera.

[0062] The audio control function controls the output of the audio for the general conversation and the audio for the individual conversation in accordance with the function of the speaker (for example, the speaker of the terminal device 12) used by the individual conversation participants who are having the individual conversation in the online conference service.

[0063] If the speakers used by an individual conversation participant in an online conference service (for example, the speakers of the individual conversation participant's terminal device 12) are stereo speakers, the audio control function outputs the audio of the individual conversation and the audio of the overall conversation from separate channels on the stereo speakers.

[0064] If the speaker used by an individual conversation participant in the online conference service (for example, the speaker of the individual conversation participant's terminal device 12) is a monaural speaker, the audio control function increases the volume of the audio for the individual conversation compared to the volume of the audio for the overall conversation, and outputs both the audio for the overall conversation and the audio for the individual conversation from the monaural speaker. If the audio for the individual conversation becomes silent, the audio control function outputs the audio for the overall conversation at a normal volume (i.e., without changing the volume of the audio for the overall conversation).

[0065] The microphone control function controls the output and sound collection of a microphone (for example, the microphone of the terminal device 12) used by an individual conversation participant who is having an individual conversation in the online conference service.

[0066] Specifically, the microphone control function sets the output destination of the sound picked up by the microphone used by an individual conversation participant (for example, the speaker of the terminal device 12 of the individual conversation participant) to only the terminal device 12 used by the other individual conversation participant who is also having the same individual conversation. In other words, during the individual conversation, the microphone control function mutes the output to the overall conversation in the microphone used by the individual conversation participant, and sets the output destination of the sound picked up by the microphone to only the individual conversation participant.

[0067] When an individual conversation participant requests a response from a conference participant participating in the general conversation, the microphone control function sets the output destination of the microphone used by that individual conversation participant to the general conversation while the "response-handling operation" continues during the individual conversation. When the "response-handling operation" ends, the microphone control function sets the output destination of the microphone used by that individual conversation participant to only the terminal device 12 used by the other individual conversation participants who are having the same individual conversation. The "response-handling operation" will be explained in detail later.

[0068] The image processing function is a function executed when an image of a user participating in the online conference service is transmitted or received in the online conference service. Specifically, when an image is taken by a camera used by an individual conversation participant in the online conference service (for example, a camera on the terminal device 12 of the individual conversation participant), the image processing function processes the image generated by the camera. For example, the image processing function processes the mouth of the individual conversation participant shown in the image to generate an image showing the mouth not opening or closing.

[0069] The communication control unit 34 transmits the processed image to the terminal device 12 of the conference participant who is participating in the general conversation without having an individual conversation, and displays it on the display of the terminal device 12 of the conference participant. The communication control unit 34 transmits the actual, unprocessed image to the terminal device 12 of the individual conversation participant who is having an individual conversation, and displays it on the display of the terminal device 12 of the individual conversation participant.

[0070] 4 is information exchanged in the overall conversation (e.g., audio, images, videos, etc.), and is information received by the terminal devices 12 of users participating in the overall conversation (e.g., conference participants, individual conversation participants). The received data and overall conversation data are shared by users participating in the overall conversation.

[0071] "Transmission data" is information (for example, audio, images, video, etc.) that is transmitted to the overall conversation by the terminal devices 12 of the conference participants who are participating in the overall conversation.

[0072] "Individual conversation side received data" is information exchanged in an individual conversation (for example, audio, images, videos, etc.), and is information received by the terminal device 12 of the individual conversation participants who are participating in the individual conversation. The individual conversation side received data is shared only with the individual conversation participants, and is not shared with users other than the individual conversation participants.

[0073] "Individual conversation side transmission data" is information (for example, audio, images, video, etc.) that is transmitted to the individual conversation by the terminal device 12 of the individual conversation party.

[0074] All or part of the voice control function, microphone control function, and image processing function may be realized by the processor 28 of each terminal device 12. Also, all or part of the functions of the individual conversation unit 32 may be realized by the processor 28 of each terminal device 12.

[0075] The processing by the individual conversation unit 32 will be described below with reference to Fig. 5. Fig. 5 shows a flowchart showing the flow of processing by the individual conversation unit 32.

[0076] The individual conversation unit 32 monitors the terminal device 12 of each user participating in the overall conversation, and when a request for an individual conversation is detected from the terminal device 12 (S01, Yes), it calls the recipient who is the other party in the individual conversation and asks the recipient whether or not to have an individual conversation to confirm whether or not the individual conversation is possible (S02). The recipient is a user who is participating in the overall conversation. For example, when a user uses his / her terminal device 12 to specify a partner for the individual conversation and request the online conference system 10 to conduct an individual conversation, the individual conversation unit 32, upon receiving the request, transmits information to the terminal device 12 of the recipient who is the other party in the individual conversation, inquiring whether or not to have an individual conversation. Hereinafter, the user who requested the individual conversation may be referred to as the "caller."

[0077] When the recipient receives the inquiry about whether to have an individual conversation, the recipient uses his / her own terminal device 12 to allow or deny the individual conversation (S03). For example, information indicating the inquiry is displayed on the display of the recipient's terminal device 12, and the recipient responds to the inquiry. Information indicating permission or denial is transmitted from the recipient's terminal device 12 to the online conference system 10.

[0078] If the recipient permits the individual conversation (S03, permitted), the individual conversation unit 32 calls the communication control unit 34 (S04).

[0079] If the recipient rejects the individual conversation (S03, rejected), the process ends. Also, if no request for individual conversation is detected (S01, absent), the process ends.

[0080] The processing by the communication control unit 34 will be described below with reference to Fig. 6. Fig. 6 shows a flowchart illustrating the flow of processing by the communication control unit 34.

[0081] When the individual conversation is permitted as described above, the individual conversation unit 32 forms an individual conversation channel between the terminal device 12 of the conference participant who requested the individual conversation (i.e., the sender) and the terminal device 12 of the conference participant who accepted the request and permitted the individual conversation (i.e., the receiver) (S11). As a result, the conference participant who requested the individual conversation and the conference participant who permitted the individual conversation conduct the individual conversation as individual conversation participants.

[0082] The voice control function of the communication control unit 34 checks the function of the speaker used by the individual conversation participants, and controls the output of the voice for the general conversation and the voice for the individual conversation according to the function (S12).

[0083] The microphone control function of the communication control unit 34 sets the output destination of the microphone used by the individual conversation participant to a channel for individual conversation, and sets the output destination of the microphone to the overall conversation while the response dialogue operation continues (S13).

[0084] The image processing function of the communication control unit 34 processes the image of the face of the individual conversation participant that is output to the overall conversation when the image is captured by the camera used by the individual conversation participant (S14).

[0085] If the individual conversation does not end (S15, No), the process returns to step S12. If the individual conversation ends (S15, Yes), the individual conversation unit 32 disconnects the channel for the individual conversation (S16). The communication control unit 34 returns the settings of the speaker and microphone used by the individual conversation participants to the settings before the individual conversation began.

[0086] The processes in steps S12 to S14 will be described in detail below with reference to FIGS.

[0087] The processing by the voice control function of the communication control unit 34 will be described with reference to Fig. 7. Fig. 7 shows a flowchart showing the flow of processing by the voice control function.

[0088] First, the voice control function of the communication control unit 34 checks the system of the speaker used by the individual conversation participant (for example, the speaker of the terminal device 12 used by the individual conversation participant) (S21).

[0089] If the speaker used by the individual conversation partner is a monaural speaker (S21, monaural), the audio control function controls the audio output in accordance with the monaural speaker system (S22).

[0090] When there is no interruption of an individual conversation (S23, none), the audio control function outputs the audio of the overall conversation (e.g., the overall conference conversation) at a normal volume (i.e., without changing the volume of the audio of the overall conversation) (S24). For example, when the audio of an individual conversation becomes silent, the audio control function outputs the audio of the overall conversation at a normal volume.

[0091] When the audio control function detects an interruption of an individual conversation (S23, Yes), it lowers the volume of the audio of the overall conversation (e.g., the overall conference conversation), increases the volume of the audio of the individual conversation higher than the volume of the audio of the overall conversation, and outputs the audio of the overall conversation and the audio of the individual conversation from the monaural speaker (S25). In this way, the audio control function prioritizes the output of the audio of the individual conversation over the output of the audio of the overall conversation. When the audio control function detects silence in the individual conversation, it returns the volume of the audio of the overall conversation to normal volume.

[0092] If the individual conversation has not ended (S26, No), the process returns to step S23, and the voice control function controls the voice output according to the monaural speaker method. If the individual conversation has ended (S26, Yes), the process of the voice control function ends.

[0093] If the speakers used by the individual conversation participants are stereo speakers (S21, stereo), the audio control function controls the audio output in accordance with the stereo speaker system (S27). That is, the audio control function outputs the audio of the individual conversation and the audio of the overall conversation from separate channels in the stereo speakers. For example, the audio control function outputs the audio of the individual conversation from one channel and the audio of the overall conversation from the other channel in the stereo speakers, thereby separating and outputting the audio of the individual conversation and the audio of the overall conversation.

[0094] If the individual conversation has not ended (S28, No), the voice control function controls the audio output in accordance with the stereo speaker method. If the individual conversation has ended (S28, Yes), the processing of the voice control function ends.

[0095] The voice control function checks the speaker system used by each of the sender and receiver, and controls the voice output of each speaker according to the monaural speaker system or stereo speaker system.

[0096] The processing performed by the microphone control function of the communication control unit 34 will be described with reference to Fig. 8. Fig. 8 shows a flowchart illustrating the flow of processing performed by the microphone control function.

[0097] The microphone control function sets the output destination of the sound picked up by a microphone used by an individual conversation participant (for example, the microphone of the terminal device 12 of the individual conversation participant) to only the terminal device 12 used by the other individual conversation participants who are having the same individual conversation. For example, the microphone control function sets the output destination of the sound picked up by the microphone used by the sender to only the terminal device 12 of the receiver, and sets the output destination of the sound picked up by the microphone used by the receiver to only the terminal device 12 of the sender. This allows the sound of the individual conversation to be exchanged only between the individual conversation participants.

[0098] If the "response handling operation" is not performed during the individual conversation (S31, No), the processing related to the "response handling operation" ends.

[0099] If the "operation during response" process is enabled during an individual conversation (S31, Yes), the microphone control function displays a warning on the display of the terminal device 12 used by each of the caller and the recipient (S32). For example, warning information indicating that the individual conversation has been interrupted is displayed on the display.

[0100] The microphone control function switches the output destination of the microphone used by the individual conversation participant performing the "response-in-progress operation" to the overall conversation (e.g., the overall conversation of the conference) while the "response-in-progress operation" processing is enabled (S33). For example, the "response-in-progress operation" functions only while a specific key or on-screen button is pressed, and does not function if the specific key or on-screen button is not pressed.

[0101] When the "response handling operation" is completed, the microphone control function sets the output destination of the microphone used by the individual conversation participant to only the terminal device 12 used by the other individual conversation participant who is having the same individual conversation.

[0102] The processing by the image processing function of the communication control unit 34 will be described with reference to Fig. 9. Fig. 9 shows a flowchart showing the flow of processing by the image processing function.

[0103] The image processing function checks whether the camera used by the individual conversation participant (for example, the in-camera installed in the terminal device 12 of the individual conversation participant) is enabled (S41). For example, the camera is enabled when the power of the camera is on and the setting for outputting images captured by the camera to the online conference service is on. The camera is disabled when the power of the camera is off or the output of images to the online conference service is off.

[0104] If the camera is not valid (S41, No), the processing by the image processing function ends.

[0105] If the camera is enabled (S41, Yes) and a "response-in-progress operation" is being performed (S42, Yes), the process returns to step S41. In this case, the image captured by the camera is sent to the terminal device 12 of the other conference participant without being processed by the image processing function, and is displayed on the display of the terminal device 12 of the other conference participant. For example, if the face of an individual conversation partner is captured by the camera, the image of the face is not processed, and the actual image is displayed on the display of the terminal device 12 of the other conference participant.

[0106] If the camera is enabled (S41, Yes) and the "response-in-progress operation" has not been performed (S42, No), the image processing function copies the video for the overall conversation (i.e., for the conversation of the entire conference) (S43). In other words, the image processing function copies the video generated by the camera used by each individual participant in real time for the overall conversation (S43).

[0107] Next, the image processing function identifies the corners of the mouth of the face from the copied video, and determines the identified corners of the mouth as the processing range (S44).

[0108] Next, the image processing function thins out images that represent open mouths from the duplicated video (S45).

[0109] Next, the image processing function combines the moving images (S46). Specifically, the image processing function interpolates images of the corners of the mouth based on the image immediately before the image representing the open mouth (i.e., the image representing the closed mouth) and the image immediately after the image representing the open mouth (i.e., the image representing the closed mouth), and combines the moving images. In this way, the image processing function generates a moving image representing the corners of the mouth when the mouth is closed.

[0110] Next, the communication control unit 34 outputs the processed video to the overall conversation (i.e., the overall conference conversation) as an image of the individual conversation participant (S47). That is, the communication control unit 34 transmits the processed video to the terminal device 12 of the conference participant who is participating in the overall conversation without having an individual conversation, and displays it on the display of the terminal device 12 of the conference participant. The communication control unit 34 transmits the actual, unprocessed video to the terminal device 12 of the individual conversation participant who is having an individual conversation, and displays it on the display of the terminal device 12 of the individual conversation participant. Thereafter, the process returns to step S41.

[0111] A specific example of processing using the image processing function will be described with reference to Fig. 10. Fig. 10 shows the corners of the mouth of the face of an individual conversation partner.

[0112] Video 36 is a video generated by shooting with a camera used by an individual conversation participant, and is a video before thinning. Video 36 is made up of a plurality of frames (i.e., images). In the example shown in FIG. 10, frames 36a to 36g included in video 36 are shown. Frames 36a to 36g were shot, and of frames 36a to 36g, frame 36a is the oldest frame and frame 36g is the newest frame.

[0113] In the video 36, frames 36b and 36e show an open mouth, and frames 36a, 36c, 36d, 36f and 36g show a closed mouth.

[0114] The moving image 38 is a moving image in which frames 36b and 36e showing an open mouth have been thinned out from the moving image 36.

[0115] The video 40 is the video after the interpolation. At the position of the thinned-out frame 36b, the frame 36a immediately preceding frame 36b is placed as frame 40b, and at the position of the thinned-out frame 36e, the frame 36d immediately preceding frame 36e is placed as frame 40e. Frames 36a and 36d represent a closed mouth. The frame where frame 36b was placed is replaced with frame 36a representing a closed mouth, thereby interpolating the frame where frame 36b was placed. Similarly, the frame where frame 36e was placed is replaced with frame 36d representing a closed mouth, thereby interpolating the frame where frame 36e was placed. Through this interpolation, the video 36 is processed to generate the video 40. Note that, by using known image processing, the corners of the mouth and whether the mouth is open or closed can be identified.

[0116] The communication control unit 34 transmits the processed video 40 as a video of the individual conversation participant to the terminal device 12 of the conference participant who is participating in the general conversation without having an individual conversation, and displays it on the display of the terminal device 12 of the conference participant. The communication control unit 34 transmits the actual, unprocessed video 36 to the terminal device 12 of the individual conversation participant who is having an individual conversation, and displays it on the display of the terminal device 12 of the individual conversation participant.

[0117] A specific example of the embodiment will be described below with reference to FIGS.

[0118] Figure 11 shows a list of conference participants. Here, as an example, five users (e.g., conference participants 1 to 3 and individual conversation participants 1 and 2) are participating in the same online conference service. Conference participants 1 to 3 are participating in the general conversation but are not conducting individual conversations. Individual conversation participants 1 and 2 are conducting individual conversations while participating in the general conversation. Individual conversation participant 1 is the sender who requests an individual conversation with individual conversation participant 2, and individual conversation participant 2 is the recipient who has received the request for an individual conversation from individual conversation participant 1.

[0119] For example, conference participant 1 participates in the online conference service using terminal device 12A, conference participant 2 participates in the online conference service using terminal device 12B, conference participant 3 participates in the online conference service using terminal device 12C, individual conversation participant 1 participates in the online conference service using terminal device 12D, and individual conversation participant 2 participates in the online conference service using terminal device 12E.

[0120] FIG. 11 shows an output before an individual conversation is taking place. Here, as an example, it is assumed that the cameras used by the conference participants 1 to 3 and the individual conversation participants 1 and 2 are enabled. The conference participants 1 to 3 and the individual conversation participants 1 and 2 are photographed by the cameras of their own terminal devices 12, and the actual images generated by the photographs are displayed for the overall conversation (i.e., the entire conference). Furthermore, the voices of the conference participants 1 to 3 and the individual conversation participants 1 and 2 (their own voices in FIG. 11) are output for the overall conversation (i.e., the entire conference), and the voices of the overall conversation (conference voices in FIG. 11) are output from the speakers of the conference participants 1 to 3 and the individual conversation participants 1 and 2. Furthermore, the displays of the terminal devices 12 used by the conference participants 1 to 3 and the individual conversation participants 1 and 2 display the items displayed for the overall conversation. In this way, when an individual conversation is not taking place, the images, voices, and screens generated by the photographs taken by the cameras are shared by the conference participants 1 to 3 and the individual conversation participants 1 and 2.

[0121] Hereinafter, with reference to Fig. 12, a description will be given of screens displayed while using the online conference service. Fig. 12 shows screen 42D. Screen 42D is a screen displayed on the display of terminal device 12D used by individual conversation party 1. A screen similar to screen 42D is also displayed on the displays of terminal devices 12 used by conference participants 1 to 3 and individual conversation party 2.

[0122] The screen 42D displays images representing users participating in the same online conference service. The image 44A is an image representing the conference participant 1 and is an image generated by capturing an image using the camera of the terminal device 12A. The image 44B is an image representing the conference participant 2 and is an image generated by capturing an image using the camera of the terminal device 12B. The image 44C is an image representing the conference participant 3 and is an image generated by capturing an image using the camera of the terminal device 12C. The image 44D is an image representing the individual conversation participant 1 and is an image generated by capturing an image using the camera of the terminal device 12D. The image 44E is an image representing the individual conversation participant 2 and is an image generated by capturing an image using the camera of the terminal device 12E. The images 44A to 44E may be moving images. Here, the images 44A to 44E are assumed to include moving images.

[0123] The images of the conference participants 1, 2, and 3 who are not having an individual conversation (i.e., images 44A, 44B, and 44C) are images generated by taking pictures with their respective cameras and are actual, unedited images. When an individual conversation is not taking place, the images of the individual conversation participants 1 and 2 (i.e., images 44D and 44E) are images generated by taking pictures with their respective cameras and are actual, unedited images. In this way, images representing each participant are shared by all of the conference participants 1 to 3 and the individual conversation participants 1 and 2.

[0124] The voice of each participant is output to the overall conversation, that is, to all other participants. For example, when no individual conversation is taking place, the voice of individual conversation participant 1 is output to conference participants 1-3 and individual conversation participant 2, and is emitted from the speakers of conference participants 1-3 and individual conversation participant 2, respectively. The same applies to the other participants. In this way, the voice of each participant is shared by all of conference participants 1-3 and individual conversation participants 1 and 2.

[0125] Furthermore, the items displayed on the screen (for example, documents, images, etc.) are displayed on the displays of the terminal devices 12 of the conference participants 1 to 3 and the individual conversation participants 1 and 2, respectively, and are shared by all of the conference participants 1 to 3 and the individual conversation participants 1 and 2.

[0126] When individual conversation party 1 designates individual conversation party 2 as the person with whom to have an individual conversation on screen 42D using his / her own terminal device 12D (for example, when clicking, touching, or tapping image 44E of individual conversation party 2), processor 20 of online conference system 10 displays menu 46 on screen 42D. Menu 46 displays buttons for requesting an individual conversation, a button for ending the individual conversation, a chat button, an email button, and the like.

[0127] When individual conversation party 1 selects and presses the "Request Individual Conversation" button from menu 46 (for example, by clicking, touching, or tapping the button), individual conversation unit 32 displays information indicating that individual conversation party 1 has requested individual conversation with individual conversation party 2 on the display of terminal device 12E used by individual conversation party 2. When individual conversation party 2 uses terminal device 12E to issue an instruction to permit the individual conversation with individual conversation party 1 in response to the request, individual conversation unit 32 receives permission for the individual conversation and forms a channel for the individual conversation between individual conversation party 1 and individual conversation party 2. Thereafter, the processes of steps S12 to S15 shown in FIG. 6 are executed, and the individual conversation is conducted between individual conversation party 1 and individual conversation party 2.

[0128] If the individual conversation party 2 rejects the individual conversation with the individual conversation party 1, the individual conversation unit 32 does not form a channel for the individual conversation between the individual conversation party 1 and the individual conversation party 2.

[0129] In addition, there are cases where a participant (e.g., individual conversation participant 2) receives requests for individual conversations from multiple different participants. In this case, the participant who receives the requests may select a participant to have an individual conversation with from the multiple different participants and may allow the individual conversation with the selected participant, or may reject the requests from all of the multiple different participants.

[0130] An individual conversation may also be held between three or more participants. For example, when an individual conversation is being held between individual conversation participants 1 and 2, individual conversation participant 1 or individual conversation participant 2 may request another participant to join the same individual conversation in which individual conversation participants 1 and 2 are participating. If the other participant allows participation, an individual conversation is held between individual conversation participants 1 and 2 and the other participant. In this case, if an individual conversation participant (e.g., individual conversation participant 2) other than the individual conversation participant (e.g., individual conversation participant 1) who requested the other participant to join allows the other participant to join, the other participant may be allowed to join.

[0131] Screen 42D also displays buttons 48 and 50. Button 48 is a button for exiting the online conference service. When button 48 is pressed, the participant who pressed button 48 is caused to exit the online conference service. Button 50 is a button for responding to a response. Responding to a response will be explained later.

[0132] FIG. 13 shows a screen 42D during an individual conversation.

[0133] When an individual conversation is taking place between the individual conversation parties 1 and 2, the individual conversation unit 32 displays information (e.g., a character string or an image) indicating that the individual conversation parties 1 and 2 are engaged in an individual conversation only on the display of the terminal device 12D used by the individual conversation party 1 and the display of the terminal device 12E used by the individual conversation party 2. For example, as indicated by the reference numeral 52, the screen 42D displays a line connecting the images 44D and 44E and a character string indicating that an individual conversation is taking place. Similar information is also displayed on the screen displayed for the individual conversation party 2. The displays of the terminal devices 12 of the conference participants 1 to 3 do not display information indicating that the individual conversation parties 1 and 2 are engaged in an individual conversation.

[0134] During an individual conversation, the audio output is controlled by the process shown in FIG. 7, the microphone output is controlled by the process shown in FIG. 8, and the image output is controlled by the process shown in FIG.

[0135] FIG. 14 shows an output when an individual conference is being held between individual conversation participants 1 and 2.

[0136] The processed images of the individual conversing parties 1 and 2 are transmitted to the terminal devices 12 of the conference participants 1 to 3 (i.e., terminal devices 12A, 12B, and 12C), and the processed images of the individual conversing parties 1 and 2 are displayed on the displays of the terminal devices 12A, 12B, and 12C. For example, images 44D and 44E displayed on the displays of the terminal devices 12A, 12B, and 12C are images processed by the image processing function of the communication control unit 34. As described with reference to FIGS. 9 and 10, image 44D displays the individual conversing party 1 with his / her mouth closed, and image 44E displays the individual conversing party 2 with his / her mouth closed. In other words, image 44D displayed on the displays of the terminal devices 12A, 12B, and 12C is not an actual image generated by capturing an image with the camera of the terminal device 12D used by the individual conversing party 1, but is an image generated by processing the actual image using the image processing function. The same applies to image 44E.

[0137] Actual images of the individual conversants 1 and 2 are transmitted to the terminal devices 12 of the individual conversants 1 and 2 (i.e., terminal devices 12D and 12E), and the actual images of the individual conversants 1 and 2 are displayed on the displays of the terminal devices 12D and 12E. For example, the images 44D and 44E displayed on the displays of the terminal devices 12D and 12E are actual images generated by photographing. In other words, the image 44D displayed on the displays of the terminal devices 12D and 12E is an actual image generated by photographing with the camera of the terminal device 12D. The same is true for the image 44E.

[0138] The displays of the terminal devices 12A, 12B, 12C, 12D, and 12E display actual images of the conference participants 1 to 3. That is, the images 44A, 44B, and 44C displayed on the displays of the terminal devices 12A, 12B, 12C, 12D, and 12E are actual images generated by capturing images with cameras.

[0139] The voices of the conference participants 1 to 3 (their own voices in FIG. 14) are output to the entire conversation (the entire conference). That is, the sounds picked up by the microphones of the conference participants 1 to 3 are output to all the participants, that is, the terminal devices 12A, 12B, 12C, 12D, and 12E, and are emitted from the speakers of the terminal devices 12A, 12B, 12C, 12D, and 12E.

[0140] The voice of individual interlocutor 1 (the person's voice in FIG. 14) is output to individual interlocutor 2. That is, the sound picked up by the microphone of individual interlocutor 1 is output only to individual interlocutor 2, that is, only to terminal device 12E, and is emitted from the speaker of terminal device 12E. Similarly, the voice of individual interlocutor 2 is output to individual interlocutor 1. That is, the sound picked up by the microphone of individual interlocutor 2 is output only to individual interlocutor 1, that is, only to terminal device 12D, and is emitted from the speaker of terminal device 12D. As a result, the voice of individual interlocutor 1 is output only to individual interlocutor 2, and the voice of individual interlocutor 2 is output only to individual interlocutor 1.

[0141] The overall conversation (that is, the conversation of the entire conference) is emitted from the speakers of the terminal devices 12 of the conference participants 1 to 3 (that is, the terminal devices 12A, 12B, and 12C).

[0142] Since the speakers of the terminal device 12D of the individual conversation party 1 are stereo speakers, the speakers of the terminal device 12D output audio in a stereo speaker format (see FIG. 7). For example, the audio of the overall conversation is output from the left speaker, and the audio of the individual conversation party 2, who is the other party in the individual conversation, is output from the right speaker.

[0143] Since the speaker of the terminal device 12E of the individual conversation party 2 is a monaural speaker, the speaker of the terminal device 12E outputs sound in a monaural manner (see FIG. 7). Specifically, when the sound of the individual conversation party 1 is output, the volume of the sound of the individual conversation party 1 is made larger than the volume of the overall conversation, and the sound of the individual conversation party 1 is emitted from the speaker.

[0144] The screen for the individual conversation may be displayed on the display of the terminal device 12 of the individual conversation party and shared only by the individual conversation party. For example, the screen for the individual conversation may be displayed only on the display of each of the terminal devices 12D and 12E, and the information displayed on the screen for the individual conversation may be shared only by the individual conversation parties 1 and 2.

[0145] Hereinafter, the operation during response dialogue will be described with reference to Fig. 15. Fig. 15 shows a screen 42D.

[0146] As described above, the communication control unit 34 outputs the voice of an individual conversation party (e.g., individual conversation party 1) only to the terminal device 12 (e.g., terminal device 12E) of another individual conversation party (e.g., individual conversation party 2) who is having an individual conversation with the individual conversation party. In this case, when the individual conversation party (e.g., individual conversation party 1) responds to the general conversation, the communication control unit 34 interrupts the individual conversation and outputs the voice of the individual conversation party to the general conversation. For example, the voice of individual conversation party 1 is output to the terminal devices 12 of conference participants 1 to 3 and individual conversation party 2, and is emitted from the speaker of each terminal device 12.

[0147] The communication control unit 34 displays an image on the display of the terminal device 12 of the individual conversation participant to interrupt the individual conversation and respond to the overall conversation. The communication control unit 34 may also display the image on the displays of the terminal devices 12 of all participants participating in the online conference service. When the image is operated by the individual conversation participant, the communication control unit 34 interrupts the individual conversation and outputs the voice of the individual conversation participant to the overall conversation.

[0148] The button 50 for responding is an example of an image for interrupting an individual conversation and responding to the general conversation. While the button 50 is pressed by a person in the individual conversation, the communication control unit 34 outputs the voice of that person to the general conversation, and when the button 50 is not pressed, the communication control unit 34 outputs the voice of that person only to the other person in the individual conversation.

[0149] Furthermore, when an individual conversation participant responds to the overall conversation, i.e., when button 50 is pressed, the communication control unit 34 displays an unprocessed image of the individual conversation participant (i.e., an actual image generated by photographing with the camera of the terminal device 12 used by the individual conversation participant) on the display of the terminal device 12 of the other participants as an image representing the individual conversation participant.

[0150] For example, suppose that while individual conversation participants 1 and 2 are having a private conversation, a conference participant (e.g., conference participant 1) calls out to individual conversation participant 1 (e.g., saying something along the lines of, "Please let us hear your opinion, individual conversation participant 1"). This call is made from conference participant 1 to individual conversation participant 1 through the entire conversation. The speakers of individual conversation participant 1 are stereo speakers, and the audio of the entire conversation is emitted from one channel, so individual conversation participant 1 can recognize that conference participant 1 is calling out to him / her.

[0151] When individual conversation party 1 responds to the call, he / she presses button 50 on screen 42D. While button 50 is pressed, communication control unit 34 switches the output destination of the microphone used by individual conversation party 1 to the general conversation. As a result, the individual conversation party 1's speech is output to the general conversation, i.e., to the terminal devices 12 of conference participants 1-3 and individual conversation party 2 (i.e., terminal devices 12A, 12B, 12C, and 12E), and is emitted from the speakers of terminal devices 12A, 12B, 12C, and 12E. In this case, as shown in FIG. 15 , communication control unit 34 displays information such as a message 54 indicating that the individual conversation is paused on the displays of terminal devices 12 of individual conversation parties 1 and 2 (i.e., terminal devices 12D and 12E). Message 54 is not displayed on the displays of terminal devices 12 of conference participants 1-3. Similarly, communication control unit 34 switches the output destination of the microphone used by individual conversation party 2, the other party in the individual conversation, to the general conversation. As a result, the utterance of individual conversation participant 2 is output to the overall conversation.

[0152] Furthermore, while the individual conversation party 1 is pressing the button 50, the communication control unit 34 causes an actual image 44D (i.e., image 44D not processed by the image processing function) generated by photographing with the camera of the terminal device 12D used by the individual conversation party 1 to be displayed on the display of the terminal device 12 of each participant (i.e., terminal devices 12A, 12B, 12C, 12D, 12E). Similarly, the communication control unit 34 causes an actual image 44E (i.e., image 44E not processed by the image processing function) generated by photographing with the camera of the terminal device 12E used by the individual conversation party 2, who is the other party in the individual conversation, to be displayed on the display of the terminal device 12 of each participant.

[0153] When individual interlocutor 1 releases button 50 (i.e., when button 50 is not pressed), communication control unit 34 switches the output destination of the microphone used by individual interlocutor 1 to the individual conversation with individual interlocutor 2. Similarly, communication control unit 34 switches the output destination of the microphone used by individual interlocutor 2 to the individual conversation. Furthermore, communication control unit 34 displays images 44D and 44E processed by the image processing function on the displays of terminal devices 12A, 12B, and 12C, respectively, and displays unprocessed images 44D and 44E on the displays of terminal devices 12D and 12E, respectively.

[0154] FIG. 16 shows the flow of operations when ending an individual conversation. For example, when individual conversation party 1 ends an individual conversation with individual conversation party 2, the individual conversation party 1 selects image 44E of individual conversation party 2, displays menu 46, and selects "End Individual Conversation" from menu 46. In response to this selection, communication control unit 34 ends the individual conversation between individual conversation parties 1 and 2 and returns the microphone output destination settings, speaker settings, and image display settings to the settings before the individual conversation began. For example, communication control unit 34 returns each setting during the individual conversation shown in FIG. 14 to the settings shown in FIG. 11. Note that individual conversation party 2 may also end the individual conversation.

[0155] During a private conversation, a conference participant who is not participating in the private conversation may request a private conversation from the individual conversation participant. In this case, the individual conversation participant who receives the request may end the private conversation and start a new private conversation with the conference participant who made the request.

[0156] The processor 20 of the online conference service may convert the content of the conversation into a character string and display it on the display of each participant's terminal device 12. The conversion may be performed by the processor 28 of each participant's terminal device 12.

[0157] The functions of the online conference system 10 and the terminal device 12 are realized, for example, by cooperation between hardware and software. For example, the processor of each device reads and executes a program stored in the memory of the device, thereby realizing the function of each device. The program is stored in the memory via a recording medium such as a CD or DVD, or via a communication path such as a network.

[0158] In the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operations of the processor in the above embodiments may not only be performed by a single processor, but may also be performed by multiple processors located in physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0159] 10 Online conference system, 12 Terminal device, 20, 28 Processor, 32 Individual conversation unit, 34 Communication control unit.

Claims

1. a processor; The processor: When a service is used in which at least voice is exchanged among a plurality of users and a conversation is conducted among the plurality of users, the voice of the individual conversation and the voice of the overall conversation of the plurality of users are output separately to a device of a user who is having an individual conversation with a specific user among the plurality of users, and the voice of the overall conversation of the plurality of users is output without outputting the voice of the individual conversation to a device of a user who is not having an individual conversation, and outputting the voice of the user having the individual conversation only to a device of a user having an individual conversation with the user having the individual conversation; When the user having the individual conversation responds to the overall conversation of the plurality of users, the individual conversation is interrupted, and the voice of the user having the individual conversation is output to devices of all of the plurality of users; displaying an image for responding to the overall conversation of the plurality of users on a display of a device of the user who is conducting the individual conversation; If the image is being operated by the user having the individual conversation, the individual conversation is interrupted, and the voice of the user having the individual conversation is output to devices of all of the plurality of users. Information processing device.

2. The processor: When the speakers used by the users who have the individual conversations in the service are stereo speakers, the stereo speakers output the audio of the individual conversation and the audio of the overall conversation of the plurality of users from separate channels. The information processing device according to claim 1 .

3. The processor: When the speaker used by the user having the individual conversation in the service is a monaural speaker, the volume of the audio of the individual conversation is made larger than the volume of the audio of the entire conversation of the multiple users, and the audio of the entire conversation of the multiple users and the audio of the individual conversation are output from the monaural speaker. The information processing device according to claim 1 .

4. In the service, images of each of the plurality of users are further transmitted and received, The processor: processing an image of the face of the user who is having the individual conversation in the service, and displaying the processed image on a display of a device of the user who is not having the individual conversation; The information processing device according to claim 1 .

5. The processor further comprises: When the user who is engaged in the individual conversation responds to the entire conversation of the plurality of users, an unedited image of the user who is engaged in the individual conversation is displayed on the displays of devices of all of the plurality of users. The information processing device according to claim 1 .

6. The computer When a service is used in which at least voice is exchanged among a plurality of users and a conversation is conducted among the plurality of users, the voice of the individual conversation and the voice of the overall conversation of the plurality of users are output separately to a device of a user who is having an individual conversation with a specific user among the plurality of users, and the voice of the overall conversation of the plurality of users is output without outputting the voice of the individual conversation to a device of a user who is not having an individual conversation, and outputting the voice of the user having the individual conversation only to a device of a user having an individual conversation with the user having the individual conversation; When the user having the individual conversation responds to the overall conversation of the plurality of users, the individual conversation is interrupted, and the voice of the user having the individual conversation is output to devices of all of the plurality of users; displaying an image for responding to the overall conversation of the plurality of users on a display of a device of the user who is conducting the individual conversation; If the image is being operated by the user having the individual conversation, the individual conversation is interrupted, and the voice of the user having the individual conversation is output to devices of all of the plurality of users. A program to make it work like this.

Images