Virtual space communication system, communication control device, and communication control method

Abstract

According to the present invention, a system enabling users using user terminals to communicate with each other by voice through a virtual space provided to a plurality of user terminals connected via a communication network is provided with a mode setting unit 11a for setting a secret conversation mode in which conversation is possible only between a first user and a second user when the first user speaks to the second user at a volume smaller than a predetermined value and the second user responds with a volume smaller than the predetermined value, among the users participating in the virtual space, and secret voice communication can be performed in a state close to secret conversation performed just between two particular persons in real space when desiring to perform secret conversation just by particular users among a plurality of the users participating in the virtual space.

Description

Virtual space communication system, communication control device, and communication control method

[0001] The present invention relates to a virtual space communication system, a communication control device, and a communication control method, and is particularly suitable for use in a system that enables users using user terminals to communicate with each other via voice through a virtual space provided to multiple user terminals connected via a communication network.

[0002] Conventionally, systems that enable users to communicate with each other by voice in a virtual space of a computer are known (see, for example, Patent Documents 1 to 4). Patent Documents 1 to 4 disclose that a user gives instructions to an avatar displayed in the virtual space and has a voice conversation while moving within the virtual space. Of these, Patent Documents 1 and 2 disclose giving voice instructions to the virtual space. Patent Document 3 discloses enabling a conversation when certain conditions are met in the virtual space. Patent Document 4 discloses adjusting the voice depending on the position of the avatar in the virtual space.

[0003] In the technology described in Patent Document 1, each user can interact in a virtual space through their own corresponding avatar. Using client software installed on a client terminal, users can instruct the avatar's behavior and speech using input devices such as a keyboard switch, pointing device, tablet, or microphone. When a host user starts a voice chat, a predetermined range centered on the host user is set as the audible range. The audible range moves in accordance with the movement of the host user's avatar. Users who enter the audible range are connected to the voice chat channel used by the host user and can engage in voice chat. When a user leaves the audible range, the user is disconnected from the voice chat channel.

[0004] In the technology described in Patent Document 2, each user uses a VR HMD system to communicate via voice through an avatar in a virtual space. When a user wearing an HMD moves their eyes or speaks, the display mode of the avatar object corresponding to the user changes in the virtual space presented by another HMD that is capable of communicating with the HMD, and voice is output from a speaker. Because the timing of the change in display mode and the timing of the voice output are synchronized, each communication partner can communicate seamlessly using voice and the avatar object in communication via the virtual space. The user can give voice instructions to the virtual space by speaking into a microphone provided in the HMD.

[0005] The space sharing support system described in Patent Document 3 includes a space display unit that displays a virtual space, an avatar display unit that displays a user's avatar in the virtual space, a movement instruction receiving unit that receives a movement instruction from the user to the avatar, and a conversation control unit that controls voice conversation between users through the avatar. The avatar display unit moves the avatar within the virtual space in accordance with the movement instruction. The conversation control unit determines that an interaction condition is met when a first avatar and a second avatar approach each other in the virtual space, and sets the first avatar and second avatar to be capable of conversation between the users corresponding to the first avatar and the second avatar.

[0006] In addition, the conversation control unit determines that the interaction condition is met when the first avatar enters the personal space of the second avatar. Note that, taking into consideration that another user's avatar may simply pass through one's personal space by chance, it is also disclosed that the conversation control unit may switch to the interaction mode when communication such as a conversation occurs at the user's command or when another user's avatar is present in one's personal space for a predetermined period of time or more.

[0007] Furthermore, the conversation control unit determines that the interaction conditions are met when the first avatar and the second avatar are simultaneously present in an area created in the virtual space (e.g., a conference room or a reception room) or when the first avatar and the second avatar are facing each other.

[0008] In a communication system described in Patent Literature 4, when multiple avatar images that can be moved by operations of users of other client devices are superimposed and displayed in a virtual space based on the user of one client device, the avatar images that are displayed farther away from the user of the first client device in the virtual space are controlled to be displayed in a smaller size. Also, when the display positions of the multiple avatar images in the virtual space are used as virtual sound sources for audio, the volume of the audio signal is controlled to be lowered as the virtual sound source is located farther away from the user of a first client device in the virtual space.

[0009] JP 2009-199198 A JP 2018-185786 A JP 2022-071314 A JP 2022-065363 A

[0010] In the systems described in Patent Documents 1 to 4, conversation conditions are set and the volume is adjusted based on the relative positions of the avatars of each user in the virtual space, enabling voice communication in a state close to that of the real space. However, in actual conversations in the real space, the conversation conditions, volume, etc. are not determined solely by the relative positions of the users.

[0011] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a virtual space in which users can communicate with each other by voice, allowing voice communication to be carried out in a state closer to that of a real space.

[0012] In order to solve the above-mentioned problems, the present invention provides a system that allows users using user terminals to communicate with each other via voice through a virtual space provided to multiple user terminals connected via a communication network, and when a first user participating in the virtual space speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value, a secret conversation mode is set in which only the first user and the second user can converse.

[0013] According to the present invention configured as described above, in a virtual space that allows users to communicate with each other by voice, if a user wishes to have a secret conversation only with a specific user, the user can carry out voice communication in a manner similar to a secret conversation that takes place only between two specific people in real space.

[0014] FIG. 1 is a diagram showing an example of the overall configuration of a virtual space communication system according to this embodiment; FIG. 2 is a block diagram showing an example of the functional configuration of a server device according to this embodiment; FIG. 3 is a flowchart showing an example of the operation of a mode setting unit according to this embodiment; FIG. 4 is a flowchart showing an example of the operation of a voice control unit according to this embodiment; FIG. 5 is a flowchart showing an example of the operation of an operation control unit according to this embodiment; and FIG. 6 is a diagram showing an example of a user interface displayed on a display of a user terminal.

[0015] An embodiment of the present invention will now be described with reference to the drawings. Fig. 1 is a diagram showing an example of the overall configuration of a virtual space communication system according to this embodiment. As shown in Fig. 1, the virtual space communication system of this embodiment is configured to include a server device 10 and a plurality of user terminals 20. The server device 10 and the user terminals 20 are connected via a communication network 30 such as the Internet and a mobile phone network.

[0016] The user terminal 20 is configured as, for example, a smartphone, a tablet, a personal computer, or the like. The user terminal 20 has a built-in audio input device such as a microphone, or is configured to be connectable via wired or wireless connection. The user terminal 20 also has a built-in audio output device such as a speaker, or is configured to be connectable via wired or wireless connection. A headset equipped with a microphone and speaker may be connected to the user terminal 20.

[0017] The server device 10 provides a virtual space to a plurality of user terminals 20 connected via a communication network 30, and enables users of the user terminals 20 to communicate with each other by voice through the virtual space. That is, the server device 10 receives a speaker's voice input from a voice input device of one user terminal 20 via the communication network 30, and transmits the received speaker's voice to another user terminal 20 via the communication network 30 and outputs it from a voice output device.

[0018] The virtual space is a space in which virtual users corresponding to the user can move freely. A user using a user terminal 20 moves his / her virtual user within the virtual space and communicates by voice with users corresponding to other virtual users he / she encounters at the destinations of the user's movement. A dedicated application program (hereinafter referred to as a conversation app) is installed in the user terminal 20 for processing the actions and voice communication of such virtual users within the virtual space. A "virtual user corresponding to a user" refers to a virtual user set in the conversation app installed on the user terminal 20 by the user.

[0019] The conversation app transmits virtual user action instructions input by the user to the server device 10. The server device 10 controls the actions of the virtual user in the virtual space in accordance with the action instructions received from the user terminal 20. The conversation app also transmits speaker voice input from a voice input device of the user terminal 20 to the server device 10. The server device 10 transmits speaker voice received from one user terminal 20 to another user terminal 20 and outputs the speaker voice from the voice output device.

[0020] In this embodiment, the virtual space provided by the server device 10 to the user terminal 20 is, for example, a non-display virtual space in which there is only audio and no images displayed on the screen of the user terminal 20. Although there are no images of the space or virtual users displayed on the user terminal 20, the virtual space itself exists, and the server device 10 manages the location information of the virtual space and the location information of the virtual users within the virtual space. For example, when a user issues an instruction to move the virtual user, the server device 10 updates the stored location information of the virtual user in accordance with the movement instruction.

[0021] The user understands the virtual user's position and surroundings in the virtual space by relying on the audio output from the audio output device. The user also moves the virtual user within the virtual space by giving audio instructions from the audio input device, and has audio conversations with users corresponding to other virtual users encountered at the destination location using the audio input and output devices.

[0022] The user terminal 20 is equipped with an image display device such as a display, or is configured to be connectable via wired or wireless connection. However, when communication is being carried out via voice through a virtual user in the virtual space, the virtual space is not displayed on the display.

[0023] 2 is a block diagram showing an example of the functional configuration of the server device 10 according to this embodiment. As shown in FIG. 2, the server device 10 of this embodiment includes, as its functional configuration, a communication control unit 11, an operation control unit 12, and a user information registration unit 13. The communication control unit 11 includes, as more specific functional configurations, a mode setting unit 11a and a voice control unit 11b. The server device 10 also includes, as storage media, a user information storage unit 14 and a spatial information storage unit 15.

[0024] The functional blocks 11 to 13 execute the processes described below through the cooperation of hardware and software. For example, the processes of the functional blocks 11 to 13 are executed by the operation of a program stored in a storage medium such as RAM, ROM, a hard disk, or a semiconductor memory under the control of a microcomputer including a CPU, RAM, ROM, etc. In addition to the microcomputer, a DSP (Digital Signal Processor) or the like may also be included.

[0025] The user information registration unit 13 registers user information about users who use the conversation app. That is, the user information registration unit 13 receives user information transmitted from the user terminal 20 and stores it in the user information storage unit 14. For example, when the user installs the conversation app in the user terminal 20, the user operates the user terminal 20 to input user information and transmits this to the server device 10 as a registration request. In response to this registration request, the server device 10 stores the user information received from the user terminal 20 in the user information storage unit 14.

[0026] The user information registered in the user information storage unit 14 includes, for example, the user's name, a nickname used as the name of the virtual user, a user ID, a login password, etc. The name and / or nickname is used as user identification information when users communicate with each other via voice. For this reason, it is preferable to register a nickname as a unique name for each individual user. For this reason, when a request to register a new nickname is received from the user terminal 20, the user information registration unit 13 determines whether the nickname is the same as a nickname already registered in the user information storage unit 14, and if so, responds to the user terminal 20 with a message prompting the user to set a different nickname.

[0027] The user information storage unit 14 also stores information about the virtual user corresponding to the user. For example, the user information storage unit 14 receives set mode information indicating a mode set for the virtual user by the mode setting unit 11a from the mode setting unit 11a and stores the information in association with the user ID. The user information storage unit 14 also stores location information and moving direction information of the virtual user moving in the virtual space under the control of the operation control unit 12 in association with the user ID.

[0028] The communication control unit 11 controls voice communication between users participating in the virtual space. A user participating in the virtual space is a user who has launched a conversation app and logged in to the server device 10. The communication control unit 11 controls the mode to be set for the virtual user by the mode setting unit 11a, and controls the dialogue voice communicated between the user terminals 20 by the voice control unit 11b according to the set mode. These processes will be described in detail later.

[0029] The action control unit 12 controls the actions of virtual users corresponding to users participating in the virtual space in response to action instructions from the users transmitted from the user terminals 20. As described above, the virtual users' action instructions are given by the users' voices. That is, the action control unit 12 controls the actions of virtual users in the virtual space in response to voice instructions input from a voice input device used by the users.

[0030] The actions of the virtual user include movement of the virtual user within the virtual space. Action instructions include, for example, instructions to move forward, backward, turn, etc. Action instructions also include instructions to enter and exit conversation spaces provided within the virtual space. One or more conversation spaces are provided within the virtual space. The space information storage unit 15 pre-stores position information indicating the overall position of the virtual space and position information indicating the location of each conversation space within the virtual space.

[0031] For example, when a user instructs the virtual user to move forward, the movement control unit 12 controls the virtual user to move forward a predetermined distance. The user may also instruct the distance to move forward. For example, when the user utters "move forward," the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The movement control unit 12 interprets the uttered voice regarding this movement instruction through voice recognition processing and controls the virtual user to move forward a predetermined distance. In other words, the virtual user's location information stored in the user information storage unit 14 is updated to information indicating a position where the virtual user has moved forward a predetermined distance in the direction indicated by the traveling direction information.

[0032] Furthermore, when a user instructs the virtual user to turn right, the movement control unit 12 controls the virtual user to turn a predetermined angle to the right. The user may also instruct the angle of the turn. For example, when the user utters "turn right," the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The movement control unit 12 interprets the uttered voice regarding this movement instruction through voice recognition processing and controls the virtual user to turn a predetermined angle to the right. In other words, the virtual user's traveling direction information stored in the user information storage unit 14 is updated to information indicating a direction displaced to the right by a predetermined angle.

[0033] Furthermore, when a user instructs the virtual user to enter the conversation space, the operation control unit 12 controls the virtual user to enter the conversation space. For example, when the user utters "enter," the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction through voice recognition processing and controls the virtual user to enter the conversation space. That is, the virtual user's location information stored in the user information storage unit 14 is updated to information indicating the virtual user's location within the conversation space. Note that such entry processing may be enabled only when the virtual user is located within a predetermined distance from the conversation space.

[0034] Furthermore, when a user instructs the virtual user to leave the conversation space, the operation control unit 12 controls the virtual user to leave the conversation space. For example, when the user utters "leave," the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction through voice recognition processing and controls the virtual user to leave the conversation space. That is, the virtual user's location information stored in the user information storage unit 14 is updated to information indicating a location outside the conversation space (e.g., the location where the virtual user was located before entering the conversation space). Note that such an exit process may be enabled only when the virtual user is within the conversation space.

[0035] When a user logs in to the server device 10 from the conversation app for the first time, the user's virtual user is set to be located at a predetermined start position in the virtual space. That is, the virtual user's location information stored in the user information storage unit 14 is initially set to the predetermined start position. On the other hand, when the user logs in for the second time or later, the user's virtual user is set to the location where the user was located at the time of previous logout. That is, information indicating the location where the virtual user was located at the time of logout is saved in the user information storage unit 14 and is used as the virtual user's location information the next time the user logs in.

[0036] The mode setting unit 11a of the communication control unit 11 sets the mode in accordance with the user's speech or the actions of the virtual user instructed by the user. The audio control unit 11b controls the audio transmitted to and received from the user terminal 20 in accordance with the mode set by the mode setting unit 11a. The audio control includes volume control and sound image localization processing.

[0037] The mode setting unit 11a sets three modes: roaming mode, normal conversation mode, and secret conversation mode. The roaming mode is a mode that is set when the virtual user is outside the conversation space. The normal conversation mode is a normal mode that is set when the virtual user is inside the conversation space. The secret conversation mode is a mode that is set when the virtual user is inside the conversation space and the user's speech voice reaches a predetermined state.

[0038] When a virtual user enters a conversation space provided within the virtual space, the mode setting unit 11a sets a normal conversation mode for the virtual user, which enables conversation between multiple users corresponding to the multiple virtual users in the conversation space. Whether the virtual user has entered the conversation space can be determined by comparing the virtual user's location information stored in the user information storage unit 14 with the location information of the conversation space stored in the space information storage unit 15. When the mode setting unit 11a detects that the virtual user has entered the conversation space, it stores set mode information indicating that the normal conversation mode has been set for the virtual user in the user information storage unit 14, in association with the user ID.

[0039] The audio control unit 11b controls the speech of a user corresponding to one virtual user who is in the conversation space and has normal conversation mode set (speech transmitted from the user terminal 20 of that user) to be output from an audio output device used by a user corresponding to one or more other virtual users who are in the same conversation space.

[0040] That is, the voice control unit 11b identifies multiple virtual users present in one conversation space by referring to the virtual user location information stored in the user information storage unit 14. Then, when the voice control unit 11b receives a user's utterance from a user terminal 20 corresponding to one virtual user present in the conversation space and set to normal conversation mode, the voice control unit 11b controls the user terminal 20 to transmit the utterance to the user terminals 20 corresponding to one or more other virtual users present in the same conversation space and output the voice from the voice output device.

[0041] Here, the one or more other virtual users to which the speaker voice is output may include both virtual users for which the normal conversation mode is set and virtual users for which the secret conversation mode is set. Alternatively, as will be described in detail later, the one or more other virtual users may be only virtual users for which the normal conversation mode is set, excluding virtual users for which the secret conversation mode is set. Furthermore, the user terminals 20 to which the spoken voice is output may be other than the user terminal 20 that transmitted the spoken voice, or may include the user terminal 20 that transmitted the spoken voice.

[0042] By the mode setting unit 11a and the voice control unit 11b performing the above-described processing, when there are multiple virtual users in the conversation space, the multiple users corresponding to the multiple virtual users can engage in voice communication in a state similar to a conversation in real space.

[0043] Furthermore, when a first user among multiple users corresponding to multiple virtual users in the conversation space speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value, the mode setting unit 11a sets the virtual users corresponding to the first and second users to a secret conversation mode in which only the first and second users can converse with each other. The mode setting unit 11a stores set mode information indicating that the secret conversation mode has been set for the two virtual users in the user information storage unit 14 in association with the user IDs.

[0044] For example, the mode setting unit 11a may set the secret conversation mode when a first user speaks the identification information (e.g., a nickname) of a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value. In this case, the content of the second user's response does not matter as long as the response is at a volume lower than the predetermined value. Alternatively, the secret conversation mode may be set only when the second user also speaks the identification information of the first user at a volume lower than the predetermined value.

[0045] That is, the mode setting unit 11a monitors the volume of the speech transmitted from the user terminal 20 of the user corresponding to the virtual user present in the conversation space, and when it detects speech at a volume lower than a predetermined value, it interprets the speech through speech recognition processing.Then, it determines whether the content of the interpreted speech matches any of the nicknames of the virtual users in the conversation space stored in the user information storage unit 14. Here, if it is determined that the content does not match any nickname, the mode setting unit 11a does not set the secret conversation mode.

[0046] On the other hand, if it is determined that the nickname matches any one of the nicknames, the mode setting unit 11a identifies the user of the user terminal 20 that is the sender of the spoken voice as the first user. Furthermore, the mode setting unit 11a monitors the volume of the spoken voice transmitted within a predetermined time from the user terminal 20 of the user corresponding to the virtual user with the matching nickname, and if the mode setting unit 11a detects a spoken voice with a volume lower than a predetermined value, the mode setting unit 11a identifies the user who responded by speaking as the second user and sets the secret conversation mode for the first user and the virtual users corresponding to the second user.

[0047] In addition, when the secret conversation mode is set only when the second user also utters the identification information of the first user, the mode setting unit 11a interprets, by voice recognition processing, a low-volume speech transmitted within a predetermined time from the user terminal 20 of a user whose nickname matches the nickname uttered by the first user, and determines whether the content of the interpreted speech matches the nickname of the first user stored in the user information storage unit 14. If it is determined that the content matches, the secret conversation mode is set.

[0048] When the secret conversation mode is set, the audio control unit 11b controls the first user's speech to be output only from the audio output device used by the second user, while the second user's speech to be output only from the audio output device used by the first user.

[0049] That is, when the voice control unit 11b receives a speech from the user terminal 20 of the first user in which the secret conversation mode is set, it controls so that the speech is transmitted only to the user terminal 20 of the second user in which the secret conversation mode is also set and output from the voice output device.Furthermore, when the voice control unit 11b receives a speech from the user terminal 20 of the second user, it controls so that the speech is transmitted only to the user terminal 20 of the first user and output from the voice output device.

[0050] The mode setting unit 11a monitors whether at least one of the first user and the second user speaks at a volume equal to or greater than a predetermined value while the secret conversation mode is set. If it detects that at least one user has spoken at a volume equal to or greater than the predetermined value, the mode setting unit 11a cancels the secret conversation mode. In other words, it returns the mode setting for the virtual users corresponding to the first user and the second user to the normal conversation mode.

[0051] By performing the above-described processing by the mode setting unit 11a and the voice control unit 11b, when multiple users are conversing in the conversation space, a secret conversation can be conducted in a low voice between only the first and second users among them, thereby enabling voice communication in a state similar to a secret conversation between only two specific people in real space.

[0052] Note that even when the first and second users are set to the secret conversation mode, other users in the same conversation space can have a normal (non-secret) conversation in the normal conversation mode. Here, the audio control unit 11b may prevent the speech of other users in the same conversation space from being transmitted to the user terminals 20 of the first and second users. In this way, the speech of the other user, which is output at a volume below a predetermined value to the audio output device of the first and second users who are having a secret conversation, can be prevented from being difficult to hear due to the speech of the other user, which is output at a volume above the predetermined value to the audio output device.

[0053] Similarly, when the spoken voice of a nickname (spoken voice at a volume lower than a predetermined value) transmitted from the user terminal 20 of the first user is transmitted to the user terminal 20 of the other user corresponding to the nickname (i.e., a candidate user who can become the second user), the spoken voices of other users in the same conversation space may not be transmitted to the user terminal 20 of the candidate user. In this way, the candidate user can avoid missing the spoken voice of the nickname, which is output to the candidate user's voice output device at a volume lower than a predetermined value, because it becomes difficult to hear due to the spoken voices of other users being output to the voice output device at a volume equal to or higher than the predetermined value.

[0054] Furthermore, when a virtual user is outside the conversation space, the mode setting unit 11a sets the roaming mode for that virtual user, and stores set mode information indicating that the roaming mode has been set in association with the user ID in the user information storage unit 14. In this case, the user for whom the roaming mode has been set may say anything, but the audio control unit 11b does not transmit the user's spoken voice to other users' user terminals 20. Meanwhile, the audio control unit 11b controls the voice of the conversation in the conversation space to be output from the audio output device used by the user corresponding to the virtual user for whom the roaming mode has been set, at a volume lower than the volume output when the virtual user is within the conversation space.

[0055] For example, the voice control unit 11b transmits a speech voice transmitted from the user terminal 20 of one user who is in the conversation space and has the normal conversation mode set to the user terminals 20 of other users who are in the same conversation space (either only users who are in the normal conversation mode, or users who are in the secret conversation mode in addition to the normal conversation mode) without lowering the volume.On the other hand, the voice control unit 11b transmits a speech voice transmitted from the user terminal 20 of one user who is in the conversation space and has the normal conversation mode set to the user terminals 20 of other users who are outside the conversation space and have the roaming mode set, with the volume lowered.

[0056] In this case, the audio control unit 11b may perform sound image localization of the audio output to the audio output device according to the positional relationship between the virtual user outside the conversation space and the conversation space. In other words, the audio control unit 11b controls the audio output to the audio output device according to the direction from the virtual user outside the conversation space toward the conversation space so that the audio is heard from a certain direction in the conversation space.

[0057] The audio control unit 11b also controls the volume of the audio output to the audio output device depending on the distance between the conversation space and a virtual user outside the conversation space, so that the longer the distance, the lower the volume, and the shorter the distance, the higher the volume. However, even if a virtual user is close to the conversation space and the highest volume is set, the volume is still lower than the volume output to the audio output device of a user who is inside the conversation space and has normal conversation mode set. Furthermore, if a virtual user is located more than a certain distance away from the conversation space, the volume output to the audio output device of the user corresponding to that virtual user is set to zero (the spoken audio is not sent to the user terminal 20 of that user).

[0058] By performing such sound image localization processing by the audio control unit 11b, even if the virtual space provided to the user terminal 20 is a non-display virtual space with only audio, the user can navigate within the virtual space while grasping the direction and distance of conversation spaces by relying on the sound image localized audio output from the audio output device, and can enter any conversation space and have a conversation. In this embodiment, all instructions to the virtual users and conversations between users can be given by audio, so even a visually impaired person can use a conversation app to enjoy audio communication in the virtual space.

[0059] Fig. 3 is a flowchart showing an example of the operation of the mode setting unit 11a according to this embodiment. Fig. 3 shows an example of the process of setting a mode for one virtual user, and the process of the flowchart shown in Fig. 3 is executed for all logged-in virtual users. Note that Fig. 3 shows an example of the operation executed at the time of first-time login.

[0060] The mode setting unit 11a sets the wandering mode as the initial mode (step A1). Then, the mode setting unit 11a determines whether the virtual user has entered the conversation space (step A2). For example, when the user utters "enter," the operation control unit 12 executes a process to place the virtual user in the conversation space, and the virtual user's location information is updated to information indicating that the virtual user is in the conversation space. When the mode setting unit 11a confirms this, it determines that the virtual user has entered the conversation space.

[0061] If the mode setting unit 11a determines that the virtual user has not entered the conversation space, the process returns to step A1. On the other hand, if the mode setting unit 11a determines that the virtual user has entered the conversation space, the mode setting unit 11a sets the virtual user to the normal conversation mode (step A3). Thereafter, the mode setting unit 11a monitors the volume of the speech transmitted from the user terminal 20 of the user corresponding to the virtual user for whom the normal conversation mode has been set, and determines whether the speech volume is lower than a predetermined value (step A4).

[0062] If the mode setting unit 11a detects a transmission of a speech sound with a volume lower than a predetermined value, the mode setting unit 11a interprets the speech sound by speech recognition processing and determines whether the content of the speech sound matches any of the nicknames of the virtual users in the conversation space stored in the user information storage unit 14 (step A5). If the mode setting unit 11a determines that the content of the speech sound does not match any of the nicknames, the process proceeds to step A12.

[0063] On the other hand, if it is determined that the nickname matches any one of the nicknames, the mode setting unit 11a identifies the user of the user terminal 20 that is the sender of the spoken voice as the first user. The mode setting unit 11a also monitors the volume of the spoken voice transmitted within a predetermined time from the user terminal 20 of the virtual user with the matching nickname, and determines whether there has been a response with a spoken voice at a volume lower than a predetermined value (step A6).

[0064] If the speech sound at a volume lower than the predetermined value is not received, the process proceeds to step A12. On the other hand, if the speech sound at a volume lower than the predetermined value is received, the mode setting unit 11a identifies the user of the user terminal 20 that is the sender of the speech sound as the second user, and sets the secret conversation mode for the virtual users corresponding to the first user and the second user (step A7).

[0065] Thereafter, the mode setting unit 11a monitors the volume of the speech transmitted from the user terminals 20 of the first and second users for which the secret conversation mode is set, and determines whether speech at a volume equal to or greater than a predetermined value has been received from at least one of the user terminals 20 (step A8). If speech at a volume equal to or greater than the predetermined value has not been received, the process returns to step A7, and the secret conversation mode continues. On the other hand, if the mode setting unit 11a determines that speech at a volume equal to or greater than the predetermined value has been received from at least one of the user terminals 20, the process returns to step A3, and the normal conversation mode is set.

[0066] In the above step A4, if no speech at a volume lower than the predetermined value is detected, the mode setting unit 11a monitors the volume of speech sent from the user terminal 20 of another user for which normal conversation mode is set to the user terminal 20 of the user corresponding to the virtual user for which normal conversation mode was set in step A3, and determines whether speech at a volume lower than the predetermined value has been received (step A9).

[0067] If the received speech has not been received at a volume lower than the predetermined value, the process proceeds to step A12. On the other hand, if the received speech has been received at a volume lower than the predetermined value, the mode setting unit 11a interprets the speech through a voice recognition process and determines whether the content of the speech matches the nickname of the user (the virtual user for whom the normal conversation mode was set in step A3) stored in the user information storage unit 14 (step A10). If the mode setting unit 11a determines that the content of the speech does not match the user's nickname, the process proceeds to step A12.

[0068] On the other hand, if it is determined that the nickname matches the user's own nickname, the mode setting unit 11a monitors the volume of the speech sent from the user's own user terminal 20 within a specified time period and determines whether the user responded with a speech at a volume lower than a specified value (step A11).

[0069] If the voice of speech at a volume lower than the predetermined value is not received, the process proceeds to step A12. On the other hand, if the voice of speech at a volume lower than the predetermined value is received, the process proceeds to step A7. In this case, the mode setting unit 11a identifies the user who responded as the second user and sets the secret conversation mode for the virtual users corresponding to the first user and the second user.

[0070] In step A12, the mode setting unit 11a determines whether the virtual user has left the conversation space. For example, when the user utters "exit," the operation control unit 12 executes a process to move the virtual user out of the conversation space, and the virtual user's location information is updated to information indicating that the virtual user is outside the conversation space. If the mode setting unit 11a confirms this, it determines that the virtual user has left the conversation space.

[0071] If the mode setting unit 11a determines that the virtual user has not left the conversation space, the process returns to step A3, where the normal conversation mode continues to be set. On the other hand, if the mode setting unit 11a determines that the virtual user has left the conversation space, the process returns to step A1, where the wandering mode is set.

[0072] As shown in Figure 3, the wandering mode is initially set when logging in for the first time, but when logging in for the next time or thereafter, processing begins by setting the mode corresponding to the location where the virtual user was located at the time of the previous logout (step A3 or step A7).

[0073] Fig. 4 is a flowchart showing an example of the operation of the voice control unit 11b according to this embodiment. Fig. 4 shows an example of the operation when processing a speaker's voice transmitted from a user terminal 20. The process of the flowchart shown in Fig. 4 is executed every time a speaker's voice transmitted from a logged-in user terminal 20 is received.

[0074] The voice control unit 11b determines whether a speaker's voice has been received from the user terminal 20 (step B1). The voice control unit 11b continues the determination in step B1 until a speaker's voice is received. When the server device 10 receives a speaker's voice, the voice control unit 11b determines the mode currently set for the virtual user corresponding to the user of the user terminal 20 who transmitted the speaker's voice (step B2).

[0075] If it is determined that the mode set for the virtual user is the wandering mode, the process returns to step B1. That is, the voice control unit 11b does not transmit the voice of the current speaker to other user terminals 20, and waits to receive the voice of the next speaker.

[0076] If it is determined that the mode set for the virtual user is the normal conversation mode, the voice control unit 11b refers to the virtual user's location information stored in the user information storage unit 14, identifies one or more other virtual users who are in the same conversation space as the speaking virtual user, and controls the voice to be sent to one or more user terminals 20 corresponding to the one or more other virtual users and output from the voice output device (step B3).

[0077] The voice control unit 11b also references the virtual user location information stored in the user information storage unit 14 to identify one or more virtual users who are outside the conversation space but are located within a certain distance from the conversation space where the speaking virtual user is located. The voice control unit 11b then controls one or more user terminals 20 corresponding to the identified one or more virtual users to transmit speech voices that have been subjected to sound image localization according to the positional relationship between each virtual user and the conversation space, and output the speech voices from the voice output devices (step B4). Then, the process returns to step B1.

[0078] If it is determined that the mode set for the virtual user is the secret conversation mode, the voice control unit 11b refers to the setting information for the secret conversation mode stored in the user information storage unit 14, identifies the other virtual user for whom the secret conversation mode is set, and controls so that the spoken voice is transmitted only to the user terminal 20 corresponding to the identified other virtual user and output from the voice output device (step B5).The process then returns to step B1.

[0079] 5 is a flowchart showing an example of the operation of the operation control unit 12 according to this embodiment. This Fig. 5 shows an example of a process for controlling the operation of one virtual user, and the process of the flowchart shown in Fig. 5 is executed for all logged-in virtual users.

[0080] The action control unit 12 determines whether or not an action instruction has been received from the user terminal 20 (step C1). The action control unit 12 continues the determination in step C1 until an action instruction is received. When the server device 10 receives the action instruction, the action control unit 12 controls the action of the virtual user in the virtual space in accordance with the action instruction (step C2). Then, the process returns to step C1.

[0081] As described above in detail, in the virtual space communication system of this embodiment, the server device 10 provides a virtual space to a plurality of user terminals 20 connected via a communication network 30, and enables users using the user terminals 20 to communicate with each other via voice through the virtual space. In particular, in this embodiment, when a first user speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value, a secret conversation mode is set, in which only the first and second users can converse. When the secret conversation mode is set, the speech of the first user is output only from the audio output device used by the second user, and the speech of the second user is output only from the audio output device used by the first user.

[0082] According to the virtual space communication system of this embodiment configured in this manner, in a virtual space that allows users to communicate with each other via voice, if a user wants to have a secret conversation only with a specific user, the user can carry out voice communication in a manner similar to a secret conversation that takes place only between two specific people in real space.

[0083] In the above embodiment, the virtual space provided to the user terminal 20 is a non-visual space that uses only audio, and instructions to the non-visual user for actions are given only by audio. However, the present invention is not limited to this. For example, the user interface for giving instructions to actions may be displayed on a screen.

[0084] 6 is a diagram showing an example of a user interface displayed on the display of the user terminal 20. As shown in FIG. 6, the virtual space and virtual users are not displayed, and only operation buttons for instructing the virtual users to move forward, backward, turn, enter, and exit are displayed. In this case, the user can instruct the virtual users to move by voice or using the user interface displayed on the screen.

[0085] As another example, if the user terminal 20 is a smartphone, tablet, or the like with a built-in acceleration sensor or the like, the virtual user may be instructed to move in accordance with the movement of the user terminal 20 detected by the acceleration sensor. For example, when the user terminal 20 is swung forward, backward, left, or right, the virtual user may be instructed to move forward, backward, turn left, or turn right, respectively, when the user terminal 20 is rotated right, the virtual user may be instructed to enter the room, and when the user terminal 20 is rotated left, the virtual user may be instructed to leave the room.

[0086] Furthermore, instead of a non-displayed virtual space, a display virtual space may be used in which a virtual space and a virtual user are displayed on a screen. Even in this case, if instructions to the virtual user are given using voice or an acceleration sensor, the user can communicate by voice without looking at the displayed virtual space. Needless to say, if the user wishes, communication by voice can be carried out while looking at the displayed virtual space.

[0087] In the above embodiment, a smartphone, a tablet, and a personal computer are given as examples of the user terminal 20, but the present invention is not limited to these. For example, the user terminal 20 may be a wearable terminal such as a goggle-type terminal, an eyeglass-type terminal, or a wristwatch-type terminal. In addition, when a virtual space is used as the display virtual space, a head-mounted display capable of displaying virtual reality (VR) images may be used as the user terminal 20.

[0088] Furthermore, in the above embodiment, when a wandering mode is set for a virtual user, an example was described in which the audio of a conversation taking place in a conversation space within a certain distance from the virtual user's position is localized as a sound image and supplied to the user terminal 20 of the user corresponding to that virtual user, but instead of or in addition to this, an audio other than speech audio may be supplied and output from the audio output device.

[0089] For example, a specific sound to notify the user of the existence of a conversation space may be localized and supplied to the user terminal 20 in the navigation mode. In this case, a special sound may be supplied to a conversation space where active conversation is taking place. The definition of active conversation may be set arbitrarily. For example, it is possible to determine that active conversation is taking place when either or both of the number of virtual users present in the conversation space is equal to or greater than a threshold, or the number of utterances per predetermined time is equal to or greater than a threshold, are satisfied.

[0090] Furthermore, in the above embodiment, an example was described in which there are three modes: roaming mode, normal conversation mode, and secret conversation mode. However, the present invention is not limited to this. For example, there may be only two modes: normal conversation mode and secret conversation mode. For example, the virtual space itself may be a single conversation space, and the logged-in virtual user may always exist within one conversation space (= virtual space), where the normal conversation mode and secret conversation mode may be switched as appropriate. Alternatively, there may be multiple conversation spaces within the virtual space, and an instruction to jump from one conversation space to another conversation space may be given to the virtual user.

[0091] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereby. In other words, the present invention can be carried out in various forms without departing from the gist or main characteristics thereof.

[0092] REFERENCE SIGNS LIST 10 Server device 11 Communication control unit 11a Mode setting unit 11b Audio control unit 12 Operation control unit 13 User information registration unit 14 User information storage unit 15 Spatial information storage unit 20 User terminal 30 Communication network

Claims

1. A virtual space communication system that enables users using multiple user terminals connected via a communication network to communicate via voice through a virtual space provided to the user terminals, the virtual space communication system comprising a communication control unit that controls voice communication between users participating in the virtual space, wherein the communication control unit sets a secret conversation mode in which only the first user and the second user can converse when a first user speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value.

2. The virtual space communication system described in claim 1, characterized in that the communication control unit sets the secret conversation mode when the first user speaks the identification information of the second user at a volume lower than the predetermined value and the second user responds at a volume lower than the predetermined value.

3. The virtual space communication system described in claim 1, characterized in that the communication control unit cancels the setting of the secret conversation mode when at least one of the first user and the second user speaks at a volume above the predetermined value while the secret conversation mode is set.

4. A virtual space communication system as described in any one of claims 1 to 3, further comprising an operation control unit that controls the operation of virtual users corresponding to users participating in the virtual space within the virtual space in accordance with instructions from the users, wherein the communication control unit sets, when the virtual user enters a conversation space provided within the virtual space, a normal conversation mode for the virtual user that enables multiple users corresponding to the multiple virtual users in the conversation space to converse with each other, and sets the secret conversation mode for virtual users corresponding to the first user and the second user among the multiple users corresponding to the multiple virtual users in the conversation space.

5. The virtual space communication system described in claim 4, characterized in that the communication control unit controls the speech of a user corresponding to one virtual user in the conversation space and for whom the normal conversation mode is set to be output from an audio output device used by users corresponding to one or more other virtual users in the same conversation space, controls the speech of the first user corresponding to a virtual user in the conversation space and for whom the secret conversation mode is set to be output only from an audio output device used by the second user, while controls the speech of the second user to be output only from an audio output device used by the first user.

6. The virtual space communication system described in claim 4, characterized in that the communication control unit controls the audio being conversed in the conversation space when the virtual user is outside the conversation space to be output from an audio output device used by a user corresponding to the virtual user outside the conversation space at a volume lower than the volume output when the virtual user is inside the conversation space.

7. The virtual space communication system described in claim 6, characterized in that the communication control unit performs sound image localization of the audio output to the audio output device depending on the positional relationship between the virtual user outside the conversation space and the conversation space.

8. The virtual space communication system according to claim 4, wherein the virtual space is a non-display virtual space in which there are no images displayed on the screen of the user terminal and only audio exists.

9. The virtual space communication system described in claim 8, characterized in that the action control unit controls the action of a virtual user corresponding to a user participating in the virtual space in accordance with voice instructions input from a voice input device used by the user.

10. A communication control device that provides a virtual space to multiple user terminals connected via a communication network and controls voice communication between users using the user terminals through the virtual space, comprising a communication control unit that controls voice communication between users participating in the virtual space, wherein the communication control unit sets a secret conversation mode in which only the first user and the second user can converse when a first user speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value.

11. A communication control method for controlling voice communication between users using multiple user terminals connected via a communication network through a virtual space provided to the user terminals, characterized in that a communication control unit of a computer sets a secret conversation mode in which only the first user and the second user can converse with each other when a first user of the multiple users participating in the virtual space speaks to a second user at a volume lower than a predetermined value and the second user responds at a volume lower than the predetermined value.

Images