Virtual Space Communication System, Communication Control Device, and Communication Control Method

Abstract

In a system that enables users using user terminals to communicate by voice through a virtual space provided to a plurality of user terminals connected via a communication network, when a first user among the users participating in the virtual space speaks to a second user in a volume smaller than a predetermined value and the second user responds in a volume smaller than the predetermined value, a mode setting unit 11a is provided that sets a secret conversation mode in which conversation is possible only between the first user and the second user. When a specific user among the plurality of users participating in the virtual space wants to have a secret conversation only with a specific other user, it is possible to perform secret voice communication in a state close to a secret conversation that is carried out only between two specific people in the real space.

Description

Technical Field

[0001] The present invention relates to a virtual space communication system, a communication control device, and a communication control method, and more particularly, to a system suitable for use in a system that enables users using user terminals to communicate with each other by voice through a virtual space provided to a plurality of user terminals connected via a communication network.

Background Art

[0002] Conventionally, a system that enables users to communicate with each other by voice in a virtual space of a computer has been known (see, for example, Patent Documents 1 to 4). Patent Documents 1 to 4 disclose that a user gives an instruction to an avatar displayed in a virtual space and conducts a voice conversation while moving within the virtual space. Among these, Patent Documents 1 and 2 disclose giving an instruction to the virtual space by voice. Patent Document 3 discloses enabling a conversation when a certain condition is satisfied in the virtual space. Patent Document 4 discloses adjusting the voice according to the position of the avatar in the virtual space.

[0003] In the technology described in Patent Document 1, each user can communicate within the virtual space via an avatar corresponding to himself / herself. The user uses an input device such as a keyboard switch, a pointing device, a tablet, a microphone, etc. via client software installed on the client terminal to instruct the speech and actions of the avatar. When the host user starts a voice chat, a predetermined range centered on the host user is set as the audible range. The audible range moves according to the movement of the avatar of the host user. A user who enters the audible range is connected to the voice chat channel used by the host user and can conduct a voice chat. When the user moves out of the audible range, the user is disconnected from the voice chat channel.

[0004] In the technology described in Patent Document 2, each user uses a VR HMD system to communicate by voice through an avatar in a virtual space. When a user wearing an HMD moves their eyes and speaks, in the virtual space presented by another HMD that can communicate with the HMD, the display mode of the avatar object corresponding to the user changes, and voice is output from the speaker. Since the timing of the change in the display mode and the timing of the voice output are synchronized, in communication via the virtual space, each communication partner can communicate using voice and the avatar object without a sense of discomfort. A user can give a voice instruction to the virtual space by speaking into the microphone provided in the HMD.

[0005] The space sharing support system described in Patent Document 3 includes a space display unit that displays a virtual space, an avatar display unit that displays a user's avatar in the virtual space, a movement instruction receiving unit that receives a movement instruction from the user to the avatar, and a conversation control unit that controls a voice conversation between users through the avatar. The avatar display unit moves the avatar within the virtual space according to the movement instruction. The conversation control unit determines that the communication condition is satisfied when the first avatar and the second avatar approach each other in the virtual space, and sets the conversation between the users corresponding to the first avatar and the second avatar to be possible.

[0006] In addition, the conversation control unit also determines that the communication condition is satisfied when the first avatar enters the personal space of the second avatar. Considering that there may be a case where the avatar of another user only accidentally passes through one's own personal space, it is disclosed that the communication mode may be switched to the communication mode at the timing when communication such as conversation occurs according to the instructions of the users or when the avatar of another user is located in one's own personal space for a predetermined time or more.

[0007] Furthermore, the conversation control unit determines that the communication condition is satisfied when the first avatar and the second avatar are simultaneously present in an area created in the virtual space (for example, a conference room or a reception room), or when the first avatar and the second avatar face each other.

[0008] In the communication system described in Patent Document 4, when a plurality of avatar images that can be moved by the operations of users of other client devices are superimposed and displayed in a virtual space based on the user of the own client device, the avatar images displayed farther away from the user of the own client device in the virtual space are controlled to be displayed in a smaller size. Further, when the display positions of a plurality of avatar images in the virtual space are used as virtual sound sources of sound, the volume of the sound signal is controlled to be lowered for the virtual sound sources located farther away from the user of the first client device in the virtual space.

Prior Art Documents

Patent Documents

[0009]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0010] In the systems described in Patent Documents 1 to 4, based on the positional relationship of the avatars of each user in the virtual space, the conversation is set to a possible state or the volume is adjusted, so that voice communication can be performed in a state close to the real space. However, in an actual conversation in the real space, the conversation conditions and volume are not determined only by the positional relationship between users.

[0011] Therefore, an object of the present invention is to enable voice communication to be performed in a virtual space that allows users to communicate with each other by voice in a state closer to the real space.

Means for Solving the Problems

[0012] In order to solve the above-described problems, in the present invention, in a system that enables users using user terminals to communicate with each other by voice through a virtual space provided to a plurality of user terminals connected via a communication network, among the users participating in the virtual space, when a first user speaks to a second user in a volume smaller than a predetermined value and the second user responds in a volume smaller than the predetermined value, a secret conversation mode in which conversation is possible only between the first user and the second user is set.

Effects of the Invention

[0013] According to the present invention configured as described above, in a virtual space that enables users to communicate with each other by voice, when it is desired to have a secret conversation only between specific users, voice communication can be performed in a state close to a secret conversation that is carried out only between two specific people in the real space.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the overall configuration of a virtual space communication system according to this embodiment. As shown in FIG. 1, the virtual space communication system of this embodiment includes a server device 10 and a plurality of user terminals 20. The server device 10 and the user terminals 20 are connected via a communication network 30 such as the Internet and a mobile phone network.

[0016] The user terminal 20 is composed of, for example, a smartphone, a tablet, a personal computer, etc. The user terminal 20 incorporates a voice input device such as a microphone or is configured to be connectable by wire or wirelessly. Further, the user terminal 20 incorporates a voice output device such as a speaker or is configured to be connectable by wire or wirelessly. A headset equipped with a microphone and a speaker may be connected to the user terminal 20.

[0017] The server device 10 provides a virtual space for a plurality of user terminals 20 connected via the communication network 30, enabling users who use the user terminals 20 to communicate by voice through the virtual space. That is, the server device 10 receives the speaker voice input from the voice input device of one user terminal 20 via the communication network 30, and transmits the received speaker voice to other user terminals 20 via the communication network 30 to be output from the voice output device.

[0018] The virtual space is a space in which a virtual user corresponding to a user can move freely. A user who uses the user terminal 20 moves his / her virtual user within the virtual space and conducts voice communication with the user corresponding to another virtual user encountered at the destination location. The user terminal 20 is installed with a dedicated application program (hereinafter referred to as a conversation app) for performing processing related to the operation of such a virtual user within the virtual space and voice communication. The "virtual user corresponding to a user" means a virtual user set in the conversation app installed by the user on the user terminal 20.

[0019] The conversation app transmits an operation instruction of the virtual user input by the user to the server device 10. The server device 10 controls the operation of the virtual user within the virtual space according to the operation instruction received from the user terminal 20. Further, the conversation app transmits the speaker voice input from the voice input device of the user terminal 20 to the server device 10. The server device 10 transmits the speaker voice received from one user terminal 20 to another user terminal 20 and causes it to be output from the voice output device.

[0020] In this embodiment, the virtual space provided by the server device 10 to the user terminal 20 is, for example, a non-display virtual space in which there is no image displayed on the user terminal 20 and only voice exists. There is no image of the space or the virtual user displayed on the user terminal 20, but the virtual space itself exists, and the server device 10 manages the position information of the virtual space and the existence position information of the virtual user within the virtual space. For example, when a movement instruction of the virtual user is given by the user, the server device 10 updates the stored existence position information of the virtual user according to the movement instruction.

[0021] The user grasps the position of the virtual user in the virtual space and the surrounding situation based on the voice output from the voice output device. Also, the user moves the virtual user in the virtual space by giving voice operation instructions from the voice input device, and has a voice conversation with the user corresponding to another virtual user encountered at the destination using the voice input device and the voice output device.

[0022] Note that the user terminal 20 is equipped with an image display device such as a display, or is configured to be connectable by wire or wirelessly. However, when having a voice communication through the virtual user in the virtual space, the screen display of the virtual space on the display is not performed.

[0023] FIG. 2 is a block diagram showing a functional configuration example of the server device 10 according to the present embodiment. As shown in FIG. 2, the server device 10 of the present embodiment includes, as functional configurations, a communication control unit 11, an operation control unit 12, and a user information registration unit 13. The communication control unit 11 includes, as more specific functional configurations, a mode setting unit 11a and a voice control unit 11b. The server device 10 also includes, as storage media, a user information storage unit 14 and a space information storage unit 15.

[0024] The above functional blocks 11 to 13 execute the processes described below through the cooperation of hardware and software. For example, the processes of the above functional blocks 11 to 13 are executed by the operation of a program stored in a storage medium such as a RAM, a ROM, a hard disk, or a semiconductor memory under the control of a microcomputer configured with a CPU, a RAM, a ROM, etc. In addition to the microcomputer, a DSP (Digital Signal Processor) or the like may be provided.

[0025] The user information registration unit 13 registers user information regarding the users who use the conversation application. That is, the user information registration unit 13 receives the user information transmitted from the user terminal 20 and stores it in the user information storage unit 14. For example, when the user installs the conversation application on the user terminal 20, the user operates the user terminal 20 to input user information and sends this to the server device 10 to make a registration request. The server device 10, upon receiving this registration request, stores the user information received from the user terminal 20 in the user information storage unit 14.

[0026] The user information registered in the user information storage unit 14 includes, for example, the user's name, nickname used as the name of the virtual user, user ID, login password, and the like. The name and / or nickname are used as identification information for the users when they communicate with each other by voice. For this reason, it is preferable to register the nickname as a name unique to each user. Therefore, when there is a new registration request for a nickname from the user terminal 20, the user information registration unit 13 determines whether it is the same as the nickname already registered in the user information storage unit 14, and if it is the same, responds to the user terminal 20 with a message prompting the setting of another nickname.

[0027] The user information storage unit 14 also stores information regarding the virtual users corresponding to the users. For example, the user information storage unit 14 receives the setting mode information indicating the mode set for the virtual user by the mode setting unit 11a from the mode setting unit 11a and stores it in association with the user ID. Further, the user information storage unit 14 stores the presence position information and the traveling direction information of the virtual user moving within the virtual space under the control of the operation control unit 12 in association with the user ID.

[0028] The communication control unit 11 controls voice communication among users participating in the virtual space. The users participating in the virtual space refer to the users who have launched the conversation application and logged in to the server device 10. The communication control unit 11 controls the mode set for the virtual user by the mode setting unit 11a, and controls the dialogue voice communicated between the user terminals 20 by the voice control unit 11b according to the set mode. Details of these processes will be described later.

[0029] The operation control unit 12 controls the operation of the virtual user in the virtual space corresponding to the user participating in the virtual space according to the operation instruction from the user transmitted from the user terminal 20. As described above, the operation instruction of the virtual user is performed by the user's voice. That is, the operation control unit 12 controls the operation of the virtual user in the virtual space according to the instruction by voice input from the voice input device used by the user.

[0030] The operation of the virtual user includes the movement of the virtual user in the virtual space. The operation instruction includes instructions such as forward, backward, and turning. The operation instruction also includes an instruction to enter or exit the conversation space provided in the virtual space. One or more conversation spaces are provided in the virtual space. The space information storage unit 15 stores in advance the position information indicating the entire position of the virtual space and the position information indicating the position where the conversation space exists in the virtual space.

[0031] For example, when the user instructs forward, the operation control unit 12 controls the virtual user to move forward by a predetermined distance. The user may also instruct the distance to move forward. For example, when the user says "forward", the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction by voice recognition processing and controls the virtual user to move forward by a predetermined distance. That is, the existence position information of the virtual user stored in the user information storage unit 14 is updated to the information indicating the position advanced by a predetermined distance in the direction indicated by the traveling direction information.

[0032] Also, when the user instructs a right turn, the operation control unit 12 controls the virtual user to turn right by a predetermined angle. The user may also be allowed to instruct the turning angle. For example, when the user says "turn right", the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction by voice recognition processing and controls the virtual user to turn right by a predetermined angle. That is, the traveling direction information of the virtual user stored in the user information storage unit 14 is updated to information indicating a direction displaced by a predetermined angle to the right.

[0033] Also, when the user instructs to enter the conversation space, the operation control unit 12 controls the virtual user to enter the conversation space. For example, when the user says "enter", the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction by voice recognition processing and controls the virtual user to enter the conversation space. That is, the presence position information of the virtual user stored in the user information storage unit 14 is updated to information indicating a position within the conversation space. Note that such an entry process may be made effective only when the virtual user exists at a location within a predetermined distance from the conversation space.

[0034] Also, when the user instructs to exit the conversation space, the operation control unit 12 controls the virtual user to exit the conversation space. For example, when the user says "exit", the uttered voice is input from the voice input device of the user terminal 20 and transmitted to the server device 10. The operation control unit 12 interprets the uttered voice related to this operation instruction by voice recognition processing and controls the virtual user to exit the conversation space. That is, the presence position information of the virtual user stored in the user information storage unit 14 is updated to information indicating a position outside the conversation space (for example, the position where the virtual user existed before entering the conversation space). Note that such an exit process may be made effective only when the virtual user is inside the conversation space.

[0035] When a user logs in to the server device 10 from the conversation application for the first time, the virtual user of that user is set to exist at a predetermined start position within the virtual space. That is, the existence position information of the virtual user stored in the user information storage unit 14 is initially set to the predetermined start position. On the other hand, when the user logs in for the second time or later, the virtual user of that user is set to the position where it existed at the time of the previous logout. That is, the information indicating the position where the virtual user existed at the time of logout is stored in the user information storage unit 14 and used as the existence position information of the virtual user at the time of the next login.

[0036] The mode setting unit 11a of the communication control unit 11 sets the mode according to the user's uttered voice or the operation of the virtual user instructed by the user. The voice control unit 11b controls the voice transmitted and received between the user terminal 20 according to the mode set by the mode setting unit 11a. The control of the voice includes the control of the volume and the sound image localization process.

[0037] The modes set by the mode setting unit 11a are three modes: the roaming mode, the normal conversation mode, and the secret conversation mode. The roaming mode is a mode set when the virtual user is outside the conversation space. The normal conversation mode is a normal mode set when the virtual user is inside the conversation space. The secret conversation mode is a mode set when the virtual user is inside the conversation space and the user's uttered voice is in a predetermined state.

[0038] When a virtual user enters a conversation space provided in the virtual space, the mode setting unit 11a sets a normal conversation mode that enables conversation among a plurality of users corresponding to the plurality of virtual users in the conversation space for that virtual user. Whether the virtual user has entered the conversation space can be determined by comparing the presence position information of the virtual user stored in the user information storage unit 14 with the position information of the conversation space stored in the space information storage unit 15. When the mode setting unit 11a detects that the virtual user has entered the conversation space, it stores, in the user information storage unit 14 in association with the user ID, setting mode information indicating that the normal conversation mode has been set for that virtual user.

[0039] The voice control unit 11b controls the voice output device used by the user corresponding to one virtual user in the conversation space and having the normal conversation mode set to output the speech voice of the user (the speech voice transmitted from the user terminal 20 of that user).

[0040] That is, the voice control unit 11b refers to the presence position information of the virtual user stored in the user information storage unit 14 to identify a plurality of virtual users existing in one conversation space. Then, when receiving the speech voice of the user from the user terminal 20 corresponding to one virtual user existing in that conversation space and having the normal conversation mode set, it transmits that speech voice to the user terminals 20 corresponding to one or more other virtual users in the same conversation space and controls it to be output from the voice output device.

[0041] Here, one or more other virtual users for which the speaker's voice is to be output may include both a virtual user in which the normal conversation mode is set and a virtual user in which the private conversation mode is set. Alternatively, as will be described in detail later, one or more other virtual users may be only virtual users in which the normal conversation mode is set, excluding virtual users in which the private conversation mode is set. Further, the user terminal 20 for which the uttered voice is to be output may be other than the user terminal 20 that transmitted the uttered voice, or may include the user terminal 20 that transmitted the uttered voice.

[0042] When the mode setting unit 11a and the voice control unit 11b perform the above-described processing, when there are a plurality of virtual users in the conversation space, the plurality of users corresponding to the plurality of virtual users can perform voice communication in a state close to a conversation in the real space.

[0043] Further, when, among the plurality of users corresponding to the plurality of virtual users in the conversation space, a first user speaks to a second user in a volume smaller than a predetermined value and the second user responds in a volume smaller than the predetermined value, the mode setting unit 11a sets a private conversation mode in which conversation is possible only between the first user and the second user for the virtual users corresponding to the first user and the second user. The mode setting unit 11a stores setting mode information indicating that the private conversation mode has been set for two virtual users in the user information storage unit 14 in association with the user ID.

[0044] For example, when the first user utters the identification information (e.g., nickname) of the second user in a volume smaller than a predetermined value and the second user responds in a volume smaller than the predetermined value, the mode setting unit 11a sets the private conversation mode. In this case, the response of the second user may be any content as long as it is an utterance in a volume smaller than the predetermined value. Alternatively, the private conversation mode may be set only when the second user also utters the identification information of the first user in a volume smaller than the predetermined value.

[0045] That is, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminal 20 of the user corresponding to the virtual user existing in the conversation space, and when detecting an uttered voice at a volume smaller than a predetermined value, interprets the uttered voice by voice recognition processing. Then, it determines whether or not the content of the interpreted uttered voice matches any of the nicknames of the virtual users in the conversation space stored in the user information storage unit 14. Here, when it is determined that there is no match with any nickname, the mode setting unit 11a does not set the secret conversation mode.

[0046] On the other hand, when it is determined that there is a match with any one of the nicknames, the mode setting unit 11a identifies the user of the user terminal 20 that is the transmission source of the uttered voice as the first user. Also, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminal 20 of the user corresponding to the virtual user with the matching nickname within a predetermined time, and when detecting an uttered voice at a volume smaller than a predetermined value, the mode setting unit 11a identifies the user who responded by uttering as the second user, and sets the secret conversation mode for the virtual users corresponding to the first user and the second user.

[0047] In addition, when setting the secret conversation mode only when the second user also utters the identification information of the first user, the mode setting unit 11a interprets by voice recognition processing the uttered voice at a small volume transmitted from the user terminal 20 of the user who matches the nickname uttered by the first user within a predetermined time. Then, it determines whether or not the content of the interpreted uttered voice matches the nickname of the first user stored in the user information storage unit 14, and when it is determined that there is a match, sets the secret conversation mode.

[0048] When the secret conversation mode is set, the voice control unit 11b controls to output the uttered voice of the first user only from the voice output device used by the second user, while outputting the uttered voice of the second user only from the voice output device used by the first user.

[0049] That is, when the voice control unit 11b receives the uttered voice from the user terminal 20 of the first user in whom the secret conversation mode is set, it controls to transmit the uttered voice only to the user terminal 20 of the second user in whom the secret conversation mode is also set so as to be output from the voice output device. Further, when the voice control unit 11b receives the uttered voice from the user terminal 20 of the second user, it controls to transmit the uttered voice only to the user terminal 20 of the first user so as to be output from the voice output device.

[0050] During the setting of the secret conversation mode, the mode setting unit 11a monitors whether at least one of the first user and the second user has uttered at a volume equal to or higher than a predetermined value. And when it is detected that at least one user has uttered at a volume equal to or higher than the predetermined value, the setting of the secret conversation mode is cancelled. That is, the mode setting for the virtual users corresponding to the first user and the second user is returned to the normal conversation mode.

[0051] By the mode setting unit 11a and the voice control unit 11b performing the above-described processing, when a plurality of users are having a conversation in the conversation space, a secret conversation can be held quietly only between the first user and the second user among them. Thereby, voice communication can be performed in a state close to a secret conversation conducted only between a specific two people in the real space.

[0052] Even when the secret conversation mode is set for the first user and the second user, other users in the same conversation space can have a normal (non-secret) conversation under the setting of the normal conversation mode. Here, the voice control unit 11b may be configured not to transmit the uttered voices of other users in the same conversation space to the user terminals 20 of the first user and the second user. By doing so, it is possible to prevent the uttered voices of the other users output to the voice output devices of the first user and the second user having a secret conversation at a volume less than the predetermined value from being difficult to hear due to the uttered voices of other users output to the voice output devices at a volume equal to or higher than the predetermined value.

[0053] Similarly, when transmitting the spoken voice of the nickname (the spoken voice with a volume smaller than a predetermined value) sent from the user terminal 20 of the first user to the user terminal 20 of the counterpart user corresponding to the nickname (that is, the candidate user who can be the second user), the spoken voices of other users in the same conversation space may not be transmitted to the user terminal 20 of the candidate user. By doing so, the candidate user can make it difficult to hear the spoken voice of the nickname output to the voice output device of the candidate user at a volume less than the predetermined value due to the spoken voices of other users output to the voice output device at a volume equal to or greater than the predetermined value, so as not to miss hearing it.

[0054] In addition, when the virtual user is outside the conversation space, the mode setting unit 11a sets the roaming mode for the virtual user, and stores the setting mode information indicating that the roaming mode is set in the user information storage unit 14 in association with the user ID. In this case, the user in whom the roaming mode is set may say anything, but the voice control unit 11b does not transmit the spoken voice of that user to the user terminals 20 of other users. On the other hand, the voice control unit 11b controls the voice being talked about in the conversation space to be output from the voice output device used by the user corresponding to the virtual user in whom the roaming mode is set at a volume smaller than the volume at which it is output when the virtual user is in the conversation space.

[0055] For example, the voice control unit 11b transmits the spoken voice sent from the user terminal 20 of a user in the conversation space with the normal conversation mode set without reducing the volume to the user terminals 20 of other users (only the users with the normal conversation mode set, or in addition to that, the users with the private conversation mode set) in the same conversation space. On the other hand, the voice control unit 11b transmits the spoken voice sent from the user terminal 20 of a user in the conversation space with the normal conversation mode set after reducing the volume to the user terminals 20 of other users outside the conversation space with the roaming mode set.

[0056] At this time, the voice control unit 11b may perform sound image localization of the voice output to the voice output device according to the positional relationship between the virtual user outside the conversation space and the conversation space. That is, the voice control unit 11b controls the voice output to the voice output device so that the voice can be heard from a certain direction of the conversation space according to the direction from the virtual user outside the conversation space to the conversation space.

[0057] In addition, the voice control unit 11b controls the volume of the voice output to the voice output device according to the distance between the virtual user outside the conversation space and the conversation space, so that the volume becomes smaller as the distance is longer and larger as the distance is shorter. However, even when the virtual user is very close to the conversation space and the maximum volume is set, it is made smaller than the volume output to the voice output device of the user who is in the conversation space and in the normal conversation mode. Also, when the virtual user is at a position more than a certain distance away from the conversation space, the volume output to the voice output device of the user corresponding to the virtual user is set to zero (the voice of speech is not transmitted to the user terminal 20 of the user).

[0058] By performing such sound image localization processing by the voice control unit 11b, even if the virtual space provided to the user terminal 20 is a non-display virtual space with only voice, the user can rely on the sound image-localized voice output from the voice output device and roam in the virtual space while grasping the direction and sense of distance of the conversation space, and enter any conversation space to have a conversation. In this embodiment, since all operations including operation instructions for the virtual user and conversations between users can be performed by voice, for example, even visually impaired people can enjoy voice communication in the virtual space using the conversation application.

[0059] FIG. 3 is a flowchart showing an operation example of the mode setting unit 11a according to the present embodiment. This FIG. 3 shows an example of a process for setting a mode for one virtual user, and the process of the flowchart shown in FIG. 3 is executed for all virtual users who are logged in. Note that FIG. 3 shows an operation example executed at the time of the first login.

[0060] The mode setting unit 11a sets the roaming mode as the initial state mode (step A1). Thereafter, the mode setting unit 11a determines whether or not the virtual user has entered the conversation space (step A2). For example, when the operation control unit 12 executes a process of allowing the virtual user to enter the conversation space by the user uttering "enter", the presence position information of the virtual user is updated to information indicating that the virtual user is in the conversation space. When the mode setting unit 11a confirms this, it determines that the virtual user has entered the conversation space.

[0061] Here, when the mode setting unit 11a determines that the virtual user has not entered the conversation space, the process returns to step A1. On the other hand, when the mode setting unit 11a determines that the virtual user has entered the conversation space, it sets the normal conversation mode for the virtual user (step A3). Thereafter, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminal 20 of the user corresponding to the virtual user for whom the normal conversation mode has been set, and determines whether or not the volume of the uttered voice is smaller than a predetermined value (step A4).

[0062] Here, when it is detected that an uttered voice with a volume smaller than the predetermined value is transmitted, the mode setting unit 11a interprets the uttered voice by voice recognition processing, and determines whether or not the content of the uttered voice matches any of the nicknames of the virtual users in the conversation space stored in the user information storage unit 14 (step A5). Here, when the mode setting unit 11a determines that the content does not match any of the nicknames, the process proceeds to step A12.

[0063] If it is determined that the voice of the utterance matches any one of the nicknames, the mode setting unit 11a identifies the user of the user terminal 20, which is the transmission source of the uttered voice, as the first user. Further, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminal 20 of the virtual user with the matching nickname within a predetermined time, and determines whether there is a response by an uttered voice with a volume smaller than a predetermined value (step A6).

[0064] Here, if an uttered voice with a volume smaller than the predetermined value has not been received, the process proceeds to step A12. On the other hand, if it is detected that an uttered voice with a volume smaller than the predetermined value has been received, the mode setting unit 11a identifies the user of the user terminal 20, which is the transmission source of the uttered voice, as the second user, and sets the secret conversation mode for the virtual users corresponding to the first user and the second user (step A7).

[0065] Thereafter, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminals 20 of the first user and the second user for whom the secret conversation mode has been set, and determines whether an uttered voice with a volume equal to or greater than the predetermined value has been received from at least one of the user terminals 20 (step A8). Here, if an uttered voice with a volume equal to or greater than the predetermined value has not been received, the process returns to step A7 and the setting of the secret conversation mode is continued. On the other hand, if the mode setting unit 11a determines that an uttered voice with a volume equal to or greater than the predetermined value has been received from at least one of the user terminals 20, the process returns to step A3 and the normal conversation mode is set.

[0066] In step A4 above, if an uttered voice with a volume smaller than the predetermined value has not been detected, the mode setting unit 11a monitors the volume of the uttered voice transmitted from the user terminal 20 of the user corresponding to the virtual user for whom the normal conversation mode has been set in step A3, from the user terminal 20 of another user for whom the normal conversation mode is set, and determines whether an uttered voice with a volume smaller than the predetermined value has been received (step A9).

[0067] Here, when no speech voice with a volume smaller than a predetermined value is received, the process proceeds to step A12. On the other hand, when it is detected that a speech voice with a volume smaller than the predetermined value is received, the mode setting unit 11a interprets the speech voice by voice recognition processing, and determines whether the content of the speech voice matches the nickname of the user himself / herself (the virtual user who set the normal conversation mode in step A3) stored in the user information storage unit 14 (step A10). Here, when the mode setting unit 11a determines that it does not match the user's own nickname, the process proceeds to step A12.

[0068] On the other hand, when it is determined that it matches the user's own nickname, the mode setting unit 11a monitors the volume of the speech voice transmitted from the user's own user terminal 20 within a predetermined time, and determines whether a speech voice with a volume smaller than the predetermined value is used for response (step A11).

[0069] Here, when no speech voice with a volume smaller than the predetermined value is received, the process proceeds to step A12. On the other hand, when it is detected that a speech voice with a volume smaller than the predetermined value is received, the process proceeds to step A7. In this case, the mode setting unit 11a identifies the responding user as the second user, and sets the secret conversation mode for the virtual users corresponding to the first user and the second user.

[0070] In step A12, the mode setting unit 11a determines whether the virtual user has left the conversation space. For example, when the operation control unit 12 executes a process of moving the virtual user outside the conversation space because the user uttered "exit", the presence position information of the virtual user is updated to information indicating that the virtual user is outside the conversation space. When the mode setting unit 11a confirms this, it determines that the virtual user has left the conversation space.

[0071] Here, when the mode setting unit 11a determines that the virtual user has not left the conversation space, the process returns to step A3, and the setting of the normal conversation mode is continued. On the other hand, when the mode setting unit 11a determines that the virtual user has left the conversation space, the process returns to step A1, and the roaming mode is set.

[0072] Note that, as shown in FIG. 3, the roaming mode is initially set at the time of the first login. However, at the time of subsequent logins, the process starts from the mode setting (step A3 or step A7) corresponding to the position where the virtual user existed at the time of the previous logout.

[0073] FIG. 4 is a flowchart showing an operation example of the voice control unit 11b according to the present embodiment. This FIG. 4 shows an operation example when processing one speaker voice transmitted from one user terminal 20, and the processing of the flowchart shown in FIG. 4 is executed each time the speaker voice transmitted from the logged-in user terminal 20 is received.

[0074] The voice control unit 11b determines whether or not it has received a speaker voice from the user terminal 20 (step B1). The voice control unit 11b continues the determination in step B1 until it receives one speaker voice. When the server device 10 receives one speaker voice, the voice control unit 11b determines the currently set mode for the virtual user corresponding to the user of the user terminal 20 that transmitted the speaker voice (step B2).

[0075] Here, if it is determined that the mode set for the virtual user is the roaming mode, the process returns to step B1. That is, the voice control unit 11b does not transmit the speaker voice to other user terminals 20 and waits for the reception of the next speaker voice.

[0076] If it is determined that the mode set for the virtual user is the normal conversation mode, the voice control unit 11b refers to the presence position information of the virtual user stored in the user information storage unit 14, identifies one or more other virtual users in the same conversation space as the virtual user who is the speaker, and transmits the uttered voice to one or more user terminals 20 corresponding to the one or more other virtual users and controls it to be output from the voice output device (step B3).

[0077] Further, the voice control unit 11b refers to the location information of the virtual user stored in the user information storage unit 14, and identifies one or more virtual users who are outside the conversation space and are within a certain distance from the conversation space where the virtual user who is both a speaker and a virtual user is located. Then, for one or more user terminals 20 corresponding to the identified one or more virtual users, the voice control unit 11b controls to transmit the uttered voice subjected to sound image localization according to the positional relationship between each virtual user and the conversation space, and output it from the voice output device (step B4). After that, the process returns to step B1.

[0078] When it is determined that the mode set for the virtual user is the private conversation mode, the voice control unit 11b refers to the setting information of the private conversation mode stored in the user information storage unit 14, identifies the virtual user on the other side for which the private conversation mode is set, and controls to transmit the uttered voice only to the user terminal 20 corresponding to the identified virtual user on the other side and output it from the voice output device (step B5). After that, the process returns to step B1.

[0079] FIG. 5 is a flowchart showing an operation example of the operation control unit 12 according to the present embodiment. This FIG. 5 shows an example of the process for controlling the operation of one virtual user, and the process of the flowchart shown in FIG. 5 is executed for all the logged-in virtual users.

[0080] The operation control unit 12 determines whether or not it has received an operation instruction from the user terminal 20 (step C1). The operation control unit 12 continues the determination in step C1 until it receives an operation instruction. When the server device 10 receives an operation instruction, the operation control unit 12 controls the operation of the virtual user in the virtual space according to the operation instruction (step C2). After that, the process returns to step C1.

[0081] As described in detail above, in the virtual space communication system of the present embodiment, the server device 10 provides a virtual space to a plurality of user terminals 20 connected via a communication network 30, and enables users using the user terminals 20 to communicate with each other by voice through the virtual space. In particular, in the present embodiment, when a first user speaks to a second user in a volume smaller than a predetermined value and the second user responds in a volume smaller than the predetermined value, a secret conversation mode is set in which conversation is possible only between the first user and the second user. When the secret conversation mode is set, the speech voice of the first user is output only from the voice output device used by the second user, while the speech voice of the second user is output only from the voice output device used by the first user.

[0082] According to the virtual space communication system of the present embodiment configured as described above, in a virtual space where users can communicate with each other by voice, when a specific user wants to have a secret conversation only with themselves, they can conduct a voice communication in a state similar to a secret conversation that takes place only between two specific people in the real space.

[0083] In the above embodiment, an example in which the virtual space provided to the user terminal 20 is a non-display virtual space with only voice and the operation instructions for the non-display virtual user are given only by voice has been described, but the present invention is not limited to this. For example, the user interface for giving operation instructions may be displayed on the screen.

[0084] FIG. 6 is a diagram showing an example of a user interface displayed on the display of the user terminal 20. As shown in FIG. 6, the virtual space and the virtual user are not displayed, and only operation buttons for giving instructions to move forward, backward, turn, enter, and exit are displayed. In this case, the user can give operation instructions to the virtual user using the voice or the user interface displayed on the screen.

[0085] As another example, when the user terminal 20 is a smartphone, tablet, etc. with a built-in acceleration sensor or the like, an operation instruction for the virtual user may be given according to the movement of the user terminal 20 detected by the acceleration sensor. For example, when the user terminal 20 is shaken in the front-back, left-right directions, the virtual user's forward movement, backward movement, left turn, and right turn may be respectively instructed. When the user terminal 20 is rotated to the right, the virtual user's entry into the room may be instructed. When the user terminal 20 is rotated to the left, the virtual user's exit from the room may be respectively instructed.

[0086] Further, instead of the non-display virtual space, a display virtual space for displaying the virtual space and the virtual user on the screen may be used. Even in this case, if an operation instruction for the virtual user is given by voice, an acceleration sensor, etc., the user can communicate by voice without looking at the display virtual space. Needless to say, when the user desires, it is possible to communicate by voice while looking at the display virtual space.

[0087] In the above embodiment, smartphones, tablets, and personal computers are cited as examples of the user terminal 20, but it is not limited thereto. For example, the user terminal 20 may be a wearable terminal such as a Google-type terminal, a glasses-type terminal, or a wristwatch-type terminal. Further, when the virtual space is a display virtual space, a head-mounted display capable of displaying VR (virtual reality) images may be used as the user terminal 20.

[0088] In the above embodiment, when the roaming mode is set for the virtual user, an example has been described in which the voice being spoken in the conversation space within a certain distance from the position of the virtual user is audio-located and supplied to the user terminal 20 of the user corresponding to the virtual user. Instead of or in addition to this, another voice other than the uttered voice may be supplied and output from the voice output device.

[0089] For example, specific sound for notifying the presence of a conversation space may be sound localized and supplied to the user terminal 20 in the roaming mode. In this case, special sound may be supplied for a conversation space where conversation is actively taking place. The definition of actively taking place conversation can be arbitrarily set. For example, when either or both of the number of virtual users existing in the conversation space being equal to or more than a threshold value and the number of utterances per predetermined time being equal to or more than a threshold value are satisfied, it is possible to determine that conversation is actively taking place.

[0090] Also, in the above embodiment, an example in which there are three modes, i.e., the roaming mode, the normal conversation mode, and the private conversation mode, has been described, but the present invention is not limited to this. For example, there may be only two modes, i.e., the normal conversation mode and the private conversation mode. For example, the virtual space itself may be regarded as one conversation space, and the virtual users who have logged in always exist in one conversation space (= virtual space), and the normal conversation mode and the private conversation mode may be appropriately switched there. Alternatively, a plurality of conversation spaces may exist in the virtual space, and as an operation instruction to the virtual user, an instruction to jump from one conversation space to another conversation space may be given.

[0091] In addition, each of the above embodiments merely shows an example of implementation when carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by this. That is, the present invention can be implemented in various forms without departing from the gist or its main features.

Explanation of reference numerals

[0092] 10 Server device 11 Communication control unit 11a Mode setting unit 11b Voice control unit 12 Operation control unit 13 User information registration unit 14 User information storage unit 15 Space information storage unit 20 User terminal 30 Communication network

Claims

1. A system that enables users of a plurality of user terminals connected via a communication network to communicate with each other by voice through a virtual space provided to the user terminals, comprising: a communication control unit that controls voice communication between users participating in the virtual space, The communication control unit sets a secret conversation mode in which only the first user and the second user can have a conversation when a first user speaks to a second user at a volume lower than a predetermined value and the second user responds within a predetermined time at a volume lower than the predetermined value. A virtual space communication system comprising:

2. The virtual space communication system of claim 1, characterized in that when the communication control unit detects that a user's identification information has been spoken at a volume lower than the predetermined value, it identifies the user who made the speech as a first user, and when it detects that a user corresponding to the identification information has responded at a volume lower than the predetermined value, it identifies the user who responded as a second user, and sets the secret conversation mode between the first user and the second user.

3. The virtual space communication system described in claim 1, characterized in that the communication control unit cancels the setting of the secret conversation mode when at least one of the first user and the second user speaks at a volume equal to or higher than the predetermined value while the secret conversation mode is set.

4. and a motion control unit that controls a motion of a virtual user corresponding to a user participating in the virtual space in response to an instruction from the user, The communication control unit includes: when the virtual user enters a conversation space provided within the virtual space, a normal conversation mode is set for the virtual user, enabling conversation between a plurality of users corresponding to the plurality of virtual users in the conversation space; The secret conversation mode is set for virtual users corresponding to the first user and the second user among the plurality of users corresponding to the plurality of virtual users in the conversation space.

4. The virtual space communication system according to claim 1, wherein the virtual space communication system comprises: a communication unit that communicates with the communication unit;

5. The communication control unit includes: controlling the speech of a user corresponding to one virtual user in the conversation space and for whom the normal conversation mode is set, to be output from a voice output device used by a user corresponding to one or more other virtual users in the same conversation space; A voice of the first user corresponding to a virtual user who is in the conversation space and for whom the secret conversation mode is set is output only from a voice output device used by the second user, while a voice of the second user is controlled to be output only from a voice output device used by the first user.

5. The virtual space communication system according to claim 4.

6. The virtual space communication system described in claim 4, characterized in that the communication control unit controls the audio being conversed in the conversation space when the virtual user is outside the conversation space to be output from an audio output device used by a user corresponding to the virtual user outside the conversation space at a volume lower than the volume output when the virtual user is inside the conversation space.

7. The virtual space communication system described in claim 6, characterized in that the communication control unit performs sound image localization of the audio output to the audio output device depending on the positional relationship between a virtual user outside the conversation space and the conversation space.

8. 5. The virtual space communication system according to claim 4, wherein the virtual space is a non-display virtual space in which there is no image displayed on the screen of the user terminal and only audio exists.

9. The virtual space communication system described in claim 8, characterized in that the action control unit controls the action of a virtual user corresponding to a user participating in the virtual space in accordance with voice instructions input from a voice input device used by the user.

10. A communication control device that provides a virtual space to a plurality of user terminals connected via a communication network and controls voice communication between users of the user terminals through the virtual space, a communication control unit that controls voice communication between users participating in the virtual space, The communication control unit sets a secret conversation mode in which only the first user and the second user can have a conversation when a first user speaks to a second user at a volume lower than a predetermined value and the second user responds within a predetermined time at a volume lower than the predetermined value. A communication control device comprising:

11. A communication control method for controlling voice communication between users using a plurality of user terminals connected via a communication network through a virtual space provided to the user terminals, the method comprising: A communication control unit of the computer sets a secret conversation mode in which only the first user and the second user can converse with each other when a first user among a plurality of users participating in the virtual space speaks to a second user at a volume lower than a predetermined value and the second user responds within a predetermined time at a volume lower than the predetermined value. A communication control method comprising:

Images