Virtual space control system, virtual space control method, and virtual space control program

Abstract

To make it easier to understand communication between avatars in a virtual space. [Solution] The virtual space control system includes a control device that places a first avatar of a first user using a first terminal device in the real world and a second avatar of a second user using a second terminal device in the real world in a virtual space, and when a conversation between the first user and the second user in the real world is detected while the first avatar and the second avatar are placed in the virtual space, generates a surrounding area in the virtual space that encloses the first avatar and the second avatar, and displays the virtual space on the display unit of the terminal device including the first terminal device and the second terminal device.

Description

Technical Field

[0006]

[0001] The present invention relates to a virtual space control system, a virtual space control method, and a virtual space control program.

Background Art

[0002] A technology in which a user of a terminal device uses an avatar in a virtual space is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the virtual space as described above, it may be difficult for the user to grasp the communication between avatars, such as which avatar is communicating with which avatar.

[0005] In view of the above problems, an object of the present invention is to provide a virtual space control system, a virtual space control method, and a virtual space control program that make it easy to grasp the communication between avatars in a virtual space.

Means for Solving the Problems

[0006] The virtual space control system according to the present invention includes a control device that places a first avatar of a first user using a first terminal device in real space and a second avatar of a second user using a second terminal device in real space in a virtual space, and when a conversation between the first user and the second user in real space is detected while the first avatar and the second avatar are placed in the virtual space, generates a surrounding area in the virtual space that encloses the first avatar and the second avatar, and displays the virtual space on the display unit of a terminal device including the first terminal device and the second terminal device.

[0007] The virtual space control method according to the present invention includes the steps of: placing a first avatar of a first user using a first terminal device in real space and a second avatar of a second user using a second terminal device in real space in a virtual space; when a conversation between the first user and the second user in real space is detected while the first avatar and the second avatar are placed in the virtual space, generating a surrounding area in the virtual space that encloses the first avatar and the second avatar; and causing the control device to display the virtual space on the display unit of a terminal device including the first terminal device and the second terminal device.

[0008] The virtual space control program according to the present invention causes a computer to execute the following processes: placing a first avatar of a first user using a first terminal device in real space and a second avatar of a second user using a second terminal device in real space in a virtual space; generating a surrounding area in the virtual space for the first and second avatars when a conversation between the first and second users in real space is detected while the first and second avatars are placed in the virtual space; and displaying the virtual space on the display unit of a terminal device including the first and second terminal devices. [Effects of the Invention]

[0009] According to the present invention, it becomes easier to understand communication between avatars in a virtual space. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic diagram showing an example configuration of a virtual space control system. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of a terminal device. [Figure 3] Figure 3 is a block diagram showing an example of a server device configuration. [Figure 4] Figure 4 is a functional block diagram showing an example of a control device. [Figure 5] Figure 5 is a flowchart showing an example of a virtual space control method. [Figure 6] Figure 6 shows an example of a virtual space displayed on the terminal device's display unit. [Figure 7] Figure 7 shows an example of a virtual space displayed on the terminal device's display unit. [Figure 8] Figure 8 shows an example of a virtual space displayed on the terminal device's display unit. [Figure 9] Figure 9 shows an example of a virtual space displayed on the terminal device's display unit. [Figure 10] Figure 10 shows an example of the correspondence between the type of conversation between the first user and the second user and the distance between the first avatar and the second avatar. [Figure 11] Figure 11 shows an example of a virtual space displayed on the terminal device's display unit. [Figure 12] Figure 12 shows an example of a virtual space displayed on the terminal device's display unit. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the virtual space control system, virtual space control method, and virtual space control program according to the present invention will be described with reference to the drawings. However, the present invention is not limited by these embodiments. Furthermore, the components in the following embodiments include those that are easily substituted or substantially identical to those that are easily substituted by those skilled in the art.

[0012] FIG. 1 is a schematic diagram showing a configuration example of a virtual space control system. The virtual space control system 1 controls a virtual space V. The virtual space V is a space in which a user configured in the virtual space can log in using a terminal device 10 described later. In the virtual space V, an avatar of the logged-in user is arranged.

[0013] In the following description, the "user" may be appropriately referred to as the "first user", and the user who communicates with the first user, such as having a conversation, may be referred to as the "second user". Also, a user other than the first user and the second user may be referred to as the third user. The terminal device used by the user who is the first user to log in to the virtual space V is referred to as the first terminal device, and the terminal device used by the second user to log in to the virtual space V may be referred to as the second terminal device. Also, the terminal device used by the third user to log in to the virtual space V may be referred to as the third terminal device. The first terminal device is any terminal device logged in to the virtual space V, the second terminal device is a terminal device different from the first terminal device, and is any terminal device logged in to the same virtual space V as the first terminal device. The third terminal device is a terminal device different from the first terminal device and the second terminal device, and is any terminal device logged in to the same virtual space V as the first terminal device and the second terminal device.

[0014] The virtual space control system 1 includes a terminal device 10 and a server device 40. The terminal device 10 is used by each user in business, daily life, etc. to log in to the virtual space V.

[0015] FIG. 2 is a block diagram showing a configuration example of the terminal device. The terminal device 10 is an information terminal device such as a personal computer, a smartphone, a tablet terminal, or a wearable device such as a head-up display. The terminal device 10 is connected to the server device 40 via a network so as to be capable of data communication.

[0016] The terminal device 10 includes a camera 11, a microphone 12, an operation unit 13, a display unit 17, an audio output unit 18, a communication unit 19, and a control unit 20.

[0017] The camera 11 can image, for example, the user using the terminal device 10 and the surroundings of the user. The camera 11 outputs the captured video to the video acquisition unit 21 of the control unit 20.

[0018] The microphone 12 picks up the sound around the terminal device 10. The microphone 12 picks up the sound in the direction facing the display unit 17 of the terminal device 10. The microphone 12 may be provided in a headset or the like connected to the terminal device 10. The microphone 12 picks up the voice spoken by the user using the terminal device 10 and the surrounding sounds. The microphone 12, for example, picks up the voices of the first user and the second user. The microphone 12 outputs information regarding the picked-up sound to the audio acquisition unit 22 of the control unit 20.

[0019] The operation unit 13 is an input device capable of inputting various operations on the terminal device 10. The operation unit 13 outputs an operation signal corresponding to the received input operation to the operation reception unit 23 of the control unit 20. The operation unit 13 includes, for example, a keyboard, a mouse, a touch panel arranged over the display unit 17, buttons, switches, and the like. The operation unit 13 can receive, for example, a key input operation, a click operation, or a touch operation. The operation unit 13 may include the microphone 12. The operation unit 13 picks up the voice regarding the operation from the microphone 12 and outputs it to the operation reception unit 23.

[0020] The operation unit 13 can receive any operation, such as an operation on the avatar corresponding to the user himself / herself in the virtual space V, on the display screen of the display unit 17.

[0021] The display unit 17 displays various types of information, including text and images. The display unit 17 is a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 17 displays, for example, the avatars of each user in the virtual space V. The display of the display unit 17 is controlled by the display control unit 29 of the control unit 20.

[0022] The audio output unit 18 is a device that outputs various sounds from the terminal device 10. The audio output unit 18 is implemented, for example, by a speaker. When a user is using the virtual space V, for example, the audio output unit 18 outputs sounds from around the user's avatar in the virtual space V. The audio output of the audio output unit 18 is controlled by the audio output control unit 30 of the control unit 20.

[0023] The communication unit 19 is a communication unit for wired or wireless communication. The communication unit 19 communicates using communication methods such as Wi-Fi (registered trademark) or mobile phone lines, and connects to an intranet or the internet. The communication unit 19 is controlled by the communication control unit 31. The communication unit 19 communicates video and audio, or data such as application operation information, with the server device 40. The communication unit 19 communicates data such as user account information and application operating status information with the server device 40.

[0024] The control unit 20 controls each part of the terminal device 10. The control unit 20 includes, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory). The control unit 20 executes a program that controls the operation of the terminal device 10 according to this disclosure. The control unit 20 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 20 loads the stored program into memory and executes the instructions contained in the program. The control unit 20 includes internal memory such as the aforementioned RAM, and the internal memory is used for temporary storage of data in the control unit. The control unit 20 may be implemented by a combination of hardware and software.

[0025] The control unit 20 includes, as functional blocks realized by circuits and executed programs, a video acquisition unit 21, an audio acquisition unit 22, an operation reception unit 23, a location information detection unit 24, a conversation processing unit 25, a storage unit 26, an application control unit 27, a display control unit 29, an audio output control unit 30, and a communication control unit 31.

[0026] The video acquisition unit 21 acquires the video captured by the camera 11. More specifically, the video acquisition unit 21 acquires video of users using the terminal device 10, etc., captured by the camera 11. The video data acquired by the camera 11 is a moving image. The video acquisition unit 21 may also acquire video captured by an external camera 60 (see Figure 1) different from the camera 11.

[0027] The voice acquisition unit 22 acquires the sound detected by the microphone 12. The voice acquisition unit 22 acquires the sound of users speaking in the vicinity of the terminal device 10. For example, the voice acquisition unit 22 acquires the voices of a first user and a second user. The voice acquisition unit 22 may also acquire the sound detected by an external microphone 70 (see Figure 1) different from the microphone 12.

[0028] The operation reception unit 23 acquires an operation signal from the operation unit 13 corresponding to the operation performed on the operation unit 13. The operation reception unit 23 outputs a control signal corresponding to the acquired operation signal to each unit.

[0029] The operation reception unit 23 receives operations on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to move the user's own avatar in the virtual space V on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to overlay or place the user's own avatar in the virtual space V on another person's avatar on the display screen of the display unit 17.

[0030] The location information detection unit 24 detects the real-world location information of the user using the terminal device 10. The real-world location information of the user using the terminal device 10 may also be the real-world location information of the terminal device 10.

[0031] The location information detection unit 24 can acquire location information of each terminal device within the real-world office using various systems capable of measuring location with high accuracy indoors, such as RTLS (Real Time Location System). In RTLS, for example, each terminal device is equipped with a tag, and communication is performed using UWB (Ultra Wide Band), and the location information of the tag is measured using the TDOA (Time Difference of Arrival) method.

[0032] The conversation processing unit 25 detects user conversations. The conversation processing unit 25 acquires audio from the audio acquisition unit 22 and detects whether or not it contains conversation. If the audio contains conversation, the conversation processing unit 25 detects that conversation. For example, the conversation processing unit 25 can detect a conversation between a first user and a second user in real space. The conversation processing unit 25 may also detect a conversation by detecting that the audio acquired from the audio acquisition unit 22 contains the voices of multiple speakers alternately. The audio acquisition unit 22 may acquire audio A acquired from terminal device 10A and audio B acquired from terminal device 10B separately. The conversation processing unit 25 may also detect a conversation by comparing audio A and audio B acquired from the audio acquisition unit 22 and detecting that audio B is included in audio A.

[0033] For example, the conversation processing unit 25 detects the user's speaking style, tone of voice, and phrases and words used from the acquired audio, and recognizes the content of the conversation based on the detection results. The conversation processing unit 25 can, for example, recognize whether the user is speaking formally or casually, whether the conversation is work-related or personal, etc. The conversation processing unit 25 may also include a machine learning model that has been machine-trained to associate detected conversations with recognized conversation content. This machine learning model can be configured to output the content of a conversation when a detected conversation is input. The conversation processing unit 25 may also use well-known techniques such as speaker dialization, which estimates who is speaking from the audio, to detect if a conversation is present when multiple speakers are estimated from the audio, or it may recognize the content of the conversation for each of the multiple speakers.

[0034] The memory unit 26 stores information such as the calculation contents of the control unit 20 and programs. The memory unit 26 is composed of, for example, RAM, a main memory such as ROM, and a storage device such as an SSD (Solid State Drive) or HDD (Hard Disk Drive).

[0035] The memory unit 26 stores programs and configuration information related to applications used in the terminal device 10, such as applications for the virtual space V. The memory unit 26 also pre-stores the types of conversations classified by the conversation processing unit 25, correspondence information (described later), and machine learning models that recognize conversation content.

[0036] The application control unit 27 controls the operation of various applications running on the terminal device 10. The applications running on the application control unit 27 may cooperate with the application control unit 44 of the server device 40, which will be described later, to control various applications.

[0037] The application control unit 27 causes the display unit 17 to display information related to the application running on the terminal device 10, via the display control unit 29, which will be described later. Application-related displays include display screens such as windows and icons that show information related to the application to the user of the terminal device 10.

[0038] When a user of terminal device 10 is logged into virtual space V, the application control unit 27 makes the virtual space V, which is running in the application control unit 44 of server device 40, available to terminal device 10.

[0039] The display control unit 29 controls the display of various types of information on the display unit 17 of the terminal device 10. The display control unit 29 also controls the display of the video of the virtual space V on the display unit 17.

[0040] The audio output control unit 30 controls the output of audio from the audio output unit 18. The audio output control unit 30 causes the audio output unit 18 to output speech such as conversations within the virtual space V.

[0041] The communication control unit 31 controls the communication unit 19 to control the transmission and reception of data such as video and audio, and the transmission and reception of application operation status information between the communication control unit 31 and the server device 40. More specifically, the communication control unit 31 controls the communication unit 19 to receive video and audio of the virtual space V from the server device 40. The communication control unit 31 controls the communication unit 19 to transmit to the server device 40 the video and audio of the user facing the display unit 17 of the terminal device 10, which have been acquired by the video acquisition unit 21 and the audio acquisition unit 22. Application operation status information refers to information such as the operation of the avatar by the operation unit 13 in a virtual space application that realizes the virtual space control system 1. The communication control unit 31 also transmits information indicating the location information detected by the location information detection unit 24 to the server device 40.

[0042] Note that the processing performed by the control unit 20 may be performed by a control device different from the one used for the terminal device 10.

[0043] Figure 3 is a block diagram showing an example of a server device configuration. The server device 40 provides services that use the virtual space V. The server device 40 may be a general-purpose server device, including a distributed type. The server device 40 constructs and controls the virtual space V, etc. The server device 40 comprises a communication unit 41 and a server control unit 42.

[0044] The communication unit 41 is a communication unit for wired or wireless communication. The communication unit 41 communicates using methods such as Wi-Fi or mobile phone lines, and connects to an intranet or the internet. The communication unit 41 is controlled by the communication control unit 43. The communication unit 41 communicates data such as video, audio, and application operating status information with multiple terminal devices 10.

[0045] The server control unit 42 controls various parts of the server device 40. The server control unit 42 includes, for example, a control device such as a CPU or MPU and a storage device such as RAM or ROM. The server control unit 42 may be implemented by an integrated circuit such as an ASIC or FPGA. The server control unit 42 may be implemented by a combination of hardware and software. The server control unit 42 executes a program that controls the operation of the server device 40 according to this disclosure.

[0046] The server control unit 42 includes a communication control unit 43, an application control unit 44, and a storage unit 45 as functional blocks realized by circuits and programs.

[0047] The communication control unit 43 controls the communication unit 41 to control the transmission and reception of data, including video and audio, and the transmission and reception of application operation status information between the terminal device 10 and the communication control unit 43. For example, the communication control unit 43 controls the communication unit 41 to receive video captured by the camera 11 on the terminal device 10 or an external camera 60, and audio picked up by the microphone 12 or an external microphone 70 from the terminal device 10. For example, the communication control unit 43 controls the communication unit 41 to transmit video and audio from the virtual space V to the terminal device 10. Application operation status information includes, for example, information that constitutes the virtual space application.

[0048] The application control unit 44 controls the operation of various applications running on the server device 40. For example, the application control unit 44 controls the operation of applications in the virtual space V.

[0049] The memory unit 45 stores information such as the calculations performed by the server control unit 42 and programs. The memory unit 45 is composed of, for example, RAM and a main memory device such as ROM, an SSD, or an HDD.

[0050] The memory unit 45 stores information that constitutes the virtual space V. In this embodiment, the memory unit 45 stores information indicating the avatar, the terminal device 10 associated with the avatar, and the account user of the terminal device 10 for each user who logs into the virtual space V. For example, the memory unit 45 stores information indicating the arrangement of objects that constitute the virtual space V, such as partitions and desks.

[0051] Furthermore, the virtual space control system 1 includes a control device 50. The control device 50 is a conceptual device realized by at least one of the multiple terminal devices 10 and server devices 40 that constitute the virtual space control system 1. In other words, at least one of the functions of the control device 50 described below may be realized in the terminal devices 10 (e.g., storage unit 26, application control unit 27, etc.), or at least one may be realized in the server device 40 (e.g., storage unit 45, application control unit 44, etc.).

[0052] Figure 4 is a functional block diagram showing an example of the control device 50. As shown in Figure 4, the control device 50 includes an avatar control unit 51, a conversation processing unit 52, a surrounding area control unit 53, a display control unit 54, and a storage unit 55.

[0053] The avatar control unit 51 places the avatars of users using the terminal device 10 in the real world into the virtual world. The avatar control unit 51 controls the state of the avatars in the virtual world, such as their placement, posture, and movement. For example, the avatar control unit 51 places the first avatar of the first user using the first terminal device 10 in the real world and the second avatar of the second user using the second terminal device 10 in the real world into the virtual world V. The avatar control unit 51 can change the distance between multiple avatars in the virtual world. If a conversation between the first user and the second user in the real world is detected while the first and second avatars are placed in the virtual world V, the avatar control unit 51 changes the distance between the first and second avatars from the distance before the conversation was detected. Correspondence information T, which associates the type of conversation with the distance between the first avatar A1 and the second avatar A2, can be stored in the storage unit 55 in advance. In this state, when a conversation between the first user and the second user in the real world is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, the avatar control unit 51 detects the type of conversation based on the processing result of the conversation processing unit 52, determines the interval corresponding to the detected type of conversation based on the corresponding information, and changes the interval between the first avatar and the second avatar to the determined interval. The detailed functions of the avatar control unit 51 will be described later.

[0054] The conversation processing unit 52 determines whether multiple users using the terminal device 10 in the real world are having a conversation with each other in the real world. If the conversation processing unit 52 determines that the multiple users are having a conversation with each other in the real world, it determines, based on the content of the conversation, whether a third party other than the multiple users is included in the conversation.

[0055] The Enclosure Area Control Unit 53 controls the state of the virtual space. The Enclosure Area Control Unit 53 forms an enclosure area in the virtual space that surrounds multiple avatars. When a conversation between the first user and the second user in the real space is detected while the first avatar and the second avatar are placed in the virtual space V, the Enclosure Area Control Unit 53 generates an enclosure area AR in the virtual space V that surrounds the first avatar and the second avatar. Furthermore, when a third user related to the content of the conversation between the first user and the second user is detected, and the third user's third avatar is detected to be placed in the virtual space V, the Enclosure Area Control Unit 53 places the third avatar near the enclosure area and changes the enclosure area so that at least a part of the enclosure area moves closer to the third avatar. Detailed functions of the Enclosure Area Control Unit 53 and details of the enclosure area will be described later.

[0056] The display control unit 54 displays the processing results from the avatar control unit 51 and the enclosed area control unit 53 on the display unit 17 of the terminal device 10.

[0057] The memory unit 55 stores data and programs for executing each operation of the control device 50. The memory unit 55 can store correspondence information that associates the type of conversation between the first user and the second user with the distance between the first avatar and the second avatar.

[0058] The memory unit 55 stores a virtual space control program that causes the computer to execute the following processes: placing the first avatar A1 of the first user using the first terminal device 10 in real space and the second avatar A2 of the second user using the second terminal device 10 in real space into the virtual space V; generating a surrounding area AR in the virtual space V when a conversation between the first user and the second user in real space is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V; and displaying the virtual space V on the display units of terminal devices including the first terminal device 10 and the second terminal device 10.

[0059] Furthermore, the memory unit 55 stores a virtual space control program that causes the computer to execute the following processes: placing the first avatar A1 of the first user using the first terminal device 10 in the real world and the second avatar A2 of the second user using the second terminal device 10 in the real world into the virtual space V; changing the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected when a conversation between the first user and the second user in the real world is detected while the first avatar A1 and the second avatar A2 are placed into the virtual space V; and displaying the virtual space V on the display units of the terminal devices including the first terminal device 10 and the second terminal device 10.

[0060] Next, a virtual space control method according to this embodiment will be described. Figure 5 is a flowchart showing an example of a virtual space control method. Figures 6 to 9 show examples of virtual spaces displayed on the display unit 17 of the terminal device 10.

[0061] When a user accesses the server device 40 using the terminal device 10 and performs an operation via the operation unit 13 to log in to the virtual space V, the details of that operation are transmitted from the communication unit 19 to the server device 40. Based on the information transmitted from the terminal device 10, the server device 40 performs the user login operation via the application control unit 44.

[0062] In the control device 50, the avatar control unit 51 places the avatar of the user who has logged into the virtual space V into the virtual space V. The following explanation will take as an example the case in which a first user using one terminal device (hereinafter referred to as the first terminal device) and a second user using another terminal device (hereinafter referred to as the second terminal device) have logged into the virtual space V. In this case, as shown in Figure 6, the avatar control unit 51 places the first avatar A1 of the first user and the second avatar A2 of the second user into the virtual space V (step S10).

[0063] The conversation processing unit 52 determines whether a conversation between the first user and the second user in the real space has been detected while the first avatar A1 and the second avatar A2 are positioned in the virtual space V (step S20). In step S20, the conversation processing unit 52 can determine that the first user and the second user are having a conversation in the real space in cases such as when, for example, video of the first user and the second user having a conversation is captured by at least one of the cameras 11 of the first user's first terminal device 10, the camera 11 of the second user's second terminal device 10, and the external camera 60, or when the microphone 12 of the first user's first terminal device 10 and the microphone 12 of the second user's second terminal device 10 are picking up the same conversation and the first terminal device 10 and the second terminal device 10 are positioned at a distance less than a predetermined threshold. On the other hand, if it is determined in step S20 that no conversation between the first user and the second user is detected (No. in step S20), steps S30 through S70 are skipped and the process proceeds to step S80.

[0064] On the other hand, if a conversation between the first user and the second user is detected in step S20 (Yes in step S20), the surrounding area control unit 53 generates a surrounding area AR in the virtual space V that surrounds the first avatar A1 and the second avatar A2, as shown in Figure 7 (step S30). As shown in Figure 7, the avatar control unit 51 can change the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected when the surrounding area AR is generated (step S40). For example, when the surrounding area AR is generated, the avatar control unit 51 may make the distance between the first avatar A1 and the second avatar A2 closer to the distance before the conversation was detected. In this case, the avatar control unit 51 may also change other states of the first avatar A1 and the second avatar A2 in the virtual space V, such as their position, posture, and size, after changing the distance. Steps S30 and S40 may be performed in reverse order. In other words, the control device 50 may change the distance between the first avatar A1 and the second avatar A2, and then form the enclosed area AR.

[0065] In the example shown in Figure 7, the surrounding area control unit 53 sets the boundary line L of the surrounding area AR to a circular shape, but is not limited to this configuration, and may use other shapes such as an ellipse, cloud shape, heart shape, polygon shape, etc. Also, the surrounding area AR is not limited to a single shape, but may be a shape that is a combination of multiple shapes. Furthermore, the surrounding area AR may be colored in a different color from other areas of the virtual space V. The surrounding area control unit 53 may also form a window that is different from other areas in the virtual space V and use that window as the surrounding area AR. Instead of displaying the boundary line L of the surrounding area AR that encloses the first avatar A1 and the second avatar A2, the surrounding area control unit 53 may display a line connecting the avatars, or it may color the avatars the same color and display them.

[0066] After the enclosed area AR is formed, the conversation processing unit 52 determines whether or not it contains content about a third user related to the content of the conversation between the first user and the second user (step S50). In step S50, the conversation processing unit 52 can determine whether or not it contains content about a third user based, for example, on the content of the conversation recognized by the conversation processing unit 25 of the terminal device 10.

[0067] If it is determined in step S50 that the information for the third user is included (Yes in step S50), the avatar control unit 51 determines whether or not the third user's third avatar is placed in the virtual space V (step S60). If the third user is logged into the virtual space V via the terminal device 10, the avatar control unit 51 places the third user's third avatar A3 in the virtual space V, as shown in Figure 8. On the other hand, if it is determined in step S50 that the information for the third user is not included (No in step S50), steps S60 to S70 are skipped, and the process proceeds to step S80.

[0068] In step S60, if it is determined that the third user's third avatar A3 is located in the virtual space V (Yes in step S60), the avatar control unit 51 places the third avatar A3 in the vicinity of the enclosed area AR in the virtual space V, as shown in Figure 9. The enclosed area control unit 53 also changes the enclosed area AR so that at least a part of the enclosed area AR moves closer to the third avatar A3 (step S70). In step S70, the enclosed area control unit 53 can change the enclosed area AR by extending a part of the boundary line L so that the boundary line L of the enclosed area AR moves toward the third avatar A3, for example, as shown in Figure 9. The enclosed area control unit 53 is not limited to the example shown in Figure 9, and may change a part of the enclosed area AR to move toward the third avatar A3 in other ways. The enclosed area control unit 53 may also move the point on the boundary line L of the enclosed area AR that is at the minimum distance from the third avatar A3 toward the third avatar A3. The enclosed area control unit 53 may select any position within the area that the third avatar A3 is displayed in, and use the selected position as the position of the third avatar A3 to calculate the distance between a point on the boundary line L and the third avatar A3. The enclosed area control unit 53 may move points on the boundary line L that are at a position where the distance to the third avatar A3 is less than a predetermined value toward the third avatar A3. The enclosed area control unit 53 may reduce the distance between a point on the boundary line L and the third avatar A3. The enclosed area control unit 53 may move multiple points on the boundary line L that are at a position where the distance to the third avatar A3 is less than a predetermined value toward the third avatar A3. For multiple points on the boundary line L that are at a position where the distance to the third avatar A3 is less than a predetermined value, the enclosed area control unit 53 may move them toward the third avatar A3 as the distance decreases. On the other hand, if it is determined in step S60 that the third user's third avatar A3 is not placed in virtual space V (No. in step S60), then step S70 is skipped and the process proceeds to step S80.

[0069] In step S30, a surrounding area AR is formed in the virtual space V, in step S40, the distance between the first avatar A1 and the second avatar A2 is changed, or at least a part of the surrounding area AR is changed to move closer to the third avatar A3. As a result of each process, the display control unit 54 displays the result on the display unit 17 of the terminal device 10 (step S80).

[0070] Figures 10 to 12 show other examples of changing the distance between the first avatar A1 and the second avatar A2 in step S40. Figure 10 shows an example of the correspondence between the type of conversation between the first user and the second user and the distance between the first avatar A1 and the second avatar A2. As shown in Figure 10, the storage unit 55 stores correspondence information T that associates the type of conversation between the first user and the second user with the distance between the first avatar A1 and the second avatar A2. Based on this correspondence information T, the avatar control unit 51 can set the distance between the first avatar A1 and the second avatar A2 according to the type of conversation between the first user and the second user.

[0071] For example, as shown in Figure 11, if the conversation is business-related, the avatar control unit 51 can set the distance between the first avatar A1 and the second avatar A2 to the first distance D1. Also, as shown in Figure 12, if the conversation is private, the avatar control unit 51 can set the distance between the first avatar A1 and the second avatar A2 to the second distance D2. In this way, by changing the distance between avatars in the virtual space V based on the type of conversation between users in the real space, it becomes easier for a third party who sees the distance between avatars in the virtual space V to understand the type of conversation. The correspondence information T may set the distance D between avatars to be the same for each type of conversation, or it may be different. For example, as shown in Figure 12, the distance D2 when the conversation is private may be wider than the distance D1 when the conversation is business-related. Alternatively, the distance D2 when the conversation is private may be narrower than the distance D1 when the conversation is business-related. The correspondence information T may include the color of the enclosed area AR, the color of the boundary line L of the enclosed area AR, the thickness of the boundary line L, the type of the boundary line L, etc., or any of these may be stored in the storage unit 55 in association with the type of conversation. The enclosed area control unit 53 may set the enclosed area AR and the boundary line L of the enclosed area AR according to the type of conversation based on the correspondence information T.

[0072] As described above, the virtual space control system 1 according to this embodiment includes a control device 50 that places a first avatar A1 of a first user using a first terminal device 10 in real space and a second avatar A2 of a second user using a second terminal device 10 in real space in virtual space V, and when a conversation between the first user and the second user in real space is detected while the first avatar A1 and the second avatar A2 are placed in virtual space V, generates a surrounding area AR in virtual space V that encloses the first avatar A1 and the second avatar A2, and displays virtual space V on the display units of terminal devices including the first terminal device 10 and the second terminal device 10.

[0073] The virtual space control method according to this embodiment includes the steps of: placing a first avatar A1 of a first user using a first terminal device 10 in real space and a second avatar A2 of a second user using a second terminal device 10 in real space into a virtual space V; when a conversation between the first user and the second user in real space is detected while the first avatar A1 and the second avatar A2 are placed into the virtual space V, generating a surrounding area AR in the virtual space V that encloses the first avatar A1 and the second avatar A2; and having the control device 50 display the virtual space V on the display unit of the terminal device including the first terminal device 10 and the second terminal device 10.

[0074] The virtual space control program according to this embodiment causes the computer to execute the following processes: placing a first avatar A1 of a first user using the first terminal device 10 in the real world and a second avatar A2 of a second user using the second terminal device 10 in the real world into a virtual space V; generating a surrounding area AR in the virtual space V when a conversation between the first user and the second user in the real world is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V; and displaying the virtual space V on the display unit of the terminal device including the first terminal device 10 and the second terminal device 10.

[0075] According to this configuration, when a conversation between a first user and a second user in the real world is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, a surrounding area AR is generated in the virtual space V, and this surrounding area AR is displayed on the display unit 17 of the terminal device 10. This makes it easier for a third party to understand that the first user corresponding to the first avatar A1 and the second user corresponding to the second avatar A2 are having a conversation, surrounded by the surrounding area AR in the virtual space V.

[0076] In the virtual space control program 1 according to this embodiment, when the control device 50 detects a third user related to the content of the conversation and detects that the third user's third avatar A3 is located in the virtual space V, it places the third avatar A3 near the surrounding area AR and changes the surrounding area AR so that at least a part of the surrounding area AR is closer to the third avatar A3.

[0077] This configuration allows a third user, corresponding to the third avatar A3, to easily understand that they are involved in the conversation between the first and second users. This encourages the third user to join the conversation.

[0078] The memory unit 55 may pre-store information about the third user, such as the third user's name, department, and job duties. The conversation processing unit 52 may detect a third user related to the content of the conversation by determining whether or not the third user information stored in the memory unit 55 is included in the conversation.

[0079] When the conversation processing unit 52 detects a conversation between the first user and the second user, the enclosed area control unit 53 may calculate the midpoint of the distance between the first avatar A1 and the second avatar A2, calculate the radius surrounding the first avatar A1 and the second avatar A2 with respect to that midpoint, and generate a circular enclosed area AR with that radius.

[0080] The avatar control unit 51 may position the third avatar A3 at a predetermined distance from the surrounding region AR. The avatar control unit 51 may also calculate the region near the surrounding region AR as the region inside a circle with a radius approximately twice the radius calculated when generating the surrounding region AR.

[0081] Without the avatar control unit 51 positioning the third avatar A3 near the surrounding area AR, the surrounding area control unit 53 may change the surrounding area AR so that at least a part of the surrounding area AR approaches the third avatar A3. The avatar control unit 51 may position the third avatar A3 at a predetermined distance from the first avatar A1, or at a predetermined distance from the second avatar A2. When the avatar control unit 51 detects a third user related to the content of a conversation, if it detects the third user from the voice of the first user, it may position the third avatar A3 at a predetermined distance from the first avatar A1, and if it detects the third user from the voice of the second user, it may position the third avatar A3 at a predetermined distance from the second avatar A2.

[0082] The control device 50 may send a notification to the third user prompting them to log in to virtual space V if it is not detected that the third user related to the content of the conversation is located in virtual space V. The control device 50 may also automatically log the third user into virtual space V and send a notification to the third user indicating that they have logged in if it is not detected that the third user related to the content of the conversation is located in virtual space V. In this case, the control device 50 may detect that the third user has logged into virtual space V in the past.

[0083] In the virtual space control program 1 according to this embodiment, when a conversation between a first user and a second user in the real space is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, the control device 50 changes the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected.

[0084] With this configuration, by changing the distance between the first avatar A1 and the second avatar A2, it becomes easier for a third party to understand that the first user corresponding to the first avatar A1 and the second user corresponding to the second avatar A2 are having a conversation.

[0085] The avatar control unit 51 may change the distance between the first avatar A1 and the second avatar A2 by moving only the other avatar while leaving the other avatar still. The avatar control unit 51 may also change the distance between the first avatar A1 and the second avatar A2 by calculating the midpoint of the distance between the two avatars and moving the avatars around that midpoint. In other words, the avatar control unit 51 may change the distance between the avatars by moving the avatars to a predetermined distance from the calculated midpoint. The avatar control unit 51 may also use a predetermined position that is within a predetermined distance from either the first avatar A1 or the second avatar A2 instead of the aforementioned midpoint.

[0086] Furthermore, the virtual space control system 1 according to this embodiment includes a control device 50 that places a first avatar A1 of a first user using a first terminal device 10 in real space and a second avatar A2 of a second user using a second terminal device 10 in real space in a virtual space V, and when a conversation between the first user and the second user in real space is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, the control device 50 changes the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected and displays the virtual space V on the display unit of the terminal device including the first terminal device 10 and the second terminal device 10.

[0087] The virtual space control method according to this embodiment includes the steps of: placing a first avatar A1 of a first user using a first terminal device 10 in real space and a second avatar A2 of a second user using a second terminal device 10 in real space into a virtual space V; changing the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected when a conversation between the first user and the second user in real space is detected while the first avatar A1 and the second avatar A2 are placed into the virtual space V, and displaying the virtual space V on the display unit of the terminal device including the first terminal device 10 and the second terminal device 10, which are executed by the control device 50.

[0088] The virtual space control program according to this embodiment causes the computer to execute the following processes: placing a first avatar A1 of a first user using the first terminal device 10 in the real world and a second avatar A2 of a second user using the second terminal device 10 in the real world into a virtual space V; changing the distance between the first avatar A1 and the second avatar A2 from the distance before the conversation was detected when a conversation between the first user and the second user in the real world is detected while the first avatar A1 and the second avatar A2 are placed into the virtual space V; and displaying the virtual space V on the display unit of the terminal device including the first terminal device 10 and the second terminal device 10.

[0089] With these configurations, by changing the distance between the first avatar A1 and the second avatar A2, it becomes easier for a third party to understand that the first user corresponding to the first avatar A1 and the second user corresponding to the second avatar A2 are having a conversation.

[0090] In the virtual space control program 1 according to this embodiment, the control device 50 stores correspondence information T in advance, which associates the type of conversation with the distance between the first avatar A1 and the second avatar A2. When a conversation between the first user and the second user in the real space is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, the control device 50 detects the type of conversation, determines the distance corresponding to the detected type of conversation based on the correspondence information, and changes the distance between the first avatar A1 and the second avatar A2 to the determined distance.

[0091] According to this configuration, based on the corresponding information T, the distance between the first avatar A1 and the second avatar A2 can be set according to the content of the conversation between the first and second users. As a result, third parties not included in the conversation can understand the type of conversation, which can encourage their participation compared to when they cannot understand the type of conversation.

[0092] In the virtual space control program 1 according to this embodiment, when a conversation between a first user and a second user in the real space is detected while the first avatar A1 and the second avatar A2 are placed in the virtual space V, the control device 50 generates a surrounding area AR in the virtual space V that encloses the first avatar A1 and the second avatar A2.

[0093] This configuration makes it easier for a third party to understand that a first user, corresponding to a first avatar A1, and a second user, corresponding to a second avatar A2, are conversing within the enclosed area AR of the virtual space V.

[0094] Although embodiments of the present invention have been described above, the embodiments are not limited by the content of these embodiments. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of Symbols]

[0095] A1…First Avatar, A2…Second Avatar, A3…Third Avatar, AR…Enclosed Area, L…Boundary Line, T…Corresponding Information, V…Virtual Space, 1…Virtual Space Control System, 10…Terminal Device, 11,60…Camera, 12,70…Microphone, 13…Operation Unit, 17…Display Unit, 18…Audio Output Unit, 19,41…Communication Unit, 20…Control Unit, 21…Video Acquisition Unit, 22…Audio Acquisition Unit, 23…Operation Reception Unit, 24…Location Information Detection Unit, 25,52…Conversation Processing Unit, 26,45,55…Storage Unit, 27,44…Application Control Unit, 29,54…Display Control Unit, 30…Audio Output Control Unit, 31,43…Communication Control Unit, 40…Server Device, 42…Server Control Unit, 50…Control Device, 51…Avatar Control Unit, 53…Enclosed Area Control Unit

Claims

1. The first avatar of a first user using a first terminal device in the real world and the second avatar of a second user using a second terminal device in the same real world are placed in a virtual space. When a conversation between the first user and the second user in the real world is detected while the first avatar and the second avatar are placed in the virtual space, a surrounding region is generated in the virtual space that encloses the first avatar and the second avatar. The system includes a control device that causes the virtual space to be displayed on the display unit of a terminal device, including the first terminal device and the second terminal device. Virtual space control system.

2. The control unit, If a third user related to the content of the conversation is detected, and it is detected that the third user's third avatar is placed in the virtual space, the third avatar is placed at a predetermined distance from the enclosed area. The enclosed area is modified so that at least a portion of the enclosed area approaches the third avatar. The virtual space control system according to claim 1.

3. The control unit, When a conversation between the first user and the second user in the real world is detected while the first avatar and the second avatar are positioned in the virtual space, the distance between the first avatar and the second avatar is changed from the distance before the conversation was detected. The virtual space control system according to claim 1.

4. The steps include: placing a first avatar of a first user using a first terminal device in the real world and a second avatar of a second user using a second terminal device in the real world into a virtual space; When a conversation between the first user and the second user in the real space is detected while the first avatar and the second avatar are placed in the virtual space, the following steps are taken: a surrounding region is generated in the virtual space that encloses the first avatar and the second avatar; The steps include: displaying the virtual space on the display unit of a terminal device including the first terminal device and the second terminal device; A virtual space control method executed by a control device.

5. A process of placing a first avatar of a first user using a first terminal device in the real world and a second avatar of a second user using a second terminal device in the same real world into a virtual space, When a conversation between the first user and the second user in the real space is detected while the first avatar and the second avatar are placed in the virtual space, the process of generating a surrounding region in the virtual space that encloses the first avatar and the second avatar, A process of displaying the virtual space on the display unit of a terminal device including the first terminal device and the second terminal device. A virtual space control program that causes a computer to execute something.

Images