Virtual space control system and virtual space control method

Abstract

To facilitate communication among participants in a virtual space. [Solution] The information processing device 50 of the virtual space control system 1 acquires user information for each user of multiple avatars contained within a predetermined space in the virtual space, detects that any avatar among the multiple avatars is isolated, and when an isolated first avatar is detected, searches for a second user that has commonalities with the user information of the first user from the user information of users other than the first user who is the user of the first avatar, places a dummy avatar near the first avatar, and has the dummy avatar perform actions that facilitate communication between the first user and the second user by communicating with the first user based on the user information of the first user.

Description

Technical Field

[0006] , , ,

[0001] The present invention relates to a virtual space control system and a virtual space control method.

Background Art

[0002] Events using virtual spaces are being held. Participants in events in virtual spaces participate in the events using their own avatars. A technique for participating in a gathering held in a virtual space using an avatar has been disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when such an event is an event in which an unspecified number of participants participate, it may be difficult for some participants to create an opportunity to communicate with other participants.

[0005] The present disclosure aims to promote communication between participants in a virtual space.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the virtual space control system according to this disclosure is a virtual space control system that causes an information terminal device to display a virtual space and displays an avatar corresponding to the user of the information terminal device in the virtual space, wherein the information processing device provided in the virtual space control system acquires user information for each user of a plurality of avatars contained in a predetermined space in the virtual space, detects that any avatar among the plurality of avatars is isolated among the plurality of avatars, and when an isolated first avatar is detected, searches for a second user that has commonalities with the user information of the first user from the user information of users other than the first user who is the user of the first avatar, places a dummy avatar near the first avatar, and causes the dummy avatar to perform actions that facilitate communication between the first user and the second user by communicating with the first user based on the user information of the first user.

[0007] The virtual space control method relating to this disclosure is a virtual space control method executed by a virtual space control system that causes an information terminal device to display a virtual space and displays an avatar corresponding to the user of the information terminal device in the virtual space, wherein the virtual space control system acquires user information for each user of a plurality of avatars contained in a predetermined space in the virtual space, detects that any avatar among the plurality of avatars is isolated among the plurality of avatars, and when an isolated first avatar is detected, searches for a second user that has commonalities with the user information of the first user from the user information of users other than the first user who is the user of the first avatar, places a dummy avatar near the first avatar, and has the dummy avatar perform actions that communicate with the first user based on the user information of the first user and facilitate communication between the first user and the second user. [Effects of the Invention]

[0008] According to this disclosure, it is possible to facilitate communication among participants in a virtual space. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example configuration of a virtual space control system according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an information terminal device according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of a server device according to the first embodiment. [Figure 4] Figure 4 is a block diagram showing an example configuration of an information processing device according to the first embodiment. [Figure 5] Figure 5 is a block diagram showing an example of the configuration of the detection unit according to the first embodiment. [Figure 6] Figure 6 is a flowchart showing an example of a virtual space control method according to the first embodiment. [Figure 7] Figure 7 shows an example of a virtual space display screen shown on the display screen of an information terminal device. [Figure 8] Figure 8 shows another example of a virtual space display screen displayed on the display screen of an information terminal device. [Figure 9] Figure 9 shows another example of a virtual space display screen displayed on the display screen of an information terminal device. [Figure 10] Figure 10 is a flowchart showing an example of a virtual space control method according to the second embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the virtual space control system and virtual space control method according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0011] [First Embodiment] (Virtual Space Control System) Figure 1 is a schematic diagram showing an example configuration of a virtual space control system according to the first embodiment. The virtual space control system 1 controls the virtual space. In the following embodiments, a business matching event (hereinafter referred to as "event") held in the virtual space will be used as an example for explanation, but it is not limited to this and can be applied to various services such as other events using avatars in the virtual space where participants communicate with each other.

[0012] In virtual business matching events and other similar events, participants communicate with each other at their own discretion. Specifically, participants find someone they want to communicate with and communicate through avatars.

[0013] In this embodiment, when the virtual space control system 1 detects a first avatar that is isolated among the avatars corresponding to users who are participants in an event in the virtual space, it places the first avatar near the first user via a dummy avatar. The virtual space control system 1 then facilitates communication between the first user and a second user who shares user information with the first user via the dummy avatar.

[0014] In the following explanation, a user who tends to be isolated will be referred to as the "first user," and a user whose user information is similar to that of the first user will be referred to as the "second user." Furthermore, the information terminal device used by the first user to log into the virtual space will be referred to as the first information terminal device, and the information terminal device used by the second user to log into the virtual space will be referred to as the second information terminal device. Additionally, the avatar corresponding to the first user will be referred to as the first avatar, and the avatar corresponding to the second user will be referred to as the second avatar.

[0015] The state in which communication with other users is possible in the virtual space means that in the virtual space, it is possible to communicate using one's own avatar and the avatars of other users. Specifically, regardless of whether there is communication or not, it is a state where mutual voices can be heard. For example, for the first user and the second user, regardless of whether it is a conversation between the first user and the second user through the first avatar and the second avatar, it is a state where mutual voices can be heard.

[0016] In this embodiment, user information is, for example, information such as that input by each user at the time of participating and registering for an event defined within a predetermined space in the virtual space. User information is, for example, the name of an organization such as the company to which the user belongs, the purpose of participating in the event, the industry in which the user is interested, the name of an organization in which the user is interested, the field in which the user is interested, and so on. User information is information indicating the purpose for which the user participates in the event, and is information that can identify the person with whom the user wants to communicate in that event and the person with whom it is useful to communicate.

[0017] An example of the outline of an event in the virtual space controlled by the virtual space control system 1 will be described. The user performs advance registration for participation in an event held in the virtual space. At the time of participation registration, the user inputs user information. When participating in the event, the user uses the information terminal device 10 to participate in the event held in the virtual space with their own account. In the virtual space, an avatar corresponding to the user is set. The user uses the information terminal device 10, which will be described later, that they use to operate the avatar corresponding to themselves to move within the virtual space and communicate with other users via the avatars. For example, when the distance between the avatars in the virtual space is close, information sharing such as mutual voices can be heard.

[0018] The virtual space control system 1 includes an information terminal device 10 used by each user to log in to the virtual space and a server device 30.

[0019] (Information terminal device) Figure 2 is a block diagram showing an example configuration of an information terminal device according to the first embodiment. The information terminal device 10 is a terminal device used by a user who is a participant in an event in a virtual space. The information terminal device 10 is a terminal device on which the user runs applications in the virtual space. The information terminal device 10 is a terminal device on which the user logs in with their own account when logging into the virtual space.

[0020] The information terminal device 10 is, for example, a general-purpose personal computer, a smartphone, or a tablet device. The information terminal device 10 is connected to the server device 30 via a network in a data communication manner.

[0021] The information terminal device 10 logs into the virtual space based on the account information of the user logged into the information terminal device 10.

[0022] The information terminal device 10 includes a camera 11, a microphone 12, an operation unit 13, a display unit 17, an audio output unit 18, a communication unit 19, and a terminal control device 20.

[0023] Camera 11 captures images in the direction facing the information terminal device 10. Camera 11 is positioned to capture images in the direction facing the display unit 17 of the information terminal device 10. In other words, camera 11 captures images in the direction where the user using the information terminal device 10 is located. Camera 11 is positioned, for example, around the display unit 17. Camera 11 outputs the captured image to the image acquisition unit 21 of the terminal control device 20.

[0024] The microphone 12 picks up sounds from the vicinity of the information terminal device 10. The microphone 12 may also pick up sounds coming from the direction opposite to the display unit 17 of the information terminal device 10, and may be provided on a headset connected to the information terminal device 10. In other words, the microphone 12 picks up the voice spoken by the user using the information terminal device 10 and the surrounding sounds. The microphone 12 outputs audio information related to the picked-up voice to the audio acquisition unit 22 of the terminal control device 20.

[0025] The operation unit 13 is an input device capable of receiving various operations for the information terminal device 10. The operation unit 13 outputs an operation signal to the terminal control device 20 corresponding to the received input operation. The operation unit 13 includes, for example, a keyboard, mouse, touch panel arranged on top of the display unit 17, buttons, switches, etc. The operation unit 13 can receive, for example, key input operations, click operations, or touch operations.

[0026] The operation unit 13 can accept operations such as moving the avatar corresponding to the user in the virtual space in any direction on the display screen of the display unit 17. For example, the operation unit 13 can accept operations such as overlapping the user's own avatar in the virtual space with another avatar or placing it adjacent to another avatar on the display screen of the display unit 17.

[0027] By performing actions such as overlaying one's own avatar on another's or placing them adjacent to each other, users can initiate communication such as video calls or voice calls within a virtual space application. Whether to conduct a video call or a voice call may be pre-configured or can be set by the user.

[0028] The display unit 17 displays various types of images, including text and images. The display unit 17 is a display including, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 17 displays images of a virtual space, etc. In this embodiment, when a video call becomes possible, the display unit 17 outputs the image of the other avatar user making the video call, in other words, the image captured by the camera 11 of the information terminal device 10 used by the other avatar user. The display of the display unit 17 is controlled by the display control unit 27.

[0029] The audio output unit 18 is an audio output device that outputs sounds from the virtual space. In this embodiment, when using the virtual space, the audio output unit 18 outputs sounds from around the user's avatar in the virtual space. Also, when voice or video calls become possible, the audio output unit 18 outputs the voice of the other avatar user making the voice or video call, in other words, the voice picked up by the microphone 12 of the information terminal device 10 used by the avatar user. The audio output of the audio output unit 18 is controlled by the audio output control unit 28.

[0030] The communication unit 19 is a communication unit for wired or wireless communication. The communication unit 19 communicates using communication methods such as Wi-Fi (registered trademark) or mobile phone lines, and connects to an intranet or the internet. The communication unit 19 is controlled by the communication control unit 29. The communication unit 19 communicates data such as video and audio, or application operation information, with the server device 30. The communication unit 19 also communicates data such as user account information and application operating status information of users logged into the information terminal device 10 with the server device 30.

[0031] (Terminal control device) The terminal control device 20 controls various parts of the information terminal device 10. The terminal control device 20 includes, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory). The terminal control device 20 executes a program that controls the operation of the information terminal device 10 according to this disclosure. The terminal control device 20 may be implemented by an integrated circuit such as an ASIC or FPGA. The terminal control device 20 loads the stored program into memory and executes the instructions contained in the program. The terminal control device 20 includes internal memory such as the aforementioned RAM, and the internal memory is used for temporary storage of data in the terminal control device 20. The terminal control device 20 may be implemented by a combination of hardware and software.

[0032] The terminal control device 20 includes, as functional blocks realized by circuits and executed programs, a video acquisition unit 21, an audio acquisition unit 22, an operation reception unit 23, an application control unit 24, a storage unit 26, a display control unit 27, an audio output control unit 28, and a communication control unit 29.

[0033] The video acquisition unit 21 acquires the video captured by the camera 11. More specifically, the video acquisition unit 21 acquires the video captured by the camera 11, which shows a user facing the information terminal device 10. The video data acquired by the camera 11 is a moving image.

[0034] The voice acquisition unit 22 acquires the sound detected by the microphone 12. The voice acquisition unit 22 acquires sounds such as the voice spoken by a user in the vicinity of the information terminal device 10.

[0035] The operation reception unit 23 acquires an operation signal from the operation unit 13 corresponding to the operation performed on the operation unit 13. The operation reception unit 23 outputs a control signal corresponding to the acquired operation signal to each unit.

[0036] The operation reception unit 23 receives operations on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to move the user's avatar in the virtual space on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to overlap or place the user's avatar in the virtual space next to another avatar on the display screen of the display unit 17.

[0037] The application control unit 24 controls the operation of various applications running on the information terminal device 10. The applications running on the application control unit 24 may be controlled in cooperation with the virtual office control unit 42 of the server device 30, which will be described later.

[0038] The application control unit 24 causes the display unit 17 to display information related to the application running on the information terminal device 10, via the display control unit 27, which will be described later. Application-related displays include display screens such as windows and icons that show information related to the application to the user of the information terminal device 10.

[0039] When a user of the information terminal device 10 is logged into the virtual space, the application control unit 24 makes the virtual space running in the virtual office control unit 42 of the server device 30 available to the information terminal device 10.

[0040] When a virtual space application is running on the information terminal device 10, the application control unit 24 performs information sharing using the display control unit 27 and the audio output control unit 28 based on the information sharing range set around the avatar corresponding to the user of the information terminal device 10. In addition, when a virtual space application is running on the information terminal device 10 and a video call is started with an avatar corresponding to another user, the application control unit 24 performs the video call using the display control unit 27 and the audio output control unit 28.

[0041] The memory unit 26 stores information such as the calculation contents and programs of the terminal control device 20. The memory unit 26 is composed of, for example, RAM, a main memory such as ROM, an SSD, an HDD (Hard Disk Drive), and other storage devices.

[0042] The memory unit 26 stores programs and configuration information related to applications used in the information terminal device 10, such as virtual space applications.

[0043] This section describes the information sharing range and information sharing that are set for avatars in a virtual space. In a virtual space, an information sharing range is set around each avatar. The information sharing range is the range in which information can be shared with users of other avatars, in other words, the range in which communication is possible, and it is the range in which information can be shared among multiple users that are mutually included in the information sharing range. One example of information sharing within the information sharing range is the sharing of voices. In other words, the information sharing range is the range in which the voices of users of other avatars can be heard by users of other avatars. Also, the information sharing range is the range in which users of other avatars can hear the voices of users of other avatars. The information that can be shared within the information sharing range is not limited to voice, but also includes other information such as video and text. In this embodiment, the information sharing range will be described as an example of sharing voices.

[0044] For example, if the second avatar enters the information sharing range of the first avatar, the second user of the second avatar can hear the voice of the first user of the first avatar. Also, if the second avatar enters the information sharing range of the first avatar, and the first avatar enters the information sharing range of the second avatar, two-way voice sharing occurs, allowing them to sense each other's presence and enabling communication without any special operation, such as conversation. Therefore, the information sharing range can also be called the voice sharing range. Specifically, when the first and second avatars are mutually included in each other's information sharing range, the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the first user of the first avatar becomes audible on the information terminal device 10 used by the second user, and the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the second user of the second avatar becomes audible on the information terminal device 10 used by the first user.

[0045] The information sharing area for each avatar is initially set to a certain size and shape when the virtual space is first used. The size and shape of the information sharing area for each avatar may be changed by user operation.

[0046] In a virtual space, even if communication is not possible outside the information sharing range, the voices of other avatar users may be heard at a lower volume than the voices of other avatar users within the information sharing range. For example, if a second avatar is located outside the information sharing range of a first avatar, the second user of the second avatar can hear the voice of the first user of the first avatar at a lower volume than if they were located within the information sharing range. In particular, within a designated space such as an event space in a virtual space, voice communication is properly possible between avatars that are mutually included in the information sharing range by hearing voices at an appropriate volume, and the voices of other avatar users outside the information sharing range may be heard at a lower volume than the voices of other avatar users within the information sharing range.

[0047] The display control unit 27 controls the display of various images on the display unit 17. The display control unit 27 controls the display of images in the virtual space on the display unit 17. In this embodiment, if the information being shared includes images or text, the display control unit 27 outputs the images of the virtual space avatar user to the display unit 17. The display control unit 27 may also output to the display unit 17 images of the second user captured by the camera 11 of the second information terminal device 10 used by the second user, who is the other party to the video call.

[0048] The display control unit 27 stops sharing video between the first user and the second user if the video call is stopped.

[0049] In other words, a video call is a form of communication that uses video footage of the user.

[0050] The audio output control unit 28 controls the audio output of the audio output unit 18. When information sharing is taking place, that is, when communication is possible between avatars, the audio output control unit 28 causes the audio output unit 18 to output the speech of other avatar users within the information sharing range in the virtual space. Also, when video calls become possible, the audio output control unit 28 causes the audio output unit 18 to output the speech of the user who will be making the video call.

[0051] The audio output control unit 28 enables the second user to share the first user's voice when the second avatar enters the information sharing range of the first avatar. More specifically, when audio sharing becomes possible, the audio output control unit 28 enables the second user's voice to be heard on the information terminal device 10 used by the first user corresponding to the first avatar. The audio output control unit 28 outputs the second user's voice, picked up by the microphone 12 of the second information terminal device 10, from the audio output unit 18 of the first information terminal device 10.

[0052] The audio output control unit 28 stops audio sharing between the first user and the second user if the first avatar and the second avatar are no longer included in the information sharing range due to avatar movement or other reasons.

[0053] The communication control unit 29 controls the communication unit 19 to control the transmission and reception of data such as video and audio, and the transmission and reception of application operation status information between the communication control unit 29 and the server device 30. More specifically, the communication control unit 29 controls the communication unit 19 to receive video and audio from the virtual space from the server device 30. The communication control unit 29 controls the communication unit 19 to transmit to the server device 30 the video and audio of the user facing the display unit 17 of the information terminal device 10, which have been acquired by the video acquisition unit 21 and the audio acquisition unit 22. Application operation status information includes, for example, in a virtual space application, the operation of the avatar by the operation unit 13, and the operation of starting and ending video calls.

[0054] (Server device) Figure 3 is a block diagram showing an example configuration of a server device according to the first embodiment. The server device 30 is a server device of an entity that provides virtual space services, and may be a general-purpose server device including a distributed type. The server device 30 constructs and controls the virtual space. The server device 30 comprises a communication unit 31 and a server control device 40.

[0055] The communication unit 31 is a communication unit for wired or wireless communication. The communication unit 31 communicates using methods such as Wi-Fi or mobile phone lines and connects to an intranet or the internet. The communication unit 31 is controlled by the communication control unit 41. The communication unit 31 communicates data such as video, audio, and application operating status information with multiple information terminal devices 10.

[0056] (Server control unit) The server control device 40 controls various parts of the server device 30. The server control device 40 includes, for example, a control device such as a CPU or MPU, and a storage device such as RAM or ROM. The server control device 40 may be implemented by an integrated circuit such as an ASIC or FPGA. The server control device 40 may be implemented by a combination of hardware and software. The server control device 40 executes a program that controls the operation of the server device 30 according to this disclosure.

[0057] The server control device 40 comprises a communication control unit 41, a virtual office control unit 42, and a storage unit 49, as functional blocks realized by circuits and programs.

[0058] The communication control unit 41 controls the communication unit 31 to control the transmission and reception of data, including video and audio, and application operation status information between the information terminal device 10 and the communication control unit 41. For example, the communication control unit 41 controls the communication unit 31 to receive video captured by the camera 11 and audio picked up by the microphone 12 of the information terminal device 10. For example, the communication control unit 41 controls the communication unit 31 to transmit video and audio of the virtual space to the information terminal device 10. Application operation status information includes, for example, configuration information of the virtual space application.

[0059] The virtual office control unit 42 controls the operation of various applications running on the server device 30. For example, the virtual office control unit 42 controls the operation of applications in the virtual space. For example, the virtual office control unit 42 controls the operation of applications used for online communication.

[0060] The virtual office control unit 42 transmits the configuration of the application running on the server device 30 to the information terminal device 10 running the application, via the communication control unit 41. Specifically, the virtual office control unit 42 transmits information indicating the configuration of the virtual space running on the server device 30 to the information terminal device 10 logged into the virtual space. In this embodiment, as the configuration of the virtual space running on the server device 30, the virtual office control unit 42 transmits information indicating the configuration of the event space to the information terminal device 10 of the user logged into the virtual space and participating in the event.

[0061] The memory unit 49 stores information such as the calculations performed by the server control device 40 and programs. The memory unit 49 is composed of, for example, RAM, a main memory such as ROM, and a storage device such as an SSD or HDD.

[0062] The memory unit 49 stores information that indicates a virtual space, such as an event space, which is configured in the virtual space. The memory unit 49 also stores information for each user participating in an event in the virtual space, including an avatar, an information terminal device 10 associated with the avatar, and an account user of the information terminal device 10.

[0063] If the information processing device 50 is implemented in the information terminal device 10, the detection unit 53 only needs to detect the first user who is logged into the information terminal device 10. If the information processing device 50 is implemented in the server device 30, the detection unit 53 only needs to detect all users.

[0064] (Information processing device) An example of the configuration of an information processing device according to the embodiment will be described using Figure 4. Figure 4 is a block diagram showing an example of the configuration of an information processing device according to the first embodiment. The information processing device 50 is a conceptual device realized through the cooperation of either or both of the server device 30 and the multiple information terminal devices 10 used in the virtual office. Either the information terminal device 10 or the server device 30 may be the information processing device 50, or the virtual space control system 1 comprising the information terminal device 10 and the server device 30 may be the information processing device 50. Each part of the information processing device 50 is realized through the cooperation of either or both of the terminal control device 20 and the server control device 40.

[0065] When the information processing device 50 detects a first avatar that is isolated among the avatars corresponding to users who are participants in an event in the virtual space, it facilitates communication between the first user and a second user who has user information common to the first user's user information, via a dummy avatar.

[0066] As shown in Figure 4, the information processing device 50 includes an application control unit 51, an information acquisition unit 52, a detection unit 53, a search unit 54, and a dummy avatar control unit 55. For example, the application control unit 51, the detection unit 53, and the search unit 54 may be implemented by the server control device 40, the information acquisition unit 52 may be implemented through the cooperation of the terminal control device 20 and the server control device 40, and the dummy avatar control unit 55 may be implemented by the terminal control device 20, but is not limited to these.

[0067] The application control unit 51 controls the display of the dummy avatar in the virtual space application running on the information terminal device 10, using the dummy avatar control unit 55, which will be described later.

[0068] The information acquisition unit 52 acquires user information for each user of multiple avatars contained within a predetermined space, such as an event space, in the virtual space. The information acquisition unit 52 acquires user information, which is various information about the user, from the application control unit 24 of the information terminal device 10 or the virtual office control unit 42 of the server device 30. For example, as user information for each user of multiple avatars, the information acquisition unit 52 acquires information about events defined within the predetermined space in the virtual space.

[0069] User information, for example, is information about events defined within a specific space in the virtual environment, which each user enters when registering to participate.

[0070] The detection unit 53 detects, based on user information acquired by the information acquisition unit 52, that any avatar among the multiple avatars tends to be isolated. The detection unit 53 detects the isolated avatar based on the positional relationship of the multiple avatars, sentiment analysis of each avatar towards each user, and conversation analysis, or based on information from the application control unit 24 of the information terminal device 10 or the virtual office control unit 42 of the server device 30. The detailed processing of the detection unit 53 will be described later.

[0071] Figure 5 is a block diagram showing an example configuration of the detection unit according to the first embodiment. The detection unit 53 includes, as an example, an emotion estimation unit 531, an avatar location information acquisition unit 532, and a conversation state acquisition unit 533. The configuration of the detection unit 53 is an example and is not limited to it.

[0072] The emotion estimation unit 531 acquires information indicating the emotions of each avatar user. The emotion estimation unit 531 estimates the user's emotions from the user's facial image using emotion recognition AI, for example, by using a server device 30 or an external AI (Artificial Intelligence) server device connected to the server device 30. The emotion estimation unit 531 estimates, for example, whether each user's emotions tend to be negative or positive, from the facial image of each user captured by the camera 11 of the information terminal device 10 used by each user. The emotion estimation unit 531 estimates emotions such as discomfort, confusion, and dissatisfaction from each user's facial image. The method for estimating emotions is not limited to known methods and can be used.

[0073] The avatar location information acquisition unit 532 acquires the location information of each avatar within a predetermined space, such as an event space, in the virtual space. The avatar location information acquisition unit 532 acquires the location information of each avatar from the application control unit 24 of the information terminal device 10 or the virtual office control unit 42 of the server device 30.

[0074] The conversation state acquisition unit 533 acquires information indicating the speech state of each avatar user within a predetermined space, such as an event space, in the virtual space. The conversation state acquisition unit 533 acquires information indicating the speech state of each avatar user from the voice of each user picked up by the microphone 12 of the information terminal device 10 used by each user. The conversation state acquisition unit 533 may, for example, measure the amount of speech as the user's speech state based on the voice acquired by the microphone 12 provided in the information terminal device 10 logged into the virtual office. The conversation state acquisition unit 533 may, for example, use a speech segment recognition AI or the like to measure the amount of speech that does not include sounds other than speech from the voice signal acquired by the microphone 12. The method for acquiring the speech state is not limited to known methods and may be used.

[0075] The processing in the detection unit 53 will now be explained. The detection unit 53 detects avatars that tend to be isolated among multiple avatars based on at least one of the following: avatar location information acquired by the avatar location information acquisition unit 532, information indicating the user's emotions estimated by the emotion estimation unit 531, and information indicating the user's speech state acquired by the conversation state acquisition unit 533.

[0076] The detection unit 53 may detect that an avatar is isolated among multiple avatars based on the location information of each avatar acquired by the avatar location information acquisition unit 532. More specifically, the detection unit 53 may detect that an avatar is isolated among multiple avatars if, for example, each user is not located near other avatars for a first predetermined period, such as 30 minutes, during the event participation period, or if the period during which the user is located near other avatars is less than a second predetermined period, such as less than 5 minutes.

[0077] Being located near another avatar means, for example, that the avatars are in a position where they are mutually within each other's information sharing range.

[0078] The detection unit 53 may further use the emotion estimation unit 531's estimated information regarding each avatar's emotion to detect that the avatar is isolated among multiple avatars. More specifically, the detection unit 53 may detect that an avatar is isolated among multiple avatars if, in addition to the location information of each avatar meeting the above-mentioned conditions, the user's emotion for each avatar is negative. This makes it possible to exclude cases where the user is intentionally isolated and is content with being isolated (in other words, not dissatisfied).

[0079] The detection unit 53 may further use the information indicating the speech state of each avatar acquired by the conversation state acquisition unit 533 to detect that an avatar tends to be isolated among multiple avatars. More specifically, the detection unit 53 may detect that an avatar tends to be isolated among multiple avatars if, in addition to the location information of each avatar meeting the above-mentioned conditions, the user of each avatar has not engaged in continuous speech (conversation) for 5 minutes or more since starting to participate in the event.

[0080] If the detection unit 53 detects a first avatar that is showing a tendency towards isolation, the search unit 54 searches for a second user that has commonalities with the user information of the first user from the user information of users other than the first user who is the user of the first avatar. The search unit 54 searches for a second user that has commonalities with the user information of the first user from the user information of the first user who is the user of the avatar that has been detected by the detection unit 53 as showing a tendency towards isolation.

[0081] The search unit 54, for example, searches for other users who have the same purpose for participating in the event as second users, based on user information about the first user. The search unit 54, for example, searches for other users who have the same purpose for participating in the event as first users, based on user information about first users, and who belong to companies in a different industry from the first user's company but are not competitors. For example, if the purpose for participating in the event in the user information about first users is "finding business partners," the search unit 54 searches for other users who belong to companies in a different industry from the first user's company but are not competitors, and whose purpose for participating is "finding business partners," as second users.

[0082] The search unit 54, for example, searches for other users as second users based on user information about the first user, who have the same purpose for participating in the event and belong to companies in the same industry as the first user's company. For example, if the purpose of participating in the event in the user information about the first user is "networking," the search unit 54 searches for other users as second users who belong to companies in the same industry as the first user's company and whose purpose of participation is "networking."

[0083] The search unit 54 may designate a user corresponding to an avatar that is not communicating with other avatars as the second user, or it may designate a user corresponding to an avatar that is communicating with other avatars as the second user. In the latter case, it is preferable that the user is communicating with other avatars that share the same purpose of participation.

[0084] The dummy avatar control unit 55 controls the display and operation of the dummy avatar on the virtual space display screen. The dummy avatar control unit 55 places the dummy avatar near the first avatar and communicates with the first user based on user information about the first user, and causes the dummy avatar to perform actions that facilitate communication between the first user and the second user.

[0085] A dummy avatar is a dummy avatar that acts as a guide, mediator, and bridge for communication between users participating in an event. A dummy avatar is a virtual personality avatar that is not associated with any user. A dummy avatar may have an appearance that is appropriate for the type of event, and may also have an inconspicuous appearance and facial features. A dummy avatar may be identifiable as a dummy by all users based on its appearance, or it may not be identifiable. A dummy avatar may be identifiable as a dummy by the appearance only to the first user, and not identifiable to other users. A dummy avatar is configured so that its expressions, actions, and speech all come across as positive.

[0086] The actions of the dummy avatar, including its speech, are controlled by the information processing device 50 or an external AI server device on which the information processing device 50 is available. The dummy avatar's personality is generated by a conversational AI, for example, to act as a listener to the user. The actions of the dummy avatar are controlled to operate in response to the actions of the first avatar and the speech of the first user, and to the actions of the second avatar and the speech of the second user.

[0087] The dummy avatar control unit 55 positions the dummy avatar near the first avatar detected by the detection unit 53. The dummy avatar control unit 55 moves the first avatar and the dummy avatar to a position where they are both within each other's information sharing range. The dummy avatar control unit 55 displays on the display screen of the information terminal device 10 used by the first user that the dummy avatar is approaching the first avatar from a distance.

[0088] The dummy avatar control unit 55 causes the dummy avatar to communicate with the first user based on user information about the first user. When the dummy avatar moves to the vicinity of the first avatar on the display screen of the information terminal device 10 used by the first user, and they are in a position where they are facing each other and within the mutual information sharing range, the dummy avatar control unit 55 causes the dummy avatar to speak to the first avatar based on user information about the first user.

[0089] If the dummy avatar control unit 55 detects that communication between the first user and the second user is continuing, it removes the dummy avatar from the information sharing range.

[0090] This section explains a specific example of a dummy avatar speaking to the primary avatar. As an example, it describes a scenario where both the primary user's and secondary user's objectives for participating in the event are "networking."

[0091] The dummy avatar control unit 55, for example, based on user information about the first user that has been acquired in advance, instructs the dummy avatar to speak to the first avatar, saying things like, "You're Mr. / Ms. XX from XXX company, right?" or "What can I help you with today?"

[0092] The dummy avatar control unit 55, for example, based on user information about the second user and the location information of the second avatar that have been acquired in advance, has the dummy avatar speak to the first avatar, saying, "There is someone from △△△ company over there. They seem like a company that would be good to share information with ○○○ company. Shall we meet with them?" and prompts the first avatar to move.

[0093] The dummy avatar control unit 55, for example, while moving with the first avatar, continues the conversation in a way that encourages the first user to be more positive about communicating with the second user.

[0094] When the first avatar, the second avatar, and the dummy avatar move to a position where they are all within the information sharing range, the dummy avatar control unit 55, for example, based on user information about the second user that has been acquired in advance and information about the common industry, has the dummy avatar speak to the second avatar, saying, "Mr. / Ms. △△, I'd like to ask you about recent ×× in the □□ industry," or "This is Mr. / Ms. ○○ from ○○○ company. If you don't mind, I'd like to join you."

[0095] If the dummy avatar control unit 55 confirms that the conversation between the first user and the second user is continuing and determines that it is a suitable time to end the conversation, it moves the dummy avatar away from the positions of the first and second avatars, for example, by saying, "Well, I'll be going now..."

[0096] A good stopping point would be, for example, when the conversation between the first and second users comes to a natural conclusion, when the topic of conversation changes, or when another user joins the conversation between the first and second users.

[0097] (Virtual space control method) The virtual space control method will be explained using Figure 6. Figure 6 is a flowchart showing an example of the virtual space control method according to the first embodiment. The process shown in Figure 6 is executed while the virtual space is provided by the virtual space control system 1, in other words, while an event is being held in the virtual space. When the process shown in Figure 6 is started, the display of the virtual space begins.

[0098] The information processing device 50 determines whether or not a first avatar, which is an avatar prone to isolation, has been detected (step S101). More specifically, the information processing device 50 uses a detection unit 53 to detect an avatar prone to isolation among multiple avatars based on at least one of the following: avatar location information obtained by an avatar location information acquisition unit 532, information indicating the user's emotions obtained by an emotion estimation unit 531, and information indicating the user's speech state obtained by a conversation state acquisition unit 533. If the information processing device 50 determines that a first avatar prone to isolation has been detected (step S101; Yes), it proceeds to step S102. If the information processing device 50 does not determine that a first avatar prone to isolation has been detected (step S101; No), it proceeds to step S108.

[0099] The information processing device 50 places a dummy avatar near the first avatar, which tends to be isolated (step S102). More specifically, the information processing device 50 uses the dummy avatar control unit 55 to move the first avatar and the dummy avatar to a position where they are both included in the information sharing range. The information processing device 50 then proceeds to step S103.

[0100] The information processing device 50 uses the search unit 54 to search for a second avatar, which is another avatar with common characteristics, based on the user information of the first avatar (step S103). More specifically, the information processing device 50 uses the search unit 54 to search for a second user whose user information is common to the user information of the first user, based on the user information of the first user, which is the user of the avatar detected by the detection unit 53 as having a tendency towards isolation. The information processing device 50 then proceeds to step S104.

[0101] The information processing device 50 facilitates communication between the first avatar and the second avatar via the dummy avatar through the dummy avatar control unit 55 (step S104). The information processing device 50 then proceeds to step S105.

[0102] The information processing device 50 determines whether or not communication between the first avatar and the second avatar is continuing (step S105). More specifically, if the information processing device 50 determines, based on the dummy avatar control unit 55, that communication between the first avatar and the second avatar is continuing (step S105; Yes), it proceeds to step S106. If the information processing device 50 does not determine, based on the dummy avatar control unit 55, that communication between the first avatar and the second avatar is not continuing (step S105; No), it proceeds to step S107.

[0103] If the information processing device 50 determines that communication between the first avatar and the second avatar is continuing (step S105; Yes), the information processing device 50 has the dummy avatar control unit 55 move the dummy avatar out of the vicinity of the first avatar (step S106). The information processing device 50 then proceeds to step S108.

[0104] If it is determined that communication between the first avatar and the second avatar is not continuing (step S105; No), the information processing device 50 changes the target second avatar using the search unit 54 (step S107). The information processing device 50 then proceeds to step S104.

[0105] The information processing device 50 determines whether or not to terminate the use of the virtual space (step S108). For example, if the information processing device 50 detects an operation to terminate the use of the virtual space, it determines that the use of the virtual space should be terminated. If the information processing device 50 determines that the use of the virtual space should be terminated (step S108; Yes), it terminates the process in this flowchart. If the information processing device 50 does not determine that the use of the virtual space should be terminated (step S108; No), it executes the process in step S101 again.

[0106] The display screen will be explained using Figures 7 to 9. As shown in Figures 7 to 9, the display unit 17 of the information terminal device 10 displays a display screen that shows the event space in the virtual space, for example, by a virtual space application.

[0107] Figure 7 shows an example of a virtual space display screen shown on the display screen of an information terminal device. In the following description, the nth (1st to 8th) avatar is the avatar corresponding to the nth user, and the information sharing range of the nth avatar is defined as information sharing range Cn. Figure 7 shows a flowchart in which the detection unit 53 detects that avatar A1 is becoming isolated (step S101; Yes), and dummy avatar AD is moving towards avatar A1, which is avatar A1. In Figure 6, avatar A1, which corresponds to the first user, is displayed, and the information sharing range C1 for avatar A1 is set. The information sharing range for each avatar may or may not be displayed. Since avatar A1 is not included in the mutual information sharing range of other avatars, the first user corresponding to avatar A1 is not communicating with users of other avatars. Even in this state, the information terminal device 10 used by the first user outputs the voices of other surrounding avatars, for example, avatars A6 and A8, at a low volume. Avatars A6 and A8 are located within mutually shared information areas C6 and C8, respectively, and are positioned to communicate with each other. User 6, corresponding to Avatar A6, and User 8, corresponding to Avatar A8, are currently communicating with each other.

[0108] Figure 8 shows another example of the display screen of the virtual space shown on the display screen of the information terminal device. In Figure 8, in step S102 of the flowchart, a dummy avatar AD is placed near the first avatar, avatar A1, and when dummy avatar AD moves near the first avatar A1, they are in a position facing each other, within the information sharing ranges C1 and CD. In Figure 8, dummy avatar AD is speaking to the first avatar A1 based on information about the first user.

[0109] Figure 9 shows another example of the display screen of the virtual space shown on the display screen of the information terminal device. In step S104 of the flowchart, Figure 9 shows that dummy avatar AD and avatar A1 move to the vicinity of avatar A5, which is set as the second avatar, and dummy avatar AD facilitates communication between avatar A1, which is the first avatar, and avatar A5, which is the second avatar.

[0110] (effect) As described above, in this embodiment, if a first avatar that tends to be isolated is detected among the avatars corresponding to users who are participants in an event in the virtual space, communication between the first user and a second user who has user information common to the first user can be facilitated via a dummy avatar. In this way, communication between participants in an event in the virtual space can be facilitated in this embodiment.

[0111] In this embodiment, user information for each user of multiple avatars is obtained, specifically information about events defined within a predetermined space in the virtual world. According to this embodiment, communication among participants can be appropriately facilitated depending on the event they are participating in.

[0112] In this embodiment, if it is detected that communication between the first user and the second user is continuing, the dummy avatar is removed. According to this embodiment, communication between the first user and the second user can be appropriately facilitated.

[0113] In this embodiment, it is possible to detect avatars that tend to be isolated based on the positional relationships of multiple avatars, sentiment analysis of each of the multiple avatars towards their respective users, and conversation analysis.

[0114] [Second Embodiment] The virtual space control system 1 relating to the first embodiment will be described using Figure 10. Figure 10 is a flowchart of an example of a virtual space control method according to the second embodiment. In the following description, components similar to those of the virtual space control system 1 of the first embodiment will be denoted by the same reference numerals or corresponding reference numerals, and their detailed descriptions will be omitted. The processing in the information processing device 50 of the virtual space control system 1 differs from that of the first embodiment.

[0115] The information acquisition unit 52 acquires user information for each user of multiple avatars that are contained within a predetermined space in the virtual space and are in a state where they can communicate with each other.

[0116] The detection unit 53 detects, based on user information acquired by the information acquisition unit 52, that any avatar among multiple avatars that are in a state where they can communicate with each other tends to be isolated among the multiple avatars. The detection unit 53 detects an avatar that tends to be isolated within a circle where multiple avatars communicate with each other, in other words, in a state where multiple avatars are included in the information sharing range and can communicate with each other.

[0117] The detection unit 53 may, for example, detect an avatar that tends to be isolated among multiple avatars, using information indicating the emotions of each avatar estimated by the emotion estimation unit 531, when multiple avatars are mutually included in the information sharing range and are able to communicate with each other. More specifically, the detection unit 53 may detect an avatar as tending to be isolated among multiple avatars when multiple avatars are mutually included in the information sharing range and are able to communicate with each other, and the avatar's user's emotions tend to be negative.

[0118] The detection unit 53 may, for example, use information indicating the speech state of each avatar acquired by the conversation state acquisition unit 533, when multiple avatars are mutually included in the information sharing range and capable of communicating with each other, to detect an avatar that tends to be isolated among multiple avatars. More specifically, the detection unit 53 may detect an avatar as tending to be isolated among multiple avatars when, while multiple avatars are mutually included in the information sharing range and capable of communicating with each other, the avatar's user has not indicated that they have engaged in continuous speech (conversation) for 5 minutes or more.

[0119] When the detection unit 53 detects a first avatar that is showing signs of isolation, the search unit 54 searches for a second user among multiple avatar users who are in a state where they can communicate with each other, based on the user information of users other than the first user who is the user of the first avatar, and who has user information that is common to the user information of the first user. The search unit 54 searches for a second user among multiple avatars in the communication circle who has user information that is common to the user information of the first user, based on the user information of the first user who is the user of the avatar that has been detected by the detection unit 53 as showing signs of isolation.

[0120] The dummy avatar control unit 55, for example, based on user information about the second user that has been acquired in advance and information about the common industry, instructs the dummy avatar to speak to the second avatar in the communication circle, saying things like, "△△, I'd like to ask you about recent ×× in the □□ industry," or "This is ○○ from ○○○ company. If you'd like, I'd like you to join us."

[0121] The virtual space control method will be explained using Figure 10. Figure 10 is a flowchart showing an example of the virtual space control method according to the second embodiment. The processing in steps S202, S204 to S208 is the same as the processing in steps S102, S104 to S108 of the flowchart shown in Figure 6.

[0122] The information processing device 50 determines whether or not a first avatar, which is an avatar that tends to be isolated within the communication circle, has been detected (step S201). More specifically, the information processing device 50 uses a detection unit 53 to detect an avatar that tends to be isolated among multiple avatars in the communication circle, based on at least one of the following: avatar location information obtained by an avatar location information acquisition unit 532, information indicating the user's emotions obtained by an emotion estimation unit 531, and information indicating the user's speech state obtained by a conversation state acquisition unit 533. If the information processing device 50 determines that a first avatar, which is an avatar that tends to be isolated within the communication circle, has been detected (step S201; Yes), it proceeds to step S202. If the information processing device 50 does not determine that a first avatar, which is an avatar that tends to be isolated within the communication circle, has been detected (step S201; No), it proceeds to step S208.

[0123] The information processing device 50 uses the search unit 54 to search for a second avatar, which is another avatar with common characteristics, within the communication circle based on the user information of the first avatar (step S203). More specifically, the information processing device 50 uses the search unit 54 to search for a second user within the communication circle that has user information common to the user information of the first user, based on the user information of the first user, which is the user of the avatar detected by the detection unit 53 as having a tendency towards isolation. The information processing device 50 then proceeds to step S204.

[0124] (effect) As described above, in this embodiment, when a first avatar that tends to be isolated is detected among the avatars corresponding to users who are participants capable of mutual communication in an event in a virtual space, communication between the first user and a second user who has user information common to the first user can be facilitated via a dummy avatar.

[0125] The components of the illustrated virtual space control system are functionally conceptual and do not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage of each device.

[0126] The configuration of the virtual space control system is implemented, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks implemented through the cooperation of hardware or software. That is, these functional blocks can be implemented in various forms using hardware alone, software alone, or a combination thereof.

[0127] The components described above include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the components described above can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention. [Explanation of Symbols]

[0128] 1. Virtual Space Control System 10 Information terminal device 11 Cameras 12 microphones 13 Control section 17 Display section 18 Audio output section 19 Communications Department 20 Terminal control device 21 Video Acquisition Unit 22 Voice acquisition unit 23 Operation Reception Section 24 Application Control Unit 26 Memory section 27 Display Control Unit 28 Audio Output Control Unit 29 Communication Control Unit 30 Server Devices 31 Communications Department 40 Server Control Units 41 Communication Control Unit 42 Virtual Office Control Unit 49 Memory section 50 Information Processing Devices 51 Application Control Unit 52 Information Acquisition Department 53 Detection unit 531 Emotion estimation part 532 Avatar Location Information Acquisition Unit 533 Conversation state acquisition unit 54 Search Section 55 Dummy Avatar Control Unit

Claims

1. A virtual space control system that displays a virtual space on an information terminal device and displays an avatar corresponding to the user of the information terminal device in the virtual space, The information processing device included in the virtual space control system is: The user information of each user of a plurality of avatars contained within a predetermined space in the virtual space is obtained. Based on the user information, it is detected that any of the multiple avatars tend to be isolated from the other multiple avatars. If a first avatar showing signs of isolation is detected, a second user with commonalities with the first user's user information is searched from the user information of users other than the first user who is the user of the first avatar. A dummy avatar is placed near the first avatar, and the dummy avatar is made to perform actions that facilitate communication between the first user and the second user, based on user information relating to the first user. Virtual space control system.

2. The aforementioned information processing device is User information is acquired for each user of multiple avatars that are contained within a predetermined space in the virtual space and are in a state where they can communicate with each other. Based on the user information, it is detected that any of the multiple avatars that are in a state where they can communicate with each other tend to be isolated among the multiple avatars. If a first avatar that is showing signs of isolation is detected, the system searches for a second user among the users of multiple avatars that are in a state where they can communicate with each other, based on the user information of users other than the first user who is the user of the first avatar, and that has common user information with the first user. The virtual space control system according to claim 1.

3. The aforementioned information processing device is As user information for each of the multiple avatars, information regarding events defined within a predetermined space in the virtual space is obtained. The virtual space control system according to claim 1.

4. The aforementioned information processing device is Based on the spatial relationships of the multiple avatars, sentiment analysis of each of the multiple avatars toward their respective users, and conversational analysis, the system detects avatars that tend to be isolated. The virtual space control system according to claim 1.

5. A virtual space control method executed by a virtual space control system that displays a virtual space on an information terminal device and displays an avatar corresponding to the user of the information terminal device in the virtual space, The user information of each user of a plurality of avatars contained within a predetermined space in the virtual space is obtained. The system detects that any of the aforementioned multiple avatars tend to be isolated from the others. If a first avatar showing signs of isolation is detected, a second user with commonalities with the first user's user information is searched from the user information of users other than the first user who is the user of the first avatar. A dummy avatar is placed near the first avatar, and the dummy avatar is made to perform actions that facilitate communication between the first user and the second user, based on user information relating to the first user. A virtual space control method performed by a virtual space control system.

Images