Virtual space control system and virtual space control method

Abstract

An information processing device 50 provided in a virtual space control system 1 reflects user information, which is information of a user, to an avatar corresponding to the user when it is detected that the user is in a state in which communication with another user is possible in a virtual space, and does not reflect the user information to the avatar when it is not detected that the user is in a state in which communication with another user is possible in the virtual space.

Description

Virtual space control system and virtual space control method

[0001] The present invention relates to a virtual space control system and a virtual space control method.

[0002] Communication using virtual spaces such as virtual offices is common. In virtual spaces such as virtual offices, users use their own avatars to interact with the virtual space. There is known a technology for reflecting the movements and facial expressions of the avatar's user on the avatar (see, for example, Patent Document 1).

[0003] JP 2023-175784 A

[0004] However, reflecting the facial expressions of the avatar user on the avatar in the virtual office means that the avatar user is constantly being photographed by a camera while logged in to the virtual office, which may cause stress due to tension, etc.

[0005] The present disclosure aims to reflect information about the user of an avatar when appropriate for the avatar.

[0006] In order to solve the above-mentioned problems and achieve the objectives, the virtual space control system of the present disclosure is a virtual space control system that displays a virtual space on an information terminal device and displays an avatar corresponding to the user of the information terminal device in the virtual space, and when the information processing device provided in the virtual space control system detects that the user is in a state where they can communicate with other users in the virtual space, it reflects user information, which is information about the user, in the avatar corresponding to the user, and when it does not detect that the user is in a state where they can communicate with other users in the virtual space, it does not reflect the user information in the avatar.

[0007] The virtual space control method of the present disclosure is a virtual space control method executed by a virtual space control system that causes an information terminal device to display a virtual space and displays an avatar corresponding to a user of the information terminal device in the virtual space, and when it is detected that the user is in a state where they can communicate with other users in the virtual space, the virtual space control system reflects user information, which is information about the user, in the avatar corresponding to the user, and when it is not detected that the user is in a state where they can communicate with other users in the virtual space, the virtual space control system does not reflect the user information in the avatar.

[0008] According to the present disclosure, information about the user of an avatar can be reflected when appropriate for the avatar.

[0009] FIG. 1 is a schematic diagram showing an example of the configuration of a virtual space control system according to a first embodiment. FIG. 2 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment. FIG. 3 is a block diagram showing an example of the configuration of a server device according to the first embodiment. FIG. 4 is a block diagram showing an example of the configuration of an information processing device according to the first embodiment. FIG. 5 is a diagram showing an example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 6 is a diagram showing another example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 7 is a diagram showing another example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 8 is a diagram showing another example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 9 is a flowchart showing an example of a virtual space control method according to the first embodiment. FIG. 10 is a block diagram showing an example of the configuration of an information processing device according to a second embodiment. FIG. 11 is a flowchart showing an example of a virtual space control method according to the second embodiment. FIG. 12 is a flowchart showing an example of a virtual space control method according to a third embodiment. FIG. 13 is a diagram showing an example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 14 is a flowchart showing an example of a virtual space control method according to a fourth embodiment. FIG. 15 is a flowchart showing an example of a virtual space control method according to a fifth embodiment.

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A virtual space control system and a virtual space control method according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0011] [First embodiment] (Virtual space control system) Fig. 1 is a schematic diagram showing an example of the configuration of a virtual space control system according to a first embodiment. The virtual space control system 1 controls a virtual office configured in a virtual space. In the following embodiment, a virtual office will be described as an example, but the present invention is not limited to virtual offices and can also be applied to various services that use avatars for communication in a virtual space.

[0012] In this embodiment, the virtual space control system 1 changes the state in which user information (hereinafter referred to as "user information") is reflected in the avatar corresponding to the user depending on whether the virtual space control system 1 detects that the user is able to communicate with other users in a virtual space such as a virtual office or not. Specifically, when the virtual space control system 1 detects that the user is able to communicate with other users in the virtual office, it reflects the user information in the avatar corresponding to the user, and when it does not detect that the user is able to communicate with other users in the virtual office, it does not reflect the user information in the avatar.

[0013] In the following description, a "user" may be referred to as a "first user" and an "other user" may be referred to as a "second user." Furthermore, an information terminal device used by a first user to log in to a virtual office may be referred to as a first information terminal device, and an information terminal device used by another second user to log in to a virtual office may be referred to as a second information terminal device. Furthermore, an avatar corresponding to a first user may be referred to as a first avatar, and an avatar corresponding to a second user may be referred to as a second avatar.

[0014] A state in which communication with other users is possible in the virtual office means a state in which communication is possible in the virtual office using one's own avatar and the avatar of another user, and more specifically, a state in which mutual voices can be heard regardless of whether communication is occurring or not. For example, a state in which a first user and a second user can hear each other's voices via the first avatar and the second avatar, regardless of whether the conversation is between the first user and the second user.

[0015] In this embodiment, user information is, for example, information indicating the user's facial expression or movement. The user's facial expression may be, for example, an open-mouthed speaking expression, a laughing expression, an angry expression, a thoughtful expression, etc. The user's movement may include the movement of the hands, arms, etc. in addition to the facial expression.

[0016] Reflecting on the avatar means, for example, displaying an image that shows the facial expression indicated by the user information on the face of the avatar. Reflecting on the avatar means, for example, displaying a video of the user's face on the face of the avatar. Reflecting on the avatar means, for example, displaying the user's facial expression reflected on the facial expression of the avatar.

[0017] Reflecting the user information on the avatar means, for example, displaying the user information on the body parts of the avatar so that the user can understand the movements of the user information. For example, reflecting the user information on the avatar means reflecting the movements of the user's hands, arms, etc. on the avatar.

[0018] An overview of the virtual office controlled by the virtual space control system 1 will be described. A user uses an information terminal device 10 to come to work (log in) to the virtual office using their own account and perform their work. In the virtual office, an avatar associated with the user is set. When a user logs in to the virtual office using their own information terminal device 10 (described later), the avatar moves to any seat in the virtual office or to a predetermined seat, and the user operates and displays the avatar to sit down. The user can operate the avatar to move around the virtual office. In the virtual office, for example, when avatars are close to each other in the virtual office or when avatars are located in the same space, such as a conference room, they can share information such that they can hear each other's voices.

[0019] The virtual space control system 1 includes an information terminal device 10 that each user uses for business purposes and logs into the virtual office, and a server device 30 .

[0020] (Information Terminal Device) Fig. 2 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment. The information terminal device 10 is a terminal device used by a user who performs work in a virtual office. The information terminal device 10 is a terminal device on which the user executes a virtual office application. The information terminal device 10 is a terminal device through which the user logs in to the virtual office using their own account when logging in to the virtual office.

[0021] The information terminal device 10 is, for example, an information terminal device such as a general-purpose personal computer, a smartphone, or a tablet terminal. The information terminal device 10 is connected to the server device 30 via a network so as to be able to communicate data with the server device 30.

[0022] The information terminal device 10 logs in to the virtual office based on the account information of the user who is logged in to the information terminal device 10 .

[0023] The information terminal device 10 includes a camera 11 , a microphone 12 , an operation unit 13 , a display unit 17 , an audio output unit 18 , a communication unit 19 , and a terminal control device 20 .

[0024] The camera 11 captures an image in a direction facing the information terminal device 10. The camera 11 is disposed at a position where it captures an image in a direction facing the display unit 17 of the information terminal device 10. In other words, the camera 11 captures an image in a direction where a user using the information terminal device 10 is present. The camera 11 is disposed, for example, near the display unit 17. The camera 11 outputs the captured image to the image acquisition unit 21 of the terminal control device 20.

[0025] The microphone 12 picks up sounds around the information terminal device 10. The microphone 12 may pick up sounds facing the display unit 17 of the information terminal device 10, or may be provided in a headset connected to the information terminal device 10. In other words, the microphone 12 picks up sounds spoken by the user using the information terminal device 10 and sounds from the surrounding area. The microphone 12 outputs audio information related to the picked-up sounds to the audio acquisition unit 22 of the terminal control device 20.

[0026] The operation unit 13 is an input device capable of inputting various operations to the information terminal device 10. The operation unit 13 outputs an operation signal corresponding to the received input operation to the terminal control device 20. The operation unit 13 includes, for example, a keyboard, a mouse, a touch panel overlaid on the display unit 17, buttons, switches, etc. The operation unit 13 can accept, for example, a key input operation, a click operation, or a touch operation.

[0027] The operation unit 13 can accept operations such as moving an avatar corresponding to the user in the virtual office in any direction on the display screen of the display unit 17. For example, the operation unit 13 accepts operations such as overlapping or placing the user's avatar in the virtual office next to another avatar on the display screen of the display unit 17.

[0028] By performing an operation such as superimposing one's avatar on another or placing it adjacent to another avatar, communication such as a video call or a voice call can be initiated between users in the virtual office application. Whether a video call or a voice call will be conducted may be set in advance or may be set by the user's operation.

[0029] The display unit 17 displays various types of video including text and images. The display unit 17 is, for example, a display including a liquid crystal display, an organic electroluminescence (EL) display, etc. The display unit 17 displays a video of a virtual office, etc. In this embodiment, when a video call becomes possible, the display unit 17 outputs a video of the user of another avatar with whom the video call is being made, in other words, a video captured by the camera 11 of the information terminal device 10 used by the user of the other avatar. The display on the display unit 17 is controlled by the display control unit 27.

[0030] The audio output unit 18 is an audio output device that outputs audio from the virtual office. In this embodiment, when the user is using the virtual office, the audio output unit 18 outputs audio around the user's avatar in the virtual office. When a voice call or video call becomes possible, the audio output unit 18 also outputs the audio of the user of another avatar making the voice call or video call, in other words, the audio picked up by the microphone 12 of the information terminal device 10 used by the user of the avatar. The audio output of the audio output unit 18 is controlled by an audio output control unit 28.

[0031] The communication unit 19 is a communication unit for performing wired communication or wireless communication. The communication unit 19 performs communication using a communication method such as Wi-Fi (registered trademark) or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 19 is controlled by a communication control unit 29. The communication unit 19 communicates data such as video and audio, or application operation information, with the server device 30. The communication unit 19 communicates data such as account information of a user logged in to the information terminal device 10 and application operation status information with the server device 30.

[0032] (Terminal Control Device) The terminal control device 20 controls each unit of the information terminal device 10. The terminal control device 20 has, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The terminal control device 20 executes a program that controls the operation of the information terminal device 10 according to the present disclosure. The terminal control device 20 may be realized by an integrated circuit such as an ASIC or FPGA. The terminal control device 20 loads a stored program into memory and executes instructions included in the program. The terminal control device 20 includes an internal memory such as the RAM described above, which is used for temporary storage of data in the terminal control device 20. The terminal control device 20 may be realized by a combination of hardware and software.

[0033] The terminal control device 20 has functional blocks realized by circuits and executed programs, including a video acquisition unit 21, an audio acquisition unit 22, an operation reception unit 23, an application control unit 24, a user information analysis unit 25, a memory unit 26, a display control unit 27, an audio output control unit 28, and a communication control unit 29.

[0034] The video acquisition unit 21 acquires video captured by the camera 11. More specifically, the video acquisition unit 21 acquires video captured by the camera 11 of a user facing the information terminal device 10, etc. The video data acquired by the camera 11 is a moving image.

[0035] The voice acquisition unit 22 acquires the voice detected by the microphone 12. The voice acquisition unit 22 acquires voice such as the voice spoken by a user present in the vicinity of the information terminal device 10.

[0036] The operation reception unit 23 acquires an operation signal corresponding to an operation on the operation unit 13 from the operation unit 13. The operation reception unit 23 outputs a control signal corresponding to the acquired operation signal to each unit.

[0037] The operation accepting unit 23 accepts operations on the display screen of the display unit 17. For example, the operation accepting unit 23 accepts an operation to move the user's own avatar in the virtual office on the display screen of the display unit 17. For example, the operation accepting unit 23 accepts an operation to overlap or place the user's own avatar in the virtual office next to another avatar on the display screen of the display unit 17.

[0038] The application control unit 24 controls the operation of various applications executed on the information terminal device 10. The applications executed by the application control unit 24 may cooperate with an application control unit 42 of the server device 30, which will be described later, to control the various applications.

[0039] The application control unit 24 causes a display control unit 27 (described later) to display on the display unit 17 a display related to an application running on the information terminal device 10. The display related to the application is a display screen such as a window or an icon that shows the display related to the application to the user of the information terminal device 10.

[0040] When the user of the information terminal device 10 is logged in to the virtual office, the application control unit 24 makes the virtual office executed by the application control unit 42 of the server device 30 available to the information terminal device 10 .

[0041] When a virtual office application is being executed on the information terminal device 10, the application control unit 24 shares information using the display control unit 27 and the audio output control unit 28 based on an information sharing range set around an avatar corresponding to the user of the information terminal device 10. Furthermore, when a video call with an avatar corresponding to another user is started while the virtual office application is being executed on the information terminal device 10, the application control unit 24 performs the video call using the display control unit 27 and the audio output control unit 28.

[0042] The user information analysis unit 25 analyzes the user information of the information terminal device 10. The user information analysis unit 25 recognizes people based on the video captured by the camera 11 acquired by the video acquisition unit 21, and analyzes facial expressions, facial movements, and movements of the upper body including the hands and arms within the range recognized as the person.

[0043] The storage unit 26 stores information such as the calculation contents and programs of the terminal control device 20. The storage unit 26 is configured with, for example, a RAM, a main storage device such as a ROM, and a storage device such as an SSD or HDD (Hard Disk Drive).

[0044] The storage unit 26 stores programs and setting information relating to applications used in the information terminal device 10, such as a virtual office application.

[0045] The following describes the information sharing range and information sharing set for avatars in a virtual office. In the virtual office, an information sharing range is set around each avatar. The information sharing range is the range in which information can be shared with users of other avatars, in other words, the range in which communication is possible, and the range in which information can be shared between multiple users who are included in each information sharing range. One example of information sharing within the information sharing range is mutual sharing of audio. In other words, the information sharing range is the range in which audio, such as the voice of a user of each avatar, can be heard by users of other avatars. Furthermore, the information sharing range is the range in which audio, such as the voice of a user of each avatar, can be heard by users of other avatars. Information that can be shared within the information sharing range is not limited to audio, but also includes other information such as video and text. In this embodiment, the information sharing range is described as an example in which audio is shared.

[0046] For example, when a second avatar enters the information sharing range of a first avatar, the second user of the second avatar can hear the voice of the first user of the first avatar. Furthermore, when a second avatar enters the information sharing range of a first avatar and a first avatar enters the information sharing range of the second avatar, two-way voice sharing occurs, allowing each user to sense the other's presence and enabling communication without any special operations, enabling conversation, etc. Therefore, the information sharing range can also be rephrased as the voice sharing range. Specifically, when a first avatar and a second avatar are within each other's information sharing range, the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the first user of the first avatar can be heard in the information terminal device 10 used by the second user, and the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the second user of the second avatar can be heard in the information terminal device 10 used by the first user.

[0047] It is also possible to set up a space that allows information sharing among all avatars within a specific range, such as a conference room set up in a virtual office. Outside of such a set space, the information sharing range is set for each avatar.

[0048] The information sharing range of each avatar is set to an initial size and shape when the user starts using the virtual office. The size and shape of the information sharing range of each avatar may be changeable by a user operation.

[0049] The display control unit 27 controls the display of various images on the display unit 17. The display control unit 27 controls the display of an image of the virtual office on the display unit 17. In this embodiment, when the information to be shared includes an image or text, the display control unit 27 outputs the image or text to the display unit 17. Furthermore, when a video call becomes possible, the display control unit 27 outputs an image of the user's virtual office avatar to the display unit 17. In other words, a video call is communication using video footage of the user.

[0050] The audio output control unit 28 controls the output of audio in the audio output unit 18. When information sharing is taking place, the audio output control unit 28 causes the audio output unit 18 to output the speech and the like of the user of another avatar who is present within the information sharing range in the virtual office. Furthermore, when a video call becomes possible, the audio output control unit 28 causes the audio output unit 18 to output the speech and the like of the user who is the other party in the video call.

[0051] The communication control unit 29 controls the communication unit 19 to transmit and receive data such as video and audio to and from the server device 30, and to transmit and receive operation status information of the application. More specifically, the communication control unit 29 controls, for example, the communication unit 19 to receive video and audio of the virtual office from the server device 30. The communication control unit 29 controls, for example, the communication unit 19 to transmit to the server device 30, video and audio of the user facing the display unit 17 of the information terminal device 10, acquired by the video acquisition unit 21 and the audio acquisition unit 22. The operation status information of the application includes, for example, in the case of a virtual office application, the operation of an avatar by the operation unit 13, and the operation of starting and ending a video call.

[0052] (Server Device) Fig. 3 is a block diagram showing an example of the configuration of a server device according to the first embodiment. The server device 30 is a server device of a company to which the users of the information terminal devices 10 belong, or a server device of a business entity that provides a virtual office service, and may be a general-purpose server device, including a distributed type. The server device 30 constructs and controls a virtual space such as a virtual office. The server device 30 includes a communication unit 31 and a server control device 40.

[0053] The communication unit 31 is a communication unit for performing wired communication or wireless communication. The communication unit 31 performs communication using a communication method such as Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication of the communication unit 31 is controlled by the communication control unit 41. The communication unit 31 communicates data such as video, audio, and application operation status information with multiple information terminal devices 10.

[0054] (Server control device) The server control device 40 controls each unit of the server device 30. The server control device 40 has, for example, a control device such as a CPU or MPU, and a storage device such as a RAM or ROM. The server control device 40 may be realized by, for example, an integrated circuit such as an ASIC or FPGA. The server control device 40 may be realized by a combination of hardware and software. The server control device 40 executes a program that controls the operation of the server device 30 according to the present disclosure.

[0055] The server control device 40 includes a communication control unit 41, an application control unit 42, and a storage unit 49 as functional blocks realized by circuits and programs to be executed.

[0056] The communication control unit 41 controls the communication unit 31 to transmit and receive data including video and audio to and from the information terminal device 10, and to transmit and receive operation status information of applications, etc. The communication control unit 41, for example, controls the communication unit 31 to receive from the information terminal device 10 video captured by the camera 11 of the information terminal device 10 and audio collected by the microphone 12 of the information terminal device 10. The communication control unit 41, for example, controls the communication unit 31 to transmit video and audio of the virtual office to the information terminal device 10. The operation status information of the application is, for example, configuration information of the virtual office application.

[0057] The application control unit 42 controls the operation of various applications running on the server device 30. For example, the application control unit 42 controls the operation of a virtual office application. For example, the application control unit 42 controls the operation of an application used for online communication.

[0058] The application control unit 42 transmits the configuration of the application running on the server device 30 to the information terminal device 10 that is executing the application via the communication control unit 41. Specifically, the application control unit 43 transmits information indicating the configuration of the virtual office running on the server device 30 to the information terminal device 10 that is logged in to the virtual office.

[0059] The storage unit 49 stores information such as the contents of calculations and programs of the server control device 40. The storage unit 49 is configured with, for example, a RAM, a main storage device such as a ROM, and a storage device such as an SSD or HDD.

[0060] The storage unit 49 stores information indicating a virtual office configured in a virtual space. For each user who logs in to the virtual office, the storage unit 49 stores information indicating an avatar, the information terminal device 10 associated with the avatar, and the account user of the information terminal device 10. The storage unit 49 stores information indicating the layout of objects that configure the virtual office, such as the sections that configure the virtual office and virtual desks.

[0061] (Information Processing Device) An example of the configuration of an information processing device according to the first embodiment will be described with reference to FIG. 4 . FIG. 4 is a block diagram showing an example of the configuration of an information processing device according to the first embodiment. The information processing device 50 is a conceptual device realized by the cooperation of either or both of the server device 30 and a plurality of information terminal devices 10 used in the virtual office. Either the information terminal devices 10 or the server device 30 may be the information processing device 50, or the virtual space control system 1 including the information terminal devices 10 and the server device 30 may be the information processing device 50. Each unit of the information processing device 50 is realized by the cooperation of either or both of the terminal control device 20 and the server control device 40.

[0062] The information processing device 50 changes the state in which user information is reflected in the avatar corresponding to the user depending on whether it detects that the user is in a state where communication with other users is possible in the virtual office or whether it does not detect that the user is in a state where communication with other users is possible in the virtual office. Specifically, when the information processing device 50 detects that the user is in a state where communication with other users is possible in the virtual office, it reflects the user information in the avatar corresponding to the user, and when it does not detect that the user is in a state where communication with other users is possible in the virtual office, it does not reflect the user information in the avatar.

[0063] 4, the information processing device 50 includes a detection unit 51, a user information acquisition unit 52, a reflection processing unit 53, a display control unit 58, and an audio output control unit 59. For example, the detection unit 51 and the reflection processing unit 53 may be realized by the server control device 40, the user information acquisition unit 52 may be realized by cooperation between the terminal control device 20 and the server control device 40, and the display control unit 58 and the audio output control unit 59 may be realized by the terminal control device 20, but this is not limitative.

[0064] The detection unit 51 detects that the user is in a state where communication with other users is possible in the virtual office. The detection unit 51 detects that the state where communication with other users is possible has ended (released). The detection unit 51 detects that the user is in a state where communication with other users is possible in the virtual office and that the state where communication is possible has ended, based on information from the application control unit 24 of the information terminal device 10 or the application control unit 42 of the server device 30.

[0065] The detection unit 51 detects the positions of a first avatar corresponding to a first user and a second avatar corresponding to a second user in the virtual office. More specifically, the detection unit 51 detects, for example, the first avatar that has entered the information sharing range of the second avatar in the virtual office. The detection unit 51 detects, for example, the first avatar that has left the information sharing range of the second avatar in the virtual office.

[0066] When the information processing device 50 is implemented in the information terminal device 10, the detection unit 51 only needs to detect the first user who is logged in to the information terminal device 10. When the information processing device 50 is implemented in the server device 30, the detection unit 51 only needs to detect all users.

[0067] The user information acquisition unit 52 acquires user information from the user information analysis unit 25 of the information terminal device 10. The user information acquisition unit 52 acquires the results of analysis by the user information analysis unit 25 of the information terminal device 10 as user information. Specifically, the user information is information indicating the user's facial expressions, user actions, etc.

[0068] The reflection processing unit 53 performs a process of reflecting user information in an avatar corresponding to the user. When the reflection processing unit 53 detects that the user is in a state where communication with other users is possible in the virtual office, the reflection processing unit 53 reflects the user information in the avatar corresponding to the user, and when the reflection processing unit 53 does not detect that the user is in a state where communication with other users is possible in the virtual office, the reflection processing unit 53 does not reflect the user information in the avatar.

[0069] The following describes the process performed by the reflection processing unit 53 to reflect the user information acquired by the user information acquisition unit 52 in an avatar. The reflection processing unit 53, for example, displays the facial expression indicated by the user information on the face of the avatar so that the facial expression indicated by the user information can be seen. The reflection processing unit 53, for example, displays a video of the user's face on the face of the avatar. The reflection processing unit 53, for example, displays the facial expression of the user reflected on the face of the avatar. The reflection processing unit 53, for example, displays the body of the avatar so that the movements indicated by the user information can be seen. The reflection processing unit 53, for example, displays the movements of the avatar's hands, arms, etc. as if they were the same as the movements of the user's hands and arms.

[0070] When user information is reflected, the avatar displays facial expressions and movements, or the facial expressions and movements change. When user information is not reflected, the avatar does not display facial expressions and movements, or the facial expressions and movements do not change from their initial states. The initial state of facial expressions is, for example, a blank expression or a smile. The initial state of movements is, for example, a state in which the avatar is not performing a specific movement.

[0071] The display control unit 58 causes the display units 17 of the multiple information terminal devices 10 to display images showing various types of information, such as a virtual office screen.

[0072] When a video call becomes possible, the display control unit 58 causes the first information terminal device 10 in which the first user is logged in to the virtual office and the second information terminal device 10 in which the second user is logged in to the virtual office to mutually display images of faces, etc. More specifically, the display control unit 58 causes the display unit 17 of the first information terminal device 10 to output an image of the second user captured by the camera 11 of the second information terminal device 10, and causes the display unit 17 of the second information terminal device 10 to output an image of the first user captured by the camera 11 of the first information terminal device 10.

[0073] When the video call is stopped, the display control unit 58 stops the video sharing between the first user and the second user.

[0074] The display screen will be described with reference to Figures 5 to 8. As shown in Figures 5 to 8, the display unit 17 of the information terminal device 10 displays a display screen W1 that displays a virtual office screen by a virtual office application, for example.

[0075] FIG. 5 is a diagram illustrating an example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 6 is a diagram illustrating another example of a virtual office display screen displayed on the display screen of an information terminal device. In FIG. 5, a first avatar A1 corresponding to a first user and a second avatar A2 corresponding to a second user are mutually present within information sharing ranges C1 and C2, enabling communication such as voice sharing. The first avatar A1 and the second avatar A2 do not display facial expressions or movements. In other words, the first avatar A1 and the second avatar A2 do not display facial expressions or movements. In FIG. 6, the first avatar A1 and the second avatar A2 are mutually present within information sharing ranges C1 and C2, enabling communication such as voice sharing. Therefore, the first avatar A1 reflects the user information of the first user, and the second avatar A2 reflects the user information of the second user. Specifically, the first avatar A1 reflects the facial expression of the first user, and the second avatar A2 reflects the facial expression of the second user.

[0076] FIG. 7 is a diagram showing another example of a virtual office display screen displayed on the display screen of an information terminal device. FIG. 8 is a diagram showing another example of a virtual office display screen displayed on the display screen of an information terminal device. FIGS. 7 and 8 show a state in which, from the state shown in FIG. 5 , an operation to overlay a second avatar on the first avatar A1 has been performed, thereby enabling communication via video call. The display screen W1 displays a video call screen WS1 displaying an image of the first user and a video call screen WS2 displaying an image of the second user. Similar to FIG. 5 , FIG. 7 shows that the first avatar A1 and the second avatar A2 do not display facial expressions or movements. Similar to FIG. 6 , FIG. 8 shows that the first avatar A1 reflects the user information of the first user, and the second avatar A2 reflects the user information of the second user.

[0077] When a second avatar enters the information sharing range of a first avatar, the audio output control unit 59 enables the second user to share the audio of the first user. Furthermore, when the second avatar is within the information sharing range of the first avatar and the first avatar is within the information sharing range of the second avatar, the second user can share the audio of the first user, and the first user can share the audio of the second user. In other words, the first and second users can share audio, enabling them to have audio conversations. More specifically, when audio sharing is enabled, the audio output control unit 59 enables the information terminal device 10 used by the first user corresponding to the first avatar and the second information terminal device 10 where the second user corresponding to the second avatar is logged in to the virtual office to hear each other's voices. More specifically, the audio output control unit 59 outputs the audio of the second user picked up by the microphone 12 of the second information terminal device 10 from the audio output unit 18 of the first information terminal device 10, and outputs the audio of the first user picked up by the microphone 12 of the first information terminal device 10 from the audio output unit 18 of the second information terminal device 10.

[0078] When the first avatar and the second avatar are no longer included in the information sharing range due to, for example, the movement of the avatar, the audio output control unit 59 stops audio sharing between the first user and the second user.

[0079] (Virtual space control method) The virtual space control method will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of a virtual space control method according to the first embodiment. The processing shown in Fig. 9 is executed while a virtual office is provided by the virtual space control system 1. When the information processing device 50 is implemented in the information terminal device 10, the processing of the flowchart shown in Fig. 9 may be executed for the first user who is logged in to the information terminal device 10. When the information processing device 50 is implemented in the server device 30, the processing of the flowchart shown in Fig. 9 may be executed for all users.

[0080] The information processing device 50 starts displaying the virtual office (step S101). More specifically, the information processing device 50 causes the display control unit 58 to display an image showing the virtual office screen on the display unit 17 of the information terminal device 10. The information processing device 50 proceeds to step S102.

[0081] The information processing device 50 determines whether or not communication with other users is possible in the virtual office (step S102). More specifically, the information processing device 50 detects, via the detection unit 51, whether or not the user is possible to communicate with other users in the virtual office. If the information processing device 50 determines that communication with other users is possible in the virtual office (Yes in step S102), the process proceeds to step S103. If the information processing device 50 does not determine that communication with other users is possible in the virtual office (No in step S102), the process proceeds to step S106.

[0082] When it is determined that communication with other users is possible in the virtual office (Yes in step S102), the information processing device 50 causes the reflection processing unit 53 to reflect and display the user information on the avatar (step S103). In other words, the information processing device 50 starts reflecting the user information on the avatar. More specifically, the information processing device 50 causes the reflection processing unit 53 to display, for example, the facial expression indicated by the user information on the avatar's face. The information processing device 50 causes the reflection processing unit 53 to display, for example, the body of the avatar's body so that the movement indicated by the user information can be seen. The information processing device 50 proceeds to step S104.

[0083] The information processing device 50 determines whether the state in which communication with other users is possible has been released (step S104). If the information processing device 50 determines that the state in which communication with other users is possible has been released (Yes in step S104), the information processing device 50 proceeds to step S105. If the information processing device 50 does not determine that the state in which communication with other users is possible has been released (No in step S104), the information processing device 50 executes the process of step S104 again.

[0084] When it is determined that the state in which communication with other users is possible has been released (Yes in step S104), the information processing device 50 causes the reflection processing unit 53 to cancel the display of the avatar that reflects the user information (step S105). More specifically, the information processing device 50 causes the reflection processing unit 53 to cancel the display that reflects, for example, the facial expression indicated by the user information, which is reflected on the face of the avatar. The information processing device 50 causes the reflection processing unit 53 to cancel the display that reflects, for example, the body of the avatar, which is indicated by the movement indicated by the user information. The information processing device 50 proceeds to step S106.

[0085] The information processing device 50 determines whether to end use of the virtual office (step S106). For example, when the information processing device 50 detects an operation to end use of the virtual office, the information processing device 50 determines to end use of the virtual office. When the information processing device 50 determines to end use of the virtual office (Yes in step S106), the information processing device 50 ends the processing of this flowchart. When the information processing device 50 does not determine to end use of the virtual office (No in step S106), the information processing device 50 executes the processing of step S102 again.

[0086] (Effects) As described above, in this embodiment, when it is detected that a user is in a state where communication with other users is possible in the virtual office, user information is reflected in the avatar corresponding to the user. However, when it is not detected that a user is in a state where communication with other users is possible in the virtual office, the user information is not reflected in the avatar. According to this embodiment, user information can be reflected in the avatar only when communication with other users is possible in the virtual office. In other words, when communication with other users is not possible in the virtual office, user information is not reflected in the avatar, and therefore no photographing is performed to obtain user information. According to this embodiment, the user is not constantly photographed by a camera, and further, the user's status is not constantly reflected in the avatar, thereby reducing stress. In this way, according to this embodiment, user information can be reflected in the avatar when appropriate.

[0087] Second Embodiment A virtual space control system 1 according to a fourth embodiment will be described using Figures 10 and 11. Figure 10 is a block diagram showing an example of the configuration of an information processing device according to the second embodiment. Figure 11 is a flowchart showing an example of a virtual space control method according to the second embodiment. In the following description, components similar to those of the virtual space control system 1 of the first embodiment are given the same or corresponding reference numerals, and detailed description thereof will be omitted. The virtual space control system 1 differs from the first embodiment in that the information processing device 50A includes an utterance detection unit 54A.

[0088] The information processing device 50A is a conceptual device realized by the cooperation of either or both of the server device 30 and the plurality of information terminal devices 10 used in the virtual office.

[0089] When the user is in a state where he or she can communicate with other users in the virtual space and speech from the user or other users is detected, the information processing device 50A reflects the user information in the avatar corresponding to the user, and when the user is in a state where he or she can communicate with other users in the virtual space and speech from the user or other users is not detected, the information processing device 50A does not reflect the user information in the avatar corresponding to the user.

[0090] The speech detection unit 54A detects speech from the user and other users. The speech detection unit 54A detects speech from the user or other users, particularly when the user is in a state where communication with other users is possible in the virtual office. The speech detection unit 54A detects, for example, a state where the user or other users are speaking continuously or alternately. In other words, the speech detection unit 54A detects that the user and other users are communicating or conversing through speech.

[0091] The speech detection unit 54A may detect that a user and another user are speaking by detecting that the speech acquired by the speech acquisition unit 22 of the information terminal device 10 used by the user and another user includes a continuous speech sound. The speech detection unit 54A detects speech from the speech acquired by the speech acquisition unit 22 using any known method, such as feature analysis or phoneme extraction by acoustic analysis.

[0092] The reflection processing unit 53 reflects the user information in the avatar corresponding to the user when the user is in a state where he or she can communicate with other users in the virtual space and when speech from the user or other users is detected, and does not reflect the user information in the avatar corresponding to the user when the user is in a state where he or she can communicate with other users in the virtual space and when speech from the user or other users is not detected.

[0093] (Virtual Space Control Method) A virtual space control method will be described with reference to Fig. 11. The processes of steps S121, S122, S124, S126, and S127 in Fig. 11 are the same as those of steps S101, S102, S103, S105, and S106 in Fig. 9.

[0094] The information processing device 50A determines whether the user or another user has spoken (step S123). More specifically, the information processing device 50A detects, by the speech detection unit 54A, speech from the user or another user when the user is in a state where communication with another user is possible in the virtual office. If the information processing device 50A determines that the user or another user has spoken (Yes in step S123), the process proceeds to step S124. If the information processing device 50A does not determine that the user or another user has spoken (No in step S123), the process proceeds to step S127.

[0095] The information processing device 50A determines whether the speech has ended or whether the state in which communication with other users is possible has been released (step S125). If the information processing device 50A determines that the speech has ended or that the state in which communication with other users is possible has been released (Yes in step S125), the information processing device 50A proceeds to step S126. If the information processing device 50A does not determine that the speech has ended and does not determine that the state in which communication with other users is possible has been released (No in step S125), the information processing device 50A executes the process of step S125 again.

[0096] (Effect) As described above, in this embodiment, when a user is in a state where they can communicate with other users in the virtual space and when speech from the user or other users is detected, the user information is reflected in the avatar corresponding to the user, and when a user is in a state where they can communicate with other users in the virtual space and when speech from the user or other users is not detected, the user information is not reflected in the avatar corresponding to the user. According to this embodiment, even if communication is possible, if no speech is being made, the user information is not reflected in the avatar, so it is possible to avoid taking a photo to obtain the user information.

[0097] Third Embodiment A virtual space control system 1 according to a fourth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of a virtual space control method according to the third embodiment. The virtual space control system 1 differs from the first embodiment in the processing performed by the detection unit 51 of the information processing device 50.

[0098] When the user is in a state where he or she can communicate with other users in the virtual space using only voice, the information processing device 50 reflects the user information in an avatar corresponding to the user, and when the user is in a state where he or she can communicate with other users in the virtual space using video footage of the user, the information processing device 50 does not reflect the user information in an avatar corresponding to the user.

[0099] A state in which voice-only communication is possible is, for example, a state in which voice sharing is possible as shown in Figures 5 and 6. In this case, as shown in Figure 6, user information is reflected in an avatar corresponding to the user.

[0100] A state in which communication using captured video of a user is possible is, for example, a state in which a video call is possible, as shown in Figures 7 and 8. In this case, as shown in Figure 7, user information is not reflected in the avatar corresponding to the user. In this case, the parties in the communication can mutually recognize the user's facial expressions and movements through the video video, so there is no need to reflect the information in the avatar. Furthermore, by reflecting the information in the avatar, there is no need to reflect the user information during the video call in the avatar that is visible to users in the virtual office other than the parties in the video call.

[0101] The detection unit 51 detects that the user and other users are in a state where they can communicate with each other by voice only in the virtual office. The detection unit 51 detects that the state where the user and other users can communicate with each other by voice only has ended (released). The detection unit 51 detects that the user is in a state where the user and other users can communicate with each other by voice only in the virtual office and that the state where the user can communicate with each other by voice only has ended, based on information from the application control unit 24 of the information terminal device 10 or the application control unit 42 of the server device 30.

[0102] The detection unit 51 may detect that the user and another user are able to communicate with each other via video call in the virtual office. The detection unit 51 may also detect that the state in which the user and another user are able to communicate with each other via video call has ended (released).

[0103] The reflection processing unit 53 reflects the user information in an avatar corresponding to the user when the user is in a state where he or she can communicate with other users in the virtual space using only voice, and does not reflect the user information in an avatar corresponding to the user when the user is in a state where he or she can communicate with other users in the virtual space using video footage of the user.

[0104] (Virtual Space Control Method) A virtual space control method will be described with reference to Fig. 12. The processes of steps S141, S142, S144, S146, and S147 in Fig. 12 are the same as those of steps S101, S102, S103, S105, and S106 in Fig. 9.

[0105] The information processing device 50 determines whether the communication is voice-only (step S143). If the information processing device 50 determines that the communication is voice-only (Yes in step S143), the process proceeds to step S144. If the information processing device 50 does not determine that the communication is voice-only (No in step S143), the process proceeds to step S147.

[0106] The information processing device 50 determines whether the state in which communication is possible only by voice has been released (step S145). If the information processing device 50 determines that the state in which communication is possible only by voice has been released (Yes in step S145), the information processing device 50 proceeds to step S146. If the information processing device 50 does not determine that the state in which communication is possible only by voice has been released (No in step S145), the information processing device 50 executes the process of step S145 again.

[0107] (Effects) As described above, in this embodiment, when a user is in a state where they can communicate with other users in a virtual space using only audio, the user information is reflected in an avatar corresponding to the user, and when a user is in a state where they can communicate with other users in a virtual space using video footage of the user, the user information is not reflected in an avatar corresponding to the user. In this embodiment, when video communication is being performed, the user and the other users can mutually recognize the user's facial expressions or movements through a video call. In this embodiment, only when communication is being performed using only audio, the user information is reflected in the avatar, and video footage can be taken to obtain the user information.

[0108] [Fourth Embodiment] A virtual space control system 1 according to a fourth embodiment will be described with reference to Figures 13 and 14. Figure 13 is a diagram showing an example of a display screen of a virtual office displayed on the display screen of an information terminal device. Figure 14 is a flowchart showing an example of a virtual space control method according to the fourth embodiment. In the following description, components similar to those of the virtual space control system 1 of the first embodiment are given the same or corresponding reference numerals, and detailed description thereof will be omitted. The virtual space control system 1 differs from the first embodiment in the processing performed by the reflection processing unit 53 of the information processing device 50.

[0109] In this embodiment, user information is information relating to the operation of the information terminal device 10 used by the user. Information relating to the operation of the information terminal device 10 used by the user includes, for example, applications running on the information terminal device 10 and the operation status of the applications. Information relating to the operation of the information terminal device 10 is acquired and analyzed by, for example, the application control unit 24 and transmitted to the server device 30. For example, when the user frequently operates the information terminal device 10, information indicating that the user is busy is transmitted to the server device 30 as information relating to the operation of the information terminal device 10.

[0110] The user information acquisition unit 52 acquires user information including applications running on the information terminal device 10 and operation states of the applications from the user information analysis unit 25 of the information terminal device 10 .

[0111] When the reflection processing unit 53 does not detect that the user is communicating with other users in the virtual office, it reflects information regarding the operation of the information terminal device 10 used by the user in the avatar corresponding to the user, and when it detects that the user is communicating with other users in the virtual office, it does not reflect information regarding the operation of the information terminal device 10 used by the user in the first avatar.

[0112] The following describes the process of reflecting information related to the operation of the information terminal device 10 in an avatar by the reflection processing unit 53. For example, the reflection processing unit 53 displays the avatar as an avatar operating an electronic device such as the information terminal device 10.

[0113] An example of a virtual office display screen displayed on the display screen of an information terminal device will be described using Figure 13. In Figure 13, the first avatar A1 and the second avatar are not in the information sharing range C1 or the information sharing range C2, and voice sharing and the like are not possible. The user information of the first user is reflected in the first avatar A1. Specifically, the first avatar A1 indicates that the third user is working using the information terminal device 10, indicating that the third user is "busy."

[0114] 13 , if the first user frequently operates the information terminal device 10, the first avatar A1 may reflect user information regarding the operation of the information terminal device 10 used by the first user. In other words, if it is not determined that the first user frequently operates the information terminal device 10, the user information is not reflected in the first avatar A1 in Fig. 13 . Also, in Fig. 13 , the first avatar A1 may display user information regarding the operation of the information terminal device 10 used by the first user, such as user information indicating that the first user frequently operates the information terminal device 10 or user information indicating that the first user rarely operates the information terminal device 10, based on the operation of the information terminal device 10 by the first user.

[0115] (Virtual Space Control Method) A virtual space control method will be described with reference to Fig. 14. The processes of steps S201, S203, S205, and S207 in Fig. 14 are the same as those of steps S101, S102, S104, and S106 in Fig. 9.

[0116] The information processing device 50 causes the reflection processing unit 53 to reflect and display information related to the operation of the information terminal device 10 in the avatar (step S202). More specifically, the information processing device 50 causes the reflection processing unit 53 to display the avatar as an avatar that appears to be operating an electronic device such as the information terminal device 10. The information processing device 50 proceeds to step S203.

[0117] If it is determined that the user is communicating with another user in the virtual office (Yes in step S203), the information processing device 50 causes the reflection processing unit 53 to cancel the display of the avatar that reflects information about the operation of the information terminal device 10 (step S204). More specifically, the information processing device 50 causes the reflection processing unit 53 to cancel the display of the avatar that appears to be operating an electronic device, for example, and displays an avatar that does not reflect user information. The information processing device 50 proceeds to step S205.

[0118] If it is determined that communication with the other user has ended (Yes in step S205), the information processing device 50 causes the reflection processing unit 53 to reflect information about the operation of the information terminal device 10 in the avatar and display it (step S206). The information processing device 50 proceeds to step S207.

[0119] (Effects) As described above, in this embodiment, when it is not detected that the user is communicating with other users in the virtual office, information regarding the operation of the information terminal device 10 used by the user is reflected in the avatar, whereas when it is detected that the user is communicating with other users in the virtual office, information regarding the operation of the information terminal device 10 used by the user is not reflected in the avatar. According to this embodiment, when the user is communicating with other users in the virtual office, it is possible to prevent user information from being reflected in the avatar. According to this embodiment, the user is not constantly photographed by a camera, which can reduce stress. In this way, according to this embodiment, it is possible to reflect user information in the avatar when it is appropriate for the avatar.

[0120] According to this embodiment, other users can check the information about the operation of the information terminal device 10 reflected in the avatar corresponding to the user, thereby determining the timing to communicate with the user.

[0121] Fifth Embodiment A virtual space control system 1 according to a fifth embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of a virtual space control method according to the fifth embodiment. The virtual space control system 1 differs from the first and fourth embodiments in the processing performed by the reflection processing unit 53 of the information processing device 50.

[0122] When the reflection processing unit 53 does not detect that the user is communicating with other users in the virtual office, it does not reflect user information in the avatar, but reflects information on the operation of the information terminal device 10 used by the user in the avatar.

[0123] (Virtual Space Control Method) The virtual space control method will be described with reference to Fig. 15. The processes of steps S301, S303, S305 to S307, and S309 in Fig. 15 are the same as those of steps S101, S102, S103 to S105, and S106 in Fig. 9. The processes of steps S302, S304, and S308 in Fig. 15 are the same as those of steps S202, S204, and S206 in Fig. 14.

[0124] (Effects) As described above, in this embodiment, when it is not detected that a user is communicating with another user in the virtual office, user information is not reflected in the avatar, and information related to the operation of the information terminal device 10 used by the user is reflected in the avatar. According to this embodiment, other users can check the user information reflected in the user's avatar to confirm the user's communication status. According to this embodiment, other users can check the information related to the operation of the information terminal device 10 reflected in the avatar corresponding to the user, thereby determining the timing to communicate with the user.

[0125] The components of the virtual space control system shown in the figure are conceptual and functional, and do not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

[0126] The configuration of the virtual space control system is realized, for example, as software, by a program loaded into memory. In the above embodiment, the functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms, using only hardware, only software, or a combination of both.

[0127] The components described above include those that can be easily imagined by a person skilled in the art and those that are substantially the same. Furthermore, the configurations described above can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the configurations are possible within the scope of the gist of the present invention.

[0128] The virtual space control system and virtual space control method of the present disclosure can be used, for example, in a communication system that performs communication using avatars in a virtual space such as a virtual office.

[0129] REFERENCE SIGNS LIST 1 Virtual space control system 10 Information terminal device 11 Camera 12 Microphone 13 Operation unit 17 Display unit 18 Audio output unit 19 Communication unit 20 Terminal control device 21 Video acquisition unit 22 Audio acquisition unit 23 Operation acceptance unit 24 Application control unit 25 User information analysis unit 26 Storage unit 27 Display control unit 28 Audio output control unit 29 Communication control unit 30 Server device 31 Communication unit 40 Server control device 41 Communication control unit 42 Application control unit 49 Storage unit 50 Information processing device 51 Detection unit 52 User information acquisition unit 53 Reflection processing unit 58 Display control unit 59 Audio output control unit

Claims

1. A virtual space control system that causes an information terminal device to display a virtual space and displays an avatar corresponding to a user of the information terminal device in the virtual space, wherein an information processing device provided in the virtual space control system, when it detects that the user is in a state where he or she can communicate with other users in the virtual space, reflects user information that is information about the user in the avatar corresponding to the user, and when it does not detect that the user is in a state where he or she can communicate with other users in the virtual space, does not reflect the user information in the avatar.

2. The virtual space control system of claim 1, wherein the information processing device detects speech from the user and the other users, and when the user is in a state where he or she can communicate with the other users in the virtual space and speech from the user or the other users is detected, reflects the user information in an avatar corresponding to the user, and when the user is in a state where he or she can communicate with the other users in the virtual space and speech from the user or the other users is not detected, does not reflect the user information in the avatar corresponding to the user.

3. The virtual space control system of claim 1, wherein the information processing device reflects the user information in an avatar corresponding to the user when the user is in a state where they can communicate with the other users in the virtual space using only voice, and does not reflect the user information in an avatar corresponding to the user when the user is in a state where they can communicate with the other users in the virtual space using video footage of the user.

4. The virtual space control system of claim 1, wherein the information processing device, when not reflecting the user information in the avatar corresponding to the user, reflects information regarding the operation of the information terminal device used by the user in the avatar.

5. A virtual space control method executed by a virtual space control system that causes an information terminal device to display a virtual space and that causes an avatar corresponding to a user of the information terminal device to be displayed in the virtual space, wherein, when it is detected that the user is in a state where they can communicate with other users in the virtual space, user information that is information about the user is reflected in the avatar corresponding to the user, and when it is not detected that the user is in a state where they can communicate with other users in the virtual space, the user information is not reflected in the avatar.

Images