Virtual space control system and virtual space control method
The virtual space control system addresses communication issues by determining avatar placement based on user state information, enhancing communication by placing avatars in proximity based on emotional, concentration, and speech volume similarities.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- JVC KENWOOD CORP
- Filing Date
- 2024-12-19
- Publication Date
- 2026-07-01
Smart Images
Figure 2026109023000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a virtual space control system and a virtual space control method.
Background Art
[0002] In recent years, in virtual spaces such as virtual offices, services have been provided that enable users to communicate with other users using avatars. In this type of virtual space, the placement positions of each avatar may be arbitrary or predetermined. In particular, in a virtual office, when avatars are placed in sections such as by department or by project, the placement positions are often predetermined within the section or are arbitrary. Patent Document 1 discloses placing an avatar at a position corresponding to the position of a user in the real space.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, even when the positions where each avatar is placed are arbitrary or predetermined, the placement of the avatars does not reflect the state and situation of each user, so there may be problems in business operations and communication between users.
[0005] The object of the present disclosure is to appropriately determine the placement positions of each avatar in a virtual space and enable appropriate communication between users.
Means for Solving the Problems
[0006] To solve the above-mentioned problems and achieve the objectives, the virtual space control system according to this disclosure is a virtual space control system that causes an information terminal device to display a virtual space, wherein the information processing device provided in the virtual space control system acquires information indicating the state of a user using the virtual space through the information terminal device, and determines the placement position of an avatar corresponding to the user in the virtual space based on the user's state.
[0007] The virtual space control method relating to this disclosure is a virtual space control method executed by a virtual space control system that causes an information terminal device to display a virtual space, wherein the virtual space control system acquires information indicating the state of a user using the virtual space through the information terminal device, and determines the placement position of an avatar corresponding to the user in the virtual space based on the user's state. [Effects of the Invention]
[0008] According to this disclosure, it will be possible to appropriately determine the placement of avatars in virtual space and enable appropriate communication between users. [Brief explanation of the drawing]
[0009] [Figure 1] Figure 1 is a schematic diagram showing an example configuration of a virtual space control system according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an information terminal device according to an embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of a server device according to the embodiment. [Figure 4] Figure 4 is a block diagram showing an example configuration of an information processing device according to the embodiment. [Figure 5] Figure 5 is an example of a score chart showing the user's emotions as a state of the user and their level. [Figure 6] Figure 6 is an example of a score chart showing the level of user concentration as a state of the user. [Figure 7]Figure 7 is an example of a score chart showing the amount of speech a user utters and its level as a state of the user. [Figure 8] Figure 8 is a flowchart showing an example of a virtual space control method according to an embodiment. [Figure 9] Figure 9 shows an example of the display screen of an information terminal device showing each avatar placed in the virtual office. [Figure 10] Figure 10 is an example of a score chart showing the emotions and levels of each user corresponding to each avatar placed in the virtual office. [Modes for carrying out the invention]
[0010] Embodiments of the virtual space control system and virtual space control method according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.
[0011] (Virtual Space Control System) Figure 1 is a schematic diagram showing an example configuration of the virtual space control system according to this embodiment. The virtual space control system 1 controls a virtual office configured in the virtual space. In this embodiment, a virtual office is used as an example to explain the virtual space, but it is not limited to a virtual office and can be applied to various services that use avatars for communication in various virtual spaces.
[0012] In the following explanation, "user" may be referred to as the "first user," and "other users" may be referred to as the "second user." Furthermore, the information terminal device used by the first user to log into the virtual office may be referred to as the first information terminal device, and the information terminal device used by the other user (the second user) to log into the virtual office may be referred to as the second information terminal device. The first information terminal device is any information terminal device logged into the virtual office, while the second information terminal device is a different information terminal device from the first information terminal device, but is any information terminal device logged into the same virtual office as the first information terminal device.
[0013] The outline of the virtual office controlled by the virtual space control system 1 will be described. The user uses the information terminal device 10 to go to work (log in) to the virtual office with their own account and conduct business. In the virtual office, an avatar associated with the user is set. When the user logs in to the virtual office using the information terminal device 10 described below that the user uses, the avatar is moved to an arbitrary seat or a predetermined seat in the virtual office, and user operations and displays are performed so that the avatar sits down. The user can move the avatar within the virtual office by operating the avatar. In the virtual office, for example, when the distance between avatars in the virtual office is close, or when the avatars are located in the same space such as a conference room in the virtual office, information sharing such as voices can be heard mutually.
[0014] The virtual space control system 1 includes an information terminal device 10 used by each user for business and the like to log in to the virtual office, and a server device 30.
[0015] (Information terminal device) FIG. 2 is a block diagram showing a configuration example of the information terminal device according to the embodiment. The information terminal device 10 is a terminal device used by a user who conducts business in the virtual office. The information terminal device 10 is a terminal device on which the user executes an application for the virtual office. The information terminal device 10 is a terminal device on which the user logs in to the virtual office with their own account when logging in to the virtual office.
[0016] The information terminal device 10 is, for example, an information terminal device such as a general-purpose personal computer, a smartphone, or a tablet terminal. The information terminal device 10 is connected to the server device 30 via a network so as to be capable of data communication.
[0017] The information terminal device 10 logs in to the virtual office based on the account information of the user logged in to the information terminal device 10 and the like.
[0018] The information terminal device 10 includes a camera 11, a microphone 12, an operation unit 13, a display unit 17, an audio output unit 18, a communication unit 19, and a terminal control device 20.
[0019] Camera 11 captures images in the direction facing the information terminal device 10. Camera 11 is positioned to capture images in the direction facing the display unit 17 of the information terminal device 10. In other words, camera 11 captures images in the direction where the user using the information terminal device 10 is located. Camera 11 is positioned, for example, around the display unit 17. Camera 11 outputs the captured image to the image acquisition unit 21 of the terminal control device 20.
[0020] The microphone 12 picks up sounds from the vicinity of the information terminal device 10. The microphone 12 may also pick up sounds coming from the direction opposite to the display unit 17 of the information terminal device 10, and may be provided on a headset connected to the information terminal device 10. In other words, the microphone 12 picks up the voice spoken by the user using the information terminal device 10 and the surrounding sounds. The microphone 12 outputs audio information related to the picked-up voice to the audio acquisition unit 22 of the terminal control device 20.
[0021] The operation unit 13 is an input device capable of receiving various operations for the information terminal device 10. The operation unit 13 outputs an operation signal to the terminal control device 20 corresponding to the received input operation. The operation unit 13 includes, for example, a keyboard, mouse, touch panel arranged on top of the display unit 17, buttons, switches, etc. The operation unit 13 can receive, for example, key input operations, click operations, or touch operations.
[0022] The operation unit 13 can accept operations such as moving the avatar corresponding to the user in the virtual office in any direction on the display screen of the display unit 17. For example, the operation unit 13 can accept operations such as overlapping or placing the user's own avatar in the virtual office next to another avatar on the display screen of the display unit 17.
[0023] By overlaying one's own avatar onto another's avatar, or by positioning them adjacent to each other, users can initiate communication such as video calls or voice calls within the virtual office application. Whether to use video or voice calls may be pre-configured or can be set by the user.
[0024] The operation unit 13 accepts requests for communication with other users via avatars within the virtual office. For example, the operation unit 13 accepts operations such as searching for other users to communicate with within the virtual office on the display screen of the display unit 17, and selecting a user from the search results to request communication.
[0025] The display unit 17 displays various types of images, including text and images. The display unit 17 is a display including, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 17 displays images of a virtual office, etc. In this embodiment, when a video call becomes possible, the display unit 17 outputs the image of the other avatar user making the video call, in other words, the image captured by the camera 11 of the information terminal device 10 used by the other avatar user. The display of the display unit 17 is controlled by the display control unit 27.
[0026] The audio output unit 18 is an audio output device that outputs audio from the virtual office, etc. In this embodiment, when using the virtual office, the audio output unit 18 outputs the audio surrounding the user's avatar in the virtual office. Also, when voice or video calls become possible, the audio output unit 18 outputs the audio of the other avatar user making the voice or video call, in other words, the audio picked up by the microphone 12 of the information terminal device 10 used by the avatar user. The audio output of the audio output unit 18 is controlled by the audio output control unit 28.
[0027] The communication unit 19 is a communication unit for wired or wireless communication. The communication unit 19 communicates using communication methods such as Wi-Fi (registered trademark) or mobile phone lines, and connects to an intranet or the internet. The communication unit 19 is controlled by the communication control unit 29. The communication unit 19 communicates data such as video and audio, or application operation information, with the server device 30. The communication unit 19 also communicates data such as user account information and application operating status information of users logged into the information terminal device 10 with the server device 30.
[0028] (Terminal control device) The terminal control device 20 controls various parts of the information terminal device 10. The terminal control device 20 includes, for example, a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory). The terminal control device 20 executes a program that controls the operation of the information terminal device 10 according to this disclosure. The terminal control device 20 may be implemented by an integrated circuit such as an ASIC or FPGA. The terminal control device 20 loads the stored program into memory and executes the instructions contained in the program. The terminal control device 20 includes internal memory such as the aforementioned RAM, and the internal memory is used for temporary storage of data in the terminal control device 20. The terminal control device 20 may be implemented by a combination of hardware and software.
[0029] The terminal control device 20 includes, as functional blocks realized by circuits and executed programs, a video acquisition unit 21, an audio acquisition unit 22, an operation reception unit 23, an application control unit 24, a storage unit 26, a display control unit 27, an audio output control unit 28, and a communication control unit 29.
[0030] The video acquisition unit 21 acquires the video captured by the camera 11. More specifically, the video acquisition unit 21 acquires the video captured by the camera 11, which shows a user facing the information terminal device 10. The video data acquired by the camera 11 is a moving image.
[0031] The voice acquisition unit 22 acquires the sound detected by the microphone 12. The voice acquisition unit 22 acquires sounds such as the voice spoken by a user in the vicinity of the information terminal device 10.
[0032] The operation reception unit 23 acquires an operation signal from the operation unit 13 corresponding to the operation performed on the operation unit 13. The operation reception unit 23 outputs a control signal corresponding to the acquired operation signal to each unit.
[0033] The operation reception unit 23 receives operations on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to move the user's avatar in the virtual office on the display screen of the display unit 17. For example, the operation reception unit 23 receives operations to overlap or place the user's avatar in the virtual office next to another avatar on the display screen of the display unit 17.
[0034] The operation reception unit 23 receives requests for communication with other users via avatars within the virtual office. For example, the operation reception unit 23 receives requests on the display screen of the display unit 17 to search for other users with whom the user wishes to communicate within the virtual office, and to select a user from the search results to request communication.
[0035] The application control unit 24 controls the operation of various applications running on the information terminal device 10. The applications running on the application control unit 24 may be controlled in cooperation with the virtual office control unit 42 of the server device 30, which will be described later.
[0036] The application control unit 24 causes the display unit 17 to display information related to the application running on the information terminal device 10, via the display control unit 27, which will be described later. Application-related displays include display screens such as windows and icons that show information related to the application to the user of the information terminal device 10.
[0037] When a user of the information terminal device 10 is logged into the virtual office, the application control unit 24 makes the virtual office running in the server device 30's virtual office control unit 42 available to the information terminal device 10.
[0038] When a virtual office application is running on the information terminal device 10, the application control unit 24 performs information sharing using the display control unit 27 and the audio output control unit 28 based on the information sharing range set around the avatar corresponding to the user of the information terminal device 10. In addition, when a virtual office application is running on the information terminal device 10, the application control unit 24 performs a video call using the display control unit 27 and the audio output control unit 28 if a video call is started with an avatar corresponding to another user.
[0039] The memory unit 26 stores information such as the calculation contents and programs of the terminal control device 20. The memory unit 26 is composed of, for example, RAM, a main memory such as ROM, an SSD, an HDD (Hard Disk Drive), and other storage devices.
[0040] The memory unit 26 stores programs and configuration information related to applications used in the information terminal device 10, such as virtual office applications.
[0041] This section describes the information sharing range and information sharing that are set for avatars in a virtual office. In a virtual office, an information sharing range is set around each avatar. The information sharing range is the range in which information can be shared with users of other avatars, and it is the range in which information can be shared among multiple users that are mutually included in the information sharing range. One example of information sharing within the information sharing range is the sharing of voices. In other words, the information sharing range is the range in which the voices of users of other avatars can be heard by users of other avatars. Also, the information sharing range is the range in which users of other avatars can hear the voices of users of other avatars. The information that can be shared within the information sharing range is not limited to voice, but also includes other information such as video and text. In this embodiment, the information sharing range will be described as an example of sharing voices.
[0042] For example, if the second avatar enters the information sharing range of the first avatar, the second user of the second avatar can hear the voice of the first user of the first avatar. Also, if the second avatar enters the information sharing range of the first avatar, and the first avatar enters the information sharing range of the second avatar, two-way voice sharing occurs, allowing them to sense each other's presence and enabling conversation without any special operation. Therefore, the information sharing range can also be called the voice sharing range. Specifically, when the first and second avatars are mutually included in each other's information sharing range, the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the first user of the first avatar becomes audible on the information terminal device 10 used by the second user, and the voice acquired by the voice acquisition unit 22 of the information terminal device 10 used by the second user of the second avatar becomes audible on the information terminal device 10 used by the first user.
[0043] Furthermore, it is possible to create spaces that enable information sharing among all avatars within a specific range, such as a meeting room set up in a virtual office. Outside of such spaces, the information sharing range is set individually for each avatar.
[0044] The information sharing area for each avatar is initially set to a certain size (area) and shape when the virtual office is first used. The size and shape of the information sharing area for each avatar may be changed by user interaction.
[0045] The display control unit 27 controls the display of various types of video on the display unit 17. The display control unit 27 controls the display of video of the virtual office on the display unit 17. In this embodiment, if the information being shared includes video or text, the display control unit 27 outputs the video or text to the display unit 17. Also, when a video call becomes possible, the display control unit 27 outputs the video of the virtual office avatar user, etc., to the display unit 17.
[0046] The audio output control unit 28 controls the audio output of the audio output unit 18. When information sharing is taking place, the audio output control unit 28 causes the audio output unit 18 to output the speech of other avatar users within the information sharing range of the virtual office. Also, when video calls become possible, the audio output control unit 28 causes the audio output unit 18 to output the speech of the user who will be making the video call.
[0047] The communication control unit 29 controls the communication unit 19 to control the transmission and reception of data such as video and audio, and the transmission and reception of application operation status information between the communication control unit 29 and the server device 30. More specifically, the communication control unit 29 controls the communication unit 19 to receive video and audio from the virtual office from the server device 30. The communication control unit 29 controls the communication unit 19 to transmit to the server device 30 the video and audio of the user facing the display unit 17 of the information terminal device 10, which have been acquired by the video acquisition unit 21 and the audio acquisition unit 22. Application operation status information includes, for example, in the case of a virtual office application, the operation of the avatar by the operation unit 13, and the operation of starting and ending video calls.
[0048] (Server device) Figure 3 is a block diagram showing an example configuration of a server device according to the embodiment. The server device 30 is a server device of a company to which the users of each information terminal device 10 belong, or a server device of a business entity that provides virtual office services, and may be a general-purpose server device including a distributed type. The server device 30 constructs and controls a virtual space such as a virtual office. The server device 30 comprises a communication unit 31 and a server control device 40.
[0049] The communication unit 31 is a communication unit for wired or wireless communication. The communication unit 31 communicates using methods such as Wi-Fi or mobile phone lines and connects to an intranet or the internet. The communication unit 31 is controlled by the communication control unit 41. The communication unit 31 communicates data such as video, audio, and application operating status information with multiple information terminal devices 10.
[0050] (Server control unit) The server control device 40 controls various parts of the server device 30. The server control device 40 includes, for example, a control device such as a CPU or MPU, and a storage device such as RAM or ROM. The server control device 40 may be implemented by an integrated circuit such as an ASIC or FPGA. The server control device 40 may be implemented by a combination of hardware and software. The server control device 40 executes a program that controls the operation of the server device 30 according to this disclosure.
[0051] The server control device 40 comprises a communication control unit 41, a virtual office control unit 42, and a storage unit 49, as functional blocks realized by circuits and programs.
[0052] The communication control unit 41 controls the communication unit 31 to control the transmission and reception of data, including video and audio, and the transmission and reception of application operation status information between the information terminal device 10 and the communication control unit 41. For example, the communication control unit 41 controls the communication unit 31 to receive video captured by the camera 11 and audio picked up by the microphone 12 of the information terminal device 10. For example, the communication control unit 41 controls the communication unit 31 to transmit video and audio of the virtual office to the information terminal device 10. Application operation status information includes, for example, configuration information of the virtual office application.
[0053] The virtual office control unit 42 controls the operation of various applications running on the server device 30. For example, the virtual office control unit 42 controls the operation of virtual office applications. For example, the virtual office control unit 42 controls the operation of applications used for online communication.
[0054] The virtual office control unit 42 transmits the configuration of the application running on the server device 30 to the information terminal device 10 running the application, via the communication control unit 41. Specifically, the application control unit 43 transmits information indicating the configuration of the virtual office running on the server device 30 to the information terminal device 10 that is logged into the virtual office.
[0055] The memory unit 49 stores information such as the calculations performed by the server control device 40 and programs. The memory unit 49 is composed of, for example, RAM, a main memory such as ROM, and a storage device such as an SSD or HDD.
[0056] The memory unit 49 stores information indicating the virtual office to be configured in the virtual space. For each user who logs into the virtual office, the memory unit 49 stores information indicating the avatar, the information terminal device 10 associated with the avatar, and the user account of the information terminal device 10. The memory unit 49 also stores information indicating the arrangement of objects that make up the virtual office, such as the sections and virtual desks that make up the virtual office.
[0057] (Information processing device) An example of the configuration of the information processing device according to the embodiment will be described using Figure 4. Figure 4 is a block diagram showing an example of the configuration of the information processing device according to the embodiment. Figures 5 to 7 are examples of score charts showing the user's status and level. The information processing device 50 is a conceptual device realized through the cooperation of either or both of the server device 30 and the multiple information terminal devices 10 used in the virtual office. Either the information terminal device 10 or the server device 30 may be the information processing device 50, or the virtual space control system 1 comprising the information terminal device 10 and the server device 30 may be the information processing device 50. Each part of the information processing device 50 is realized through the cooperation of either or both of the terminal control device 20 and the server control device 40.
[0058] When multiple users log into a virtual office, the placement of each user's avatar is often determined based on their seating arrangement and physical distance in the real world, or it may be arbitrary or predetermined, with little consideration given to each user's state or circumstances. As a result, when multiple users communicate in the same virtual office via avatars, differences in emotions, psychological states, and behaviors between users can lead to communication problems. For example, if the avatar of a user experiencing positive emotions is placed within the information sharing range of another user's avatar experiencing negative emotions, the significant difference in their emotions may make appropriate communication difficult. Similarly, if the avatar of a user with low concentration is placed within the information sharing range of another user's avatar with high concentration, the other user's concentration may be disrupted, causing discomfort and making appropriate communication difficult.
[0059] In this embodiment, the information processing device 50 acquires information indicating the status of a user utilizing the virtual space through the information terminal device 10, and determines the placement position of the avatar corresponding to the user in the virtual space based on this user status.
[0060] As shown in Figure 4, the information processing device 50 includes a user state acquisition unit 51, an application control unit 52, an avatar position control unit 53, a display control unit 58, and an audio output control unit 59. For example, the user state acquisition unit 51 and the application control unit 52 may be implemented in cooperation with the terminal control unit 20 and the server control unit 40, the avatar position control unit 53 may be implemented in the server control unit 40, and the display control unit 58 and the audio output control unit 59 may be implemented in the terminal control unit 20, but it is not limited to these.
[0061] The user state acquisition unit 51 acquires information indicating the state of a user using the virtual space through the information terminal device 10. That is, the user state acquisition unit 51 acquires information indicating the state of the user of the information terminal device 10 through the information terminal device 10 that is logged into the virtual office. Here, the user state refers to a state in which the user is able to accept communication with other users, and includes, for example, the user's psychological state such as "user's emotions," and the state of the user's actions such as "user's level of concentration" and "user's speaking volume." In the following explanation, we will describe the case in which information indicating "user's emotions" is acquired as information indicating the user state, but it is not limited to this, and information indicating "user's level of concentration" or "user's speaking volume" may also be acquired.
[0062] The user state acquisition unit 51 estimates the user's emotions as described above, based on the user's image captured by the camera 11 installed in the information terminal device 10 that is logged into the virtual office, and acquires information indicating the user's emotions.
[0063] The user state acquisition unit 51 obtains information indicating the user's emotions by having the server device 30 estimate the user's emotions from the captured facial video, for example, by using emotion recognition AI (Artificial Intelligence). Alternatively, the user state acquisition unit 51 may have the server device 30 estimate the user's emotions from the captured facial video using an external AI server.
[0064] The estimated user emotions include basic emotions such as joy, sadness, surprise, anger, disgust, and fear, and the user state acquisition unit 51 can estimate the user's emotions, including the level of each emotional element. Alternatively, the user state acquisition unit 51 may estimate an emotion selected from the above-mentioned emotional elements. For example, the user state acquisition unit 51 may estimate whether the user is experiencing positive or negative emotions by estimating the emotions of "joy" and "disgust."
[0065] The user state acquisition unit 51 estimates whether the emotions of the user of the information terminal device 10 logged into the virtual office are positive or negative, or in other words, whether they have a positive tendency or a negative tendency. Specifically, as shown in Figure 5, the user state acquisition unit 51 uses an emotion level score chart to score the user's emotional tendency and its emotional level on a 5-point scale for both negative and positive tendencies, and acquires information indicating this scored emotion. In this case, a higher score for the emotional level in the positive tendency indicates stronger positive emotions, and a higher score for the emotional level in the negative tendency indicates stronger negative emotions.
[0066] Furthermore, the estimation of the user's emotions and emotion level by the user state acquisition unit 51 is not limited to those described above. For example, the estimation of the user's emotions is not limited to estimation based on the user's facial image, but may also be estimated using the user's spoken voice, the user's pulse wave information based on the user's facial image, etc.
[0067] Furthermore, the user status acquisition unit 51 may estimate the user's level of concentration as described above, based on video footage captured by the camera 11 installed in the information terminal device 10 that is logged into the virtual office, and acquire information indicating this user level of concentration.
[0068] In this case, the user state acquisition unit 51 acquires information indicating the user's concentration level when the server device 30 estimates the user's concentration level from the captured facial image of the user, for example, by using a concentration level estimation AI. Alternatively, the user state acquisition unit 51 may also use an external AI server to estimate the user's concentration level from the captured facial image of the user when the server device 30 uses an external AI server.
[0069] The user status acquisition unit 51 estimates whether the concentration level of the user of the information terminal device 10 logged into the virtual office is high or low, in other words, whether it is a concentration-oriented or non-concentration-oriented user. Specifically, as shown in Figure 6, the user status acquisition unit 51 uses a concentration level score chart to score the user's concentration tendency and level on a 5-point scale for both concentration-oriented and non-concentration-oriented users, and acquires this scored information indicating concentration. In this case, a higher score for the concentration-oriented level indicates a higher degree of concentration, and a higher score for the non-concentration-oriented level indicates a higher degree of non-concentration, i.e., a lower degree of concentration. Note that the estimation of the user's concentration level and concentration level by the user status acquisition unit 51 is not limited to the above.
[0070] Furthermore, the user status acquisition unit 51 may measure the amount of speech per unit time of the user as the user status described above, based on the sound picked up by the microphone 12 provided in the information terminal device 10 that is logged into the virtual office, and acquire information indicating this amount of speech of the user.
[0071] In this case, the user state acquisition unit 51 measures the amount of speech per unit time spoken by the user, excluding other sounds other than speech, using, for example, a speech segment recognition AI, based on the audio signal acquired by the microphone 12 by the voice acquisition unit 22 of the information terminal device 10 or the virtual office control unit 42 of the server device 30. This amount of speech spoken by the user may exclude, for example, speech spoken when the user is not logged into the virtual office, speech spoken by the user in a conference room in the virtual office, and speech spoken by the user during meetings with specific parties.
[0072] The user status acquisition unit 51 measures whether the user of the information terminal device 10 logged into the virtual office speaks frequently or infrequently. Specifically, as shown in Figure 7, the user status acquisition unit 51 uses a speech volume level score chart to score the user's speech volume level on a 5-point scale for both low and high speech volume, and acquires information indicating this scored speech volume. In this case, a higher score for the high speech volume level indicates a higher speech volume per unit time, and a higher score for the low speech volume level indicates a lower speech volume per unit time. Note that the measurement of the user's speech volume by the user status acquisition unit 51 is not limited to the above.
[0073] The application control unit 52 determines the placement position of the avatar corresponding to the user in the virtual space based on the acquired user status.
[0074] The application control unit 52 determines the placement of avatars in the virtual space so that avatars corresponding to users with similar user state information are placed in close proximity. Similar user state information means that the "user's emotions," "user's level of concentration," and "user's speaking volume" are all similar to each other.
[0075] The application control unit 52 determines the placement position of each avatar so that, for example, the avatars corresponding to users whose emotions are similar to each other are placed in close proximity to each other in the virtual space. Similar emotions among users means, for example, that the emotions of each user are either predominantly positive or predominantly negative, and it is preferable that the difference in emotion levels on the score chart described above is less than or equal to a predetermined value (e.g., 2 scores). The application control unit 52 determines the placement positions of the first and second avatars so that the first and second avatars, corresponding to the first and second users who are estimated to have a positive tendency and a high positive emotion level (e.g., score 4), are within each other's information sharing range.
[0076] Furthermore, the application control unit 52 may determine the placement position of each avatar so that, for example, the avatars corresponding to users whose levels of concentration are close to each other are placed in close proximity in the virtual space. Close levels of user concentration means, for example, that the levels of concentration of each user are either concentrated or non-concentrated, and it is preferable that the difference in the levels of concentration or non-concentration on the score chart described above is less than or equal to a predetermined value (e.g., 2 scores). The application control unit 52 may also determine the placement positions of the first and second avatars so that the first and second avatars corresponding to the first and second users, respectively, who are estimated to be concentrated and have a high concentration level (e.g., score 3), are within each other's information sharing range.
[0077] Furthermore, the application control unit 52 may determine the placement position of each avatar, for example, so that each avatar corresponding to users whose speech volume is similar to that of other users is placed in a position close to that of other users in the virtual space. Similar speech volume means, for example, that the speech volume per unit time of each user is either high or low, and it is preferable that the difference in the levels of high or low speech volume on the score chart described above is less than or equal to a predetermined value (e.g., 2 scores). The application control unit 52 may also determine the placement positions of the first and second avatars, corresponding to the first and second users, respectively, who are estimated to have a high speech volume level (e.g., a high speech volume score of 3), so that they are within each other's information sharing range.
[0078] Furthermore, the application control unit 52 determines the placement of avatars such that avatars corresponding to users whose differences in information indicating the user's state exceed a predetermined level are outside the information sharing range where information is shared between them.
[0079] For example, if a first user, who is estimated to have a positive tendency and a high score for positive emotional level (e.g., score 4), and a second user, who is estimated to have a negative tendency and a high score for negative emotional level (e.g., score 3), are logged into the virtual office, the application control unit 52 determines the placement positions of the first and second avatars, which correspond to the first and second users respectively, so that they are outside the range of mutual information sharing.
[0080] Furthermore, the application control unit 52 may, for example, determine the placement positions of the first and second avatars so that the first and second avatars corresponding to the first and second users are outside the mutual information sharing range, if, for example, a first user who is estimated to have a tendency towards concentration and a high concentration level score (e.g., score 4) and a second user who is estimated to have a tendency towards non-concentration and a high non-concentration level score (e.g., score 3) are logged into the virtual office, because the difference in concentration levels between the first and second users is greater than or equal to a predetermined value (e.g., score 5).
[0081] Furthermore, the application control unit 52 may, for example, determine the placement positions of the first and second avatars so that the first and second avatars corresponding to the first and second users are outside the mutual information sharing range, if the difference in the speech volume levels between the first and second users is greater than or equal to a predetermined value (e.g., score 5).
[0082] In the example above, the difference in information indicating the user's state was defined as a case where the score for any of the levels related to emotional level, concentration level, or speech volume was above a predetermined value (5 points). However, it is not limited to this, and for example, it could also be a case where the emotional tendency (positive or negative tendency) is different.
[0083] Furthermore, the application control unit 52 acquires the user's affiliation information and determines the placement of avatars in the virtual office corresponding to users who share this affiliation information. Typically, users log in to the virtual office of the company or organization to which they belong. Therefore, when a user logs into the virtual office, the application control unit 52 acquires the user's affiliation information and determines the placement of avatars in the virtual office corresponding to other users who share the same affiliation information as the user. In other words, the application control unit 52 determines the placement positions of the first and second avatars corresponding to the first and second users in the virtual office of the department to which the first and second users belong, respectively.
[0084] The avatar position control unit 53 controls the position of each avatar in the virtual office. When the placement position of each avatar is determined, the avatar position control unit 53 controls the avatar so that each avatar is displayed in the determined position in the virtual office. Furthermore, when an avatar movement operation is performed via the operation unit 13 of the information terminal device 10, the avatar position control unit 53 controls the corresponding avatar to move in response to this operation.
[0085] The display control unit 58 causes the display units 17 of the multiple information terminal devices 10 to display video showing various types of information, such as virtual office screens.
[0086] The audio output control unit 59 enables the second user to share the first user's voice when the second avatar enters the first avatar's information sharing range. Furthermore, if the second avatar is within the first avatar's information sharing range, and the first avatar is also within the second avatar's information sharing range, the audio output control unit 59 enables the second user to share the first user's voice, and the audio output control unit 59 enables voice communication between the first and second users when voice sharing becomes possible. More specifically, when voice sharing becomes possible, the audio output control unit 59 enables voices and other sounds to be heard by both the information terminal device 10 used by the first user corresponding to the first avatar and the second information terminal device 10 where the second user, corresponding to the second avatar, is logged into the virtual office. More specifically, the audio output control unit 59 causes the audio output unit 18 of the first information terminal device 10 to output the audio of the second user picked up by the microphone 12 of the second information terminal device 10, and the audio output unit 18 of the second information terminal device 10 to output the audio of the first user picked up by the microphone 12 of the first information terminal device 10.
[0087] The audio output control unit 59 stops audio sharing between the first user and the second user if the first avatar and the second avatar are no longer included in the information sharing range due to avatar movement or other reasons, or if an operation to stop audio sharing is performed.
[0088] (Virtual space control method) The virtual space control method will be explained using Figures 8 to 10. Figure 8 is a flowchart of an example of a virtual space control method according to the embodiment. Figure 9 is an example of the display screen of an information terminal device showing each avatar placed in the virtual office. Figure 10 is an example of a score chart showing the emotions of each user corresponding to each avatar placed in the virtual office and their level. The process shown in Figure 8 is executed at predetermined time intervals while the virtual office is provided by the virtual space control system 1. While logged into the virtual office on the information terminal device 10, a screen showing the virtual office is displayed on the display screen of the information terminal device 10, as shown in Figure 9, or becomes displayable. The premise of this process is that each user is logged into the virtual office using the information terminal device 10, and the user state acquisition unit 51 of the information processing device 50 acquires information indicating the emotions of all users who use the virtual space through each information terminal device 10. In addition, the range in which a user can place an avatar and perform work in the virtual office may be arbitrary, but in this embodiment, the avatar is placed and work is performed within the range of the department to which the user belongs in the company to which the user belongs. Furthermore, users' avatars can, of course, communicate with other users' avatars belonging to other departments and move to other locations such as meeting rooms in the virtual office by operating the control unit 13 of each information terminal device 10.
[0089] The information processing device 50 starts displaying the virtual office (step S01). More specifically, when a user logs in to a virtual office of their company or organization using their own account information, the information processing device 50, via the display control unit 58, starts displaying an image of the virtual office on the display unit 17 of the user's information terminal device 10.
[0090] Next, the information processing device 50 determines the placement position of the avatar in the virtual space (step S02). More specifically, the information processing device 50, with the help of the application control unit 52, determines the placement position of the avatar within the department to which the user belongs when the user starts using the virtual office, in other words, the placement position of the avatar corresponding to users who belong to the same department. The application control unit 52 determines the placement position of the avatar based, for example, on information indicating the user's emotions acquired by the user state acquisition unit 51 during the period before the user starts using the virtual office (for example, 30 minutes before starting the virtual office).
[0091] Here, using Figures 9 and 10, we will explain an example of determining the placement of an avatar corresponding to a user in a virtual office based on the user's emotions as a state of the user. In the following explanation, the nth (1st to 8th) avatar is the avatar corresponding to the nth user, and the information sharing range of the nth avatar is defined as the information sharing range Cn. Figure 9 illustrates the display screen W1 of the virtual office on the first information terminal device 10 when the first user logs into the virtual office using the first information terminal device 10 and the first avatar A1 is placed in the section of department T to which the first user belongs. In this example, the display screen W1 of the information terminal device 10 used by the first user shows the range of department T in the virtual office, but a predetermined range centered on the first avatar A1 corresponding to the first user may be displayed, or only a part of department T may be displayed, or a wider range than shown in the figure above, including areas other than department T, may be displayed.
[0092] In Figure 9, within the section of department T in the virtual office, the first avatar A1 to the eighth avatar A8, corresponding to the first to eighth users belonging to department T, are placed. The first user uses the first information terminal device 10 to perform their work using the first avatar A1 placed in the section of department T in the virtual office. The second to eighth users are colleagues of the first user and perform their work using the second avatar A2 to the eighth avatar A8, each placed in the section of department T in the virtual office. Furthermore, information sharing ranges C1 to C8 are set around the first to eighth avatars, and when multiple avatars are included in each other's information sharing ranges, they can communicate with each other via voice (voice sharing). In Figure 9, the information sharing ranges C1 to C8, shown by dashed lines, represent the range set in the virtual space and do not necessarily need to be displayed on the display screen W1.
[0093] Figure 10 also shows the emotional tendencies and levels of the first to eighth users on a score chart. In this example of Figure 10, the first user is shown as (1), and the second to eighth users are shown as (2) to (8). In Figure 10, the first user's emotions when they logged into the virtual office (for example, in the 30 minutes immediately preceding login) showed a negative tendency, and the score of their emotional level (for example, the average over 30 minutes) showed a negative tendency with a score of approximately 3.3. Emotional tendencies and emotional level scores were also obtained for the second to eighth users.
[0094] The information processing device 50, via the application control unit 52, determines the placement position of each avatar so that avatars corresponding to users with similar emotional level scores are placed close together. In the example shown in Figure 9, the application control unit 52 determines the placement position of the first avatar A1, which corresponds to the first user who logged into the virtual office, so that it is near the fifth avatar A5, which corresponds to the fifth user with a similar emotional level score. The application control unit 52 also determines the placement position of the first avatar A1, which corresponds to the first user who logged into the virtual office, so that its information sharing ranges C1 and C7 do not overlap with those of the seventh avatar A7, which corresponds to the seventh user with an emotional level score difference of more than a predetermined value (outside the range where voices are shared between them).
[0095] Let's return to Figure 8 for further explanation. Next, the information processing device 50 places the avatar at the determined location (step S03). More specifically, the information processing device 50 uses the avatar position control unit 53 to control the placement of the first avatar A1 at the determined predetermined location within the section of department T in the virtual office. In this case, as shown in Figure 9, the first avatar A1 and the fifth avatar A5 are within each other's information sharing range C1, C5, so the first user and the fifth user can have a voice conversation. In this way, the placement position of each avatar is determined so that each avatar corresponding to a user with similar emotional information is placed in a close position in the virtual space, making it possible for users with similar emotions or psychological states to communicate appropriately. Also, since the first avatar A1 is placed in a position where the information sharing range C1, C7 does not overlap with that of the seventh avatar A7, the first user does not have a voice conversation with the seventh user. In other words, the first user does not hear anything from the seventh user's conversation. In this way, the placement of each avatar is determined so that users whose emotional differences exceed a certain threshold are outside the range of information sharing with each other. This prevents users with different emotions or psychological states from inadvertently interacting with each other.
[0096] Next, the information processing device 50 determines whether or not the use of the virtual office has ended (step S04). That is, the information processing device 50, with the application control unit 52, determines whether or not the user has logged off from the virtual office. In this determination, if the user has logged off from the virtual office (step S04; Yes), the process is terminated.
[0097] On the other hand, if the user has not logged off from the virtual office (step S04; No), the information processing device 50 determines whether or not there has been a change in the user's state (step S05). More specifically, the information processing device 50 uses the user state acquisition unit 51 to determine whether or not there has been a change in the user's emotions (user's state). For example, even if the user's emotions were negative when they logged in from the virtual office (in the morning), it is quite possible that their emotions may change to a positive trend through conversations with users corresponding to nearby avatars in the virtual office. In this case, the user state acquisition unit 51 acquires information on the user's emotions for the 30 minutes immediately preceding the login of all avatars that belong to the same department and are located within the same section, or for specific avatars (for example, excluding avatars of users with specific roles), at predetermined intervals (for example, every 1 or 2 hours), and determines whether or not there has been a change in the user's emotions. Furthermore, the user state acquisition unit 51 may determine whether or not a change has occurred in the user's emotions based on the change in the user's emotional level score during a predetermined period (for example, the 30 minutes immediately preceding a certain point in time) and the emotional level score during the preceding period.
[0098] In this determination, if the information processing device 50 determines that there has been no change in the user's emotions (step S05; No), it returns to step S04. On the other hand, if the information processing device 50 determines that there has been a change in the user's emotions (step S05; Yes), it changes the position of the avatar in the virtual space (step S06) and returns to step S04.
[0099] More specifically, the information processing device 50 determines the placement position of the first avatar A1 based on the newly acquired information indicating the emotions of the first user via the application control unit 52, and controls the placement of the first avatar A1 at the determined predetermined position via the avatar position control unit 53. In this case, if the first user's emotions tend to be positive and the positive emotion level score is 2, for example, the first avatar A1 can be placed in a position adjacent to the sixth avatar A6. In this configuration, based on emotion level scores acquired periodically or irregularly, the placement position of the first avatar A1 corresponding to the first user is changed so that it is near the sixth avatar A6 corresponding to the sixth user, whose emotion level score is similar, thus enabling appropriate communication with users with similar emotions or psychological states at all times.
[0100] In the above explanation, based on the arrangement of each avatar determined by the information processing device 50, voice communication is possible between avatars whose information sharing ranges overlap, and voice communication is not possible between avatars whose information sharing ranges do not overlap. For example, in the department T area shown in Figure 9, among avatars belonging to department T, voice communication may be properly possible between avatars whose information sharing ranges overlap because the voices can be heard at an appropriate volume, and voice communication may not be properly possible between avatars whose information sharing ranges do not overlap because the voices can be heard at a low volume.
[0101] (effect) As described above, in this embodiment, the information processing device 50 acquires information indicating the state of users using the virtual space through the information terminal device 10, and determines the placement of avatars so that avatars corresponding to users with similar user state information are placed in close proximity in the virtual space. This promotes natural communication and smooth interaction among users with similar user states (e.g., user emotions). This not only makes it easier for users to ask questions and exchange opinions with others, but also provides an environment where cooperation and co-creation within the virtual space can proceed smoothly. Furthermore, since the placement of avatars is displayed on the display screen of the information terminal device 10, it is possible to visually represent the differences in user states (e.g., user emotions), and the emotional distance between users can be grasped at a glance. As a result, users can intuitively understand how much their own emotional state matches or differs from that of others, and adjust their actions as needed.
[0102] Furthermore, in this embodiment, the information processing device 50 determines the placement of avatars such that avatars corresponding to users whose information indicating the user's state differs by a predetermined amount are outside the information sharing range where information is shared between users. This prevents users with different states from inadvertently interacting with each other. This not only reduces user stress and improves the quality of communication, but also contributes to improving the overall user experience. It can also reduce misunderstandings and friction between users in different emotional states, providing a foundation for building smoother and more effective cooperative relationships.
[0103] Furthermore, in this embodiment, the information processing device 50 acquires user affiliation information and determines the placement position in the virtual space of avatars corresponding to users with the same affiliation information. This enables appropriate communication between users with similar states (e.g., user emotions) in the virtual space where affiliation information is shared, thereby improving work efficiency.
[0104] The components of the illustrated virtual space control system are functionally conceptual and do not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage of each device.
[0105] The configuration of the virtual space control system is implemented, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks implemented through the cooperation of hardware or software. That is, these functional blocks can be implemented in various forms using hardware alone, software alone, or a combination thereof.
[0106] The components described above include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the above-described configurations can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the configurations are possible without departing from the spirit of the present invention.
[0107] The above describes a configuration in which the placement of avatars is determined primarily based on the user's emotions, but this is not the only configuration. The placement of avatars may also be determined based on the user's level of concentration or the amount of user speech. Alternatively, one of the user's emotions, level of concentration, or amount of user speech may be selected for predetermined periods, and the placement of avatars may be determined based on the selected user state. For example, the placement of avatars may be determined based on the user's emotions in the morning and based on the user's level of concentration in the afternoon. Furthermore, while the above configuration places avatars with similar emotions close together, avatars with different emotions may also be placed close together. For example, by placing the avatar of a user with positive tendencies near the avatar of a user with negative tendencies, it is possible to expect that the user with negative tendencies may be encouraged by the user with positive tendencies and change to a more positive state. [Explanation of Symbols]
[0108] 1. Virtual Space Control System 10 Information terminal device 20 Terminal control device 40 Server Control Units 42 Virtual Office Control Unit 50 Information Processing Devices 51 User Status Acquisition Unit 53 Avatar Position Control Unit 52 Application Control Unit 58 Display Control Unit 59 Audio Output Control Unit A1 First Avatar
Claims
1. A virtual space control system that displays a virtual space on an information terminal device, The information processing device included in the virtual space control system is: Information indicating the status of the user utilizing the virtual space is acquired through the aforementioned information terminal device. Based on the user's state, the placement position of the avatar corresponding to the user in the virtual space is determined. Virtual space control system.
2. The aforementioned information processing device is The placement position of the avatar is determined such that the avatar corresponding to a user whose status information is close to the avatar is placed in a close position in the virtual space. The virtual space control system according to claim 1.
3. The aforementioned information processing device is The placement of avatars corresponding to users whose differences in information indicating the user's status exceed a predetermined level is determined so that they are outside the information sharing range where information is shared between users. The virtual space control system according to claim 1.
4. The aforementioned information processing device is Obtain the user's affiliation information, The placement position in the virtual space of the avatar corresponding to the user whose affiliation information is the same as the above-mentioned user is determined. The virtual space control system according to claim 1.
5. A virtual space control method executed by a virtual space control system that displays a virtual space on an information terminal device, Information indicating the status of the user utilizing the virtual space is acquired through the aforementioned information terminal device. Based on the user's state, the placement position of the avatar corresponding to the user in the virtual space is determined. A virtual space control method performed by a virtual space control system.