Information processing device, information processing method, and program

Abstract

To provide a technique which in a case of displaying a user's avatar on a terminal of another user participating in a virtual space, displays the avatar in an appropriate form in accordance with the standpoint of the user mentioned above as another user.SOLUTION: An information processing device comprises: first acquisition means which acquires information of a plurality of display forms of the avatar of a first user; second acquisition means which acquires information of a second user within a virtual space in which the first user participates; and determination means which determines on the basis of the information of the second user a display form of the avatar of the first user for the second user, out of the display forms of the avatar of the first user.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] With the development and spread of virtual reality (VR) technology, the use of virtual space for various purposes such as distribution, business, and medical care is being considered. For example, in the medical field, patients who have physical or mental disabilities that make it difficult to go outside, or who live in remote islands or other areas where it is difficult to travel to a hospital, can receive medical treatment and counseling in a virtual space. In addition, by using an avatar (the user's alter ego in the system) in a virtual space, even patients whose faces are widely known or who are concerned about the public's views can speak honestly without being seen by other users, making it easier for them to participate in medical examinations and counseling.

[0003] Group counseling is a type of counseling where multiple patients gather together. In group counseling, patients can see themselves more clearly through the reactions and relationships of others, and can see things more flexibly by incorporating the thoughts and perspectives of others.

[0004] Research results have shown that users are more likely to self-disclose if they use an avatar that does not resemble them. It is expected that group counseling sessions held in virtual space will be more effective if patients use an avatar that does not resemble them. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2009-104482 A Summary of the Invention [Problem to be solved by the invention]

[0006] In group counseling, it is desirable for patients to use avatars that are far removed from their actual facial images, from the viewpoint of protecting privacy and because patients can speak their true feelings without worrying about the eyes of others. However, it is undesirable for patients to use avatars, because psychiatrists and counselors can estimate the mental state and severity of symptoms by looking at the patient's actual facial expressions. In other words, when displaying a patient's avatar on another user's device, the appropriate display form of the avatar differs depending on whether the other user is a psychiatrist or counselor, or another patient.

[0007] Even for purposes other than group counseling, when a user's avatar is displayed on the terminals of other users in a virtual space, the appropriate display form of the avatar may differ depending on the position of the other users. As an example, when a user is conducting a business negotiation in a virtual space, the user may want to display the avatar that he or she normally uses to his or her colleague users, and to display an avatar with a businessman's appearance to give a better impression to the users of the business partner. As another example, when holding a live performance in a virtual space, the organizer may want to display a real avatar that faithfully imitates the actual appearance of the performer to users who have purchased tickets, and a deformed avatar of the performer to users who have not purchased tickets. By switching avatars, the organizer of the live performance can hold a live performance for fans and promote it to users who are not fans.

[0008] As a technique for changing the display of an avatar, Patent Document 1 discloses a technique for displaying avatars of users with more common activity histories in a distinctive manner, for example by adding a speech bubble to the avatar. However, even when considering the common activity history, it may be difficult to appropriately switch avatars depending on the user's position.

[0009] The present invention aims to provide a technique for displaying a user's avatar in an appropriate display form depending on the position of another user participating in a virtual space when the user's avatar is displayed on the terminal of the other user. [Means for solving the problem]

[0010] The information processing device of the present invention is characterized in having a first acquisition means for acquiring information on multiple display forms of an avatar of a first user, a second acquisition means for acquiring information of a second user in a virtual space in which the first user participates, and a determination means for determining a display form of the first user's avatar to be displayed to the second user from the multiple display forms of the avatar of the first user based on the information of the second user. Effect of the Invention

[0011] According to the present invention, when a user's avatar is displayed on the terminal of another user participating in a virtual space, the avatar can be displayed in an appropriate display form depending on the position of the other user. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of the configuration of a communication system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of a server. [Figure 4] FIG. 13 is a diagram illustrating an example of a user interface for setting how an avatar is displayed. [Diagram 5] FIG. 13 is a diagram showing a specific example of changing the way an avatar is displayed. [Figure 6A] 3 is a flowchart illustrating the processing of the system according to the first embodiment. [Figure 6B] 6 is a flowchart illustrating another process of the system according to the first embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of the configuration of a communication system according to a second embodiment. [Figure 8] 10 is a flowchart illustrating the processing of the system according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] <Embodiment 1> 1 is a diagram showing a configuration example of a communication system 100 according to the first embodiment. The communication system 100, which is an example of an information processing system, is a system constructed as a client-server system. The communication system 100 is applied to, for example, a remote medical care support system.

[0015] The communication system 100 includes a server 101 and a plurality of user terminals 102 connected to the server 101 via a network such as the Internet. The server 101 is, for example, a personal computer (PC). The user terminals 102 are, for example, electronic devices such as a PC, a smartphone, a tablet, a head mounted display (HMD), and a controller capable of communicating with the HMD. In the following description, the user terminal 102 is an HMD. The communication system 100 may include other electronic devices as the user terminals 102, not limited to an HMD, and may include electronic devices of different types. In the following description, the HMD has a function capable of being directly connected to a network. The HMD may be connected to the network via another PC or a smartphone.

[0016] 2 is a diagram illustrating an example of the hardware configuration of user terminal 102. CPU 201 performs overall control of various functions of user terminal 102 via internal bus 206 using programs stored in ROM (Read Only Memory) 203. Various processes executed by user terminal 102 are realized by CPU 201. CPU 201 can project and display the results of program execution as an image in the user's field of vision using display 202.

[0017] The ROM 203 is, for example, a flash memory, and stores various setting information, application programs, etc. The RAM (Random Access Memory) 204 functions as a memory and work area for the CPU 201. The network I / F (interface) 205 is a module for connecting to a network.

[0018] The sensor unit 209 includes one or more sensors, although it is simply illustrated in Fig. 2. Specifically, the sensor unit 209 includes at least one sensor selected from a GPS (Global Positioning System), a gyro sensor, an acceleration sensor, a proximity sensor, and a blood pressure, heart rate, and brain wave measurement sensor. The sensor unit 209 may include a sensor capable of detecting biometric information for implementing fingerprint authentication, vein authentication, iris authentication, and the like.

[0019] The camera 210 is, for example, a fisheye camera attached inside the HMD serving as the user terminal 102, and has a function of photographing the user's face. The photographed image data is stored in the RAM 204 after distortion caused by the fisheye lens is removed. The storage 212 is a storage medium, and is a device for storing various data such as applications.

[0020] The short-range communication I / F 213 is an interface used for communication with the controller. The user can input gestures to the user terminal 102 by moving the controller he / she holds, or can give instructions to the user terminal 102 by operating a button or a joystick provided on the controller. The controller may include a sensor that measures the user's heart rate, pulse, sweat, and the like. The user terminal 102 may communicate with a wearable device worn by the user via the short-range communication I / F 213 to acquire the user's heart rate, pulse, sweat, and the like. The user terminal 102 may also communicate with a camera and a group of sensors installed in a room where the user is present via the short-range communication I / F 213 to acquire information about the room and information about the user.

[0021] The microphone 208 picks up the voice of the user. The speaker 211 plays back the voices of other users participating in the communication system 100, sound effects, background music, and the like.

[0022] 3 is a diagram illustrating an example of a hardware configuration of server 101 (information processing device). Server 101 has a display unit 301, a video random access memory (VRAM) 302, a bit move unit (BMU) 303, a keyboard 304, and a pointing device (PD) 305. Server 101 also has a CPU 306, a storage 307, a RAM 308, a ROM 309, a memory card 310, a network I / F 311, and a bus 312.

[0023] The display unit 301 displays, for example, live view images, icons, messages, menus, and other user interface information. The VRAM 302 stores information on moving images to be displayed on the display unit 301. Data generated in the VRAM 302 is transferred to the display unit 301 in accordance with a predetermined standard and displayed on the display unit 301.

[0024] The BMU 303 controls, for example, data transfer between memories (eg, between the VRAM 302 and other memories) and data transfer between memories and each I / O device (eg, the network I / F 313).

[0025] The keyboard 304 has various keys for inputting characters, etc. The PD 305 is used, for example, to select and specify icons, menus, and other content displayed on the display unit 301, and to drag and drop objects.

[0026] The CPU 306 controls each device of the server 101 based on a control program, such as an OS stored in the storage 307, the ROM 309, or the memory card 310, and various programs for implementing the functions of the server 101. Various processes executed by the server 101 are implemented by the CPU 306.

[0027] The storage 307 includes, for example, an HDD and an SSD. The storage 307 stores the control program and various data to be temporarily stored. The RAM 308 is used as a work area for the CPU 306, an area for saving data in error processing, an area for loading the control program, etc. The ROM 309 is a non-volatile memory, and stores the control program executed by the server 101. The memory card 310 stores the control program and various data to be temporarily stored, etc.

[0028] The network I / F 311 is an interface for communicating with the user terminal 102 and the like via a network. The bus 312 includes an address bus, a data bus, and a control bus. The CPU 306 may obtain and execute a control program from the storage 307, the ROM 309, or the memory card 310, or may obtain and execute a control program from another information processing device or the like via the network I / F 311 and over the network.

[0029] Fig. 4 is a diagram showing an example of a user interface (UI) for setting how an avatar is to be displayed (display form). Fig. 4 shows a UI for setting how an avatar of a user named Suzuki Taro is to be displayed. Information relating to how an avatar is to be displayed includes information such as which of one's avatars is to be displayed and how one's avatar is to be displayed. How one's avatar is to be displayed may include, for example, a display form in which an image of the actual user is displayed together with the avatar.

[0030] The setting UI 401 is displayed on the display 202 of the user terminal 102. Using the setting UI 401, the user can set how his / her avatar is to be displayed for each other user or for each position (category) of the other users. The user can register in advance a plurality of display forms as the way his / her avatar is to be displayed.

[0031] The server 101 may set priorities for the manner in which avatars are displayed that are set for each category of other users in the setting UI 401. For example, the server 101 may set priorities in ascending order from the category shown at the top of the screen of the setting UI 401. For other users who belong to multiple categories, the server 101 may determine the manner in which they are displayed based on the category with the highest priority.

[0032] The other user's category 402 is a UI element for setting the category of other users whose avatars are to be displayed. The user can set, for example, a specific user name, qualification, a specific group, etc. as the other user's category 402.

[0033] Section 404 shows an example of directly specifying a specific user name (e.g., Suzuki Hanako). Section 405 shows an example of specifying the qualifications of another user (e.g., counselor). Sections 406 and 407 show examples of specifying a predetermined group to which the other user belongs (e.g., colleagues, employees of X Co., Ltd.). The predetermined group is a group created by a user (Suzuki Taro in the example of FIG. 4) who sets the appearance in the setting UI 401, or a group that is registered in advance. The predetermined group may be a group that indicates the relationship between the user and the other user, such as family, friends, or seniors. Section 408 shows an example of specifying another user other than sections 404 to 407.

[0034] The display method 403 is a UI component for setting how the avatars are displayed for each category of other users. For example, the user can set which avatars are displayed and how they are displayed as the display method 403 of the avatars.

[0035] Presentation 409 corresponding to category 404 shows an example of setting to display the facial image of the user (Suzuki Taro) captured by camera 210 together with avatar X. That is, server 101 controls so that the facial image of avatar X and Suzuki Taro are displayed on user terminal 102 of another user (Suzuki Hanako) in category 404.

[0036] The presentation 410 corresponding to category 405 shows an example in which the face image of the user (Suzuki Taro) captured by the camera 210 is set to be displayed together with the avatar Y. That is, the server 101 controls the user terminal 102 of the other user (counselor) in category 404 so that the face images of avatar Y and Suzuki Taro are displayed. Note that avatar X and avatar Y are avatars of Suzuki Taro and are different from each other.

[0037] Presentation 411 corresponding to category 406 and presentation 413 corresponding to category 408 show examples where the presentation is set to display avatar Y. Presentation 412 corresponding to category 407 shows an example where the presentation is set to display a real avatar whose avatar is a 3D shape obtained from the actual face of the user (Suzuki Taro) for whom the presentation is set.

[0038] With the above settings, when a user (Suzuki Taro) participates in group counseling in a virtual space, the server 101 can show avatar Y with a facial image to other users who are counselors by way of presentation 410. On the other hand, the server 101 can show avatar Y without a facial image to other users who are patients other than Suzuki Taro by way of presentation 413.

[0039] In addition, when a user (Suzuki Taro) participates in a virtual space with a colleague and a business partner (for example, an employee of X Corporation) to conduct business negotiations, the server 101 can display to the colleague user an avatar Y that is familiar to the colleague user by way of presentation 411. On the other hand, the server 101 can display to the business partner user a real avatar by way of presentation 412.

[0040] Although an example of how to display the avatar (display form) has been described above, various forms may be used for how the avatar is displayed. For example, the avatar may be displayed in a form in which the facial expression of the user is reflected in the facial expression of the real avatar. The server 101 can reflect the facial expression of the user in the facial expression of the real avatar by analyzing the facial expression from the facial image of the user using facial expression analysis technology. This display form enables the server 101 to convey the facial expression of the user (patient) to the counselor without displaying the facial image.

[0041] The avatar may be displayed in a form in which a part of the face image of the user is displayed together with the avatar of the user, instead of displaying the entire face image of the user. Depending on the purpose of selling, etc., the face image may be displayed with the eyes hidden, or only the eyes or mouth may be displayed. By displaying only a part of the face image, it becomes difficult to identify an individual.

[0042] The avatar may be displayed in a manner in which the voice spoken by the user is played back as is on the user terminal 102 of the other user together with the display of the avatar, or the voice spoken by the user may be processed and played back together with the display of the avatar so that the individual cannot be identified. The avatar may also be displayed in a manner in which the voice spoken by the user is displayed as subtitles on the user terminal 102 of the other user together with the display of the avatar, without being played back. By processing the voice or displaying it as subtitles, the risk of an individual being identified from the voice can be reduced.

[0043] The server 101 may use the purpose of communication together with information of other users as a condition for determining how to display the avatar. The purpose of communication may be set by the user, for example, or may be set based on information about the virtual space. The information about the virtual space is information that is set in advance for the virtual space, and includes information such as the purpose of communication in the virtual space and the type of the virtual space (whether it is a game venue or a counseling venue) from which the purpose of communication can be inferred.

[0044] For example, if the other user is a counselor, the server 101 controls so that the avatar of the user (patient) is displayed with a facial image if the purpose of communication is counseling. On the other hand, the server 101 controls so that the avatar of the user is displayed without a facial image if the purpose of communication is casual conversation. By controlling whether or not to display a facial image based on the purpose of communication, it is possible to obtain the effect of suppressing the display of a facial image in a situation where a facial image is not necessary.

[0045] Server 101 may use information about the virtual space together with information about other users as conditions for determining how to display the avatar. For example, if the other user is a counselor, server 101 controls the avatar of the user (patient) to be displayed with a facial image when the user (patient) communicates with the counselor at a counseling venue. On the other hand, server 101 controls the avatar to be displayed without a facial image when the user communicates with the counselor at a game venue. By controlling whether or not to display a facial image based on information about the virtual space, for example, information about the location where the avatar is located, an effect of suppressing the display of a facial image in places where a facial image is unnecessary can be obtained.

[0046] The server 101 may use the number of times or the time of past communication together with the information of the other user as a condition for deciding how to display the avatar. For example, if the other user is a counselor, the server 101 determines that the counselor is a trustworthy counselor if the number of counseling sessions or the counseling time exceeds a threshold value, and controls the display of the avatar to include a facial image. By taking the number of times or the time of communication into consideration, an effect is obtained in which a facial image is displayed for counselors who have become trustworthy through repeated counseling sessions.

[0047] Server 101 may use information on the appearance and clothing of other users' avatars as well as information on other users as conditions for determining how to display the avatar. For example, when negotiating with a business partner in a virtual space, server 101 controls so that the user's formal avatar is displayed to other users (business partner users) who use formal avatars. On the other hand, server 101 controls so that the user's casual avatar is displayed to other users (business partner users) who use casual avatars. By taking into account information on other avatars, an effect is achieved in which an avatar appropriate for the time, place, and occasion is automatically selected.

[0048] The server 101 may use information about other users' actions together with information about other users as conditions for determining how to display the avatar. The information about other users' actions is, for example, information about whether the user has purchased tickets for a live performance or whether the user has paid for a product. For example, when an artist holds a live performance in a virtual space, the server 101 can perform control so that a real avatar is displayed to users who have purchased tickets, and a deformed avatar of the artist is displayed to users who have not purchased tickets.

[0049] The server 101 may use information on other users and information on the time spent by other users in the virtual space as conditions for determining how to display the avatar. For example, when an artist holds a live concert in the virtual space, the server 101 controls the display so that a real avatar is displayed to users who have not purchased tickets until a predetermined time has elapsed, and after the predetermined time has elapsed, a deformed avatar of the artist is displayed. In other words, the server 101 can change the way the avatar is displayed based on the time spent by other users in the virtual space.

[0050] The server 101 may process the avatar to display it by changing the resolution when displaying the avatar, applying effects such as mosaic and blurring, changing the color to monochrome, changing the shader to a simple one, etc. For example, when an artist user holds a live performance in a virtual space, the server 101 controls the user terminal 102 of a user who has purchased a ticket to display the avatar as is. On the other hand, the server 101 controls the user terminal 102 of a user who has not purchased a ticket to display an avatar with a lower resolution or with mosaic or blurring applied.

[0051] It is assumed that the setting UI 401 is used before the user causes the avatar to participate in the virtual space, but the server 101 may be configured to allow the user to change the settings of how the user's avatar appears to other avatars while the user's avatar is participating in the virtual space. A specific example of a UI for changing the way the user's avatar appears to other avatars while the user's avatar is participating in the virtual space will be described later using 5(C).

[0052] 5(A) to 5(D) are diagrams showing specific examples of changing the way avatars are displayed. FIG. 5(A) to 5(D) show the state of group counseling in a virtual space. There are four participants in the group counseling: a main counselor, a sub-counselor, patient A, and patient B. As in the example described in FIG. 4, patient A sets it so that avatar Y with a facial image is displayed to other users who are counselors, and avatar Y without a facial image is displayed to other users who are patients (other users in the "other" category).

[0053] 5(A) shows the virtual space as viewed by the sub-counselor on his / her own user terminal 102. In the sub-counselor's user terminal 102, the display 202 displays the main counselor's avatar 501 and patient A's avatar 502. Since patient A has set the avatar to be shown to the counselor along with a facial image, the display 202 also displays patient A's facial image 503. In the main counselor's user terminal 102, the display 202 also displays patient A's facial image 503 along with the sub-counselor's avatar and patient A's avatar 502.

[0054] FIG. 5(B) shows the virtual space as seen by Patient B on his / her own user terminal 102. Patient B is in a different location in the virtual space from the sub-counselor, and therefore the angle from which Patient B views the main counselor and Patient A is different from that of the sub-counselor. Since Patient B is not in the sub-counselor's field of vision, Patient B's avatar is not displayed on the display 202 of the sub-counselor's user terminal 102, as shown in FIG. 5(A). Also, Patient B's Since the sub-counselor is not within the field of view, the sub-counselor's avatar is not displayed on the display 202 of the user terminal 102 of the patient B, as shown in FIG. 5(B).

[0055] Furthermore, since patient A sets the avatar to be shown to patient B without a facial image, the facial image of patient A is not displayed on the display 202 of the user terminal 102 of patient B, as shown in Fig. 5(B). In other words, the facial image of patient A is not displayed in the virtual space seen by patient B.

[0056] When patient A's avatar is displayed for each other user participating in the same virtual space, the server 101 can perform display control to display patient A's avatar in different display formats on the user terminals 102 of users in different categories based on patient A's settings.

[0057] When a PC or the like is used as the user terminal 102, the size of the face image 503 of Patient A shown in FIG. 5(A) may cause the sub-counselor to miss small changes in the patient's facial expression. Therefore, for a user in the position of a counselor, the user terminal 102 may display the face images of other users (Patient A and Patient B) larger than those in FIG. 5(A) as shown in FIG. 5(C) to make it easier to observe their facial expressions. FIG. 5(C) shows an example in which the face image 503 of Patient A and the face image 505 of Patient B are displayed larger than those in FIG. 5(A) along with a reduced screen of the screen of FIG. 5(A). The layout change to the screen of FIG. 5(C) may be performed by the user terminal 102 based on information such as the user's position, or may be performed by instructions from the server 101. The layout change to the screen of FIG. 5(C) may be performed by instructions from users such as the sub-counselor, Patient A, and Patient B.

[0058] FIG. 5(D) shows an example of a UI for setting how the user's avatar is to be displayed while participating in the virtual space. For example, when patient B selects the main counselor's avatar 501 on the screen of FIG. 5(B) using a controller or the like and issues an instruction to set how the user's avatar is to be displayed, the display 202 displays a setting screen 504 in the virtual space. On the setting screen 504, patient B can set how the main counselor is to see the user's avatar. The user terminal 102 accepts the setting from the user and records or updates how the main counselor sees patient B's avatar. In this way, the user can change how the user's avatar is to be displayed even while participating in the virtual space.

[0059] The user terminal 102 may accept settings for whether or not to process the user's own voice while the user is participating in the virtual space, in addition to the display of the avatar. For example, patient B can select another patient A's avatar and set to process patient B's own voice for patient A, so that patient A cannot hear his or her actual voice.

[0060] 6A and 6B are flowcharts illustrating the processing of the communication system 100 according to the first embodiment. The processing shown in Fig. 6A is processing using a technique called remote rendering, in which the server 101 renders the video displayed on each user terminal 102. The processing shown in Fig. 6B is processing using a technique called local rendering, in which the user terminal 102 renders the video. The processing of the communication system 100 according to the first embodiment may be realized using either technique.

[0061] First, the process using remote rendering in Fig. 6A will be described. Fig. 6A illustrates the process between the user terminal 102 of a first user (hereinafter, referred to as the first user terminal 102) and the server 101. The server 101 executes the same process between the user terminal 102 of a user other than the first user who participates in the virtual space as the process between the server 101 and the first user terminal 102.

[0062] Steps S601 to S603 are processing in which the first user sets how the avatar will be presented to other users. In step S601, the first user terminal 102 accepts an instruction from the first user and sets in what display form the first user's avatar will be presented to other users. The process of setting how the avatar will be presented is as described with reference to FIG. 4. In step S602, the first user terminal 102 transmits information on how the avatar will be presented, set in S601, to the server 101. In step S603, the server 101 records the received information on how the first user's avatar will be presented.

[0063] Steps S604 to S606 are a process for allowing a first user to participate in a virtual space. In step S604, the first user terminal 102 receives an instruction to participate in the virtual space from the first user. The first user terminal 102 acquires, from the storage 212 or the like, identification information for the virtual space in which the first user has instructed the user to participate.

[0064] In step S605, the first user terminal 102 transmits the identification information of the virtual space acquired in step S604 to the server 101, and requests the first user to participate in the virtual space. In step S606, the server 101 allows the first user to participate in the virtual space corresponding to the identification information received from the first user terminal 102.

[0065] Steps S607 to S619 are a loop process that is repeated until all users, including the first user, have left the virtual space. In step S608, the first user terminal 102 acquires various pieces of information from the first user.

[0066] The user terminal 102 can acquire, for example, the following information. The user terminal 102 can capture the user's face with the camera 210 and acquire information on the user's facial expression from the captured face of the user using facial expression analysis technology. The user terminal 102 can acquire the voice uttered by the user with the microphone 208. When the user terminal 102 is an HMD or when the user terminal 102 is communicably connected to an HMD worn by the user, the user terminal 102 can detect the movement of the HMD with a gyro sensor and acquire information on the movement of the user's head. The user terminal 102 can communicate with a controller held by the user via the short-range communication I / F 213 and acquire information on the operation instructions of the avatar input by the user using the controller. The user terminal 102 can acquire information on the movement of the user's arm from the controller, which is detected by a gyro sensor built into the controller.

[0067] In step S609, the first user terminal 102 transmits the user information acquired in step S608 to the server 101. The first user terminal 102 selects which information of the user information acquired in step S608 to transmit to the server 101 according to the way the avatar is displayed set in step S601. For example, when the avatar of the first user and a facial image of the first user are displayed on the user terminal 102 of the second user (hereinafter referred to as the second user terminal 102), the first user terminal 102 transmits the facial image of the first user together with the avatar information. In step S610, the server 101 receives the information of the first user from the first user terminal 102.

[0068] Steps S611 to S618 are loop processes that are repeated the number of times equal to the number of second users who participate in the virtual space and view the images in the virtual space. In step S612, the server 101 acquires information about the second user, for example, as follows. The server 101 can acquire the second user's user name, information about qualifications, information about groups to which the second user belongs, and the like, by referring to the account information of the second user.

[0069] The server 101 receives the status (qualifications, groups to which the user belongs, etc.) of the second user from an external system. For example, the server 101 may inquire of an electronic medical record system in a hospital to obtain whether or not the second user is registered as a counselor. By obtaining information on the position of the second user in cooperation with an external system, the server 101 can reduce the risk of impersonation.

[0070] The server 101 can also classify the second user based on group information set by the first user and change the way the avatar is displayed for each group to which the user belongs. The server 101 may set the group set by the first user as the position (classification) of the other user. For example, the first user who is a patient classifies the second user who is a counselor into one of the groups of "trusted counselors" and "untrusted counselors". The server 101 can control so that the face image of the first user is displayed to the users in the "trusted counselors" group and the face image of the first user is not displayed to the users in the "untrusted counselors" group. By allowing the first user to set the group and classify the second user, the server 101 can display the avatar of the first user in a different way to users who are objectively in the same position.

[0071] In step S613, the server 101 determines the avatar of the first user to be shown to the second user and how the avatar will be displayed, based on the information on how the avatar of the first user will be displayed recorded in step S603.

[0072] In step S614, server 101 determines a 3D scene in the virtual space to be shown to the second user, and generates data of the 3D scene in a data format capable of describing three-dimensional computer graphics, such as X3D. Server 101 generates a 3D model of the avatar of the first user in the virtual space according to the way the avatar of the first user is to be shown, determined in step S613.

[0073] In step S615, the server 101 renders the data of the 3D scene of the virtual space generated in step S614, and generates a video seen from the viewpoint of the second user in a data format such as MP4. In step S616, the server 101 transmits the video generated in step S615 to the second user terminal 102. In step S617, the second user terminal 102 plays the video received from the server 101 on the display 202.

[0074] 4, when the purpose of communication is used as a condition for determining how the avatar is to be displayed, the server 101 executes a process of acquiring the purpose of communication after executing step S612. The server 101 can accept, for example, a setting of the purpose of communication inputted by any user participating in the virtual space via the user terminal 102.

[0075] Furthermore, when the purpose of communication is linked to information about the virtual space, such as "virtual space for counseling," the server 101 can set the purpose of communication based on the information about the virtual space.

[0076] The server 101 may also infer the purpose of communication from a user participating in the virtual space. For example, when a user with an account that has counselor attributes participates in the virtual space, the server 101 can infer that the purpose of communication is "counseling." Furthermore, the server 101 can analyze the appearance of the avatar, and infer that the purpose of communication is "examination or counseling" when, for example, an avatar wearing a white coat is present.

[0077] As described with reference to Fig. 4, when information about the virtual space is used as a condition for determining how the avatar is to be displayed, the server 101 executes a process of acquiring information about the virtual space after executing step S612. The information about the virtual space is, for example, the name of the virtual space registered in the communication system 100, a description of the virtual space entered by an administrator of the communication system 100 or a user participating in the virtual space, furnishings present in the virtual space, etc. The server 101 can recognize the situation of the virtual space, for example, by AI technology, based on the information about the virtual space, and change the way the avatar is displayed according to the situation of the virtual space, etc.

[0078] 4, when the number of communications or the duration is used as a condition for determining how the avatar is to be displayed, the server 101 acquires the number of communications and the duration between the first user and the second user after executing step S612. To acquire the number of communications and the duration, the server 101 records the number of communications and the duration between the first user and the second user.

[0079] The number of communications may be, for example, the number of times that a first user and a second user participate in the same virtual space. The number of communications may also be the number of times that a first user and a second user participate in the same virtual space with a specific purpose. The number of communications may also be the number of times that a first user and a second user participate in a specific virtual space together.

[0080] The communication time may be, for example, the time during which the first user and the second user converse with each other through the avatar. The communication time may be the time during which the first user and the second user participate together in the same virtual space. The communication time may be the time during which the first user and the second user participate together in the same virtual space with a predetermined purpose. The communication time may be the time during which the first user and the second user participate together in a specific virtual space.

[0081] As described with reference to FIG. 4, when information on the appearance of another user's avatar is used as a condition for determining how the avatar is to be displayed, the server 101 executes a process of acquiring information on the second user's avatar after executing step S612. For example, the server 101 may acquire information on the appearance of the second user's avatar by analyzing a 3D model of the second user's avatar using AI. The server 101 may also acquire, as the information on the second user's avatar, the name of the second user's avatar or an explanation added to the avatar by the second user, and determine how the first user's avatar is to be displayed based on the acquired information on the second user's avatar.

[0082] As described with reference to FIG. 4, when information on the behavior of other users is used as a condition for determining how to display an avatar, the server 101 executes a process of acquiring information on the behavior of the second user (other user) after executing step S612. The server 101 can acquire information on the behavior of the second user by, for example, recording the behavior history of the second user in the storage 307 or the like. The server 101 may also acquire information on the behavior of the second user by inquiring of an external system.

[0083] The information on the user's behavior is, for example, information on whether or not the user has purchased a ticket for a live performance. In such an example of a live performance in a virtual space, the server 101 can obtain information on whether or not the second user has purchased a ticket from a ticket sales system. The server 101 can change the way the avatar of the first user is displayed depending on whether or not the user has performed a predetermined behavior such as purchasing a ticket.

[0084] Next, a process using local rendering in Fig. 6B will be described. The processes in steps S601 to S614 are the same as those in Fig. 6A denoted by the same reference numerals, and therefore a description thereof will be omitted.

[0085] After determining the 3D scene of the virtual space to be shown to the second user in step S614, in step S631, the server 101 notifies the second user terminal 102 of the determined 3D scene. The 3D scene is represented in a data format capable of describing three-dimensional computer graphics, such as X3D.

[0086] In step S632, the second user terminal 102 renders the 3D scene of the virtual space notified by the server 101, and generates an image seen from the viewpoint of the second user. In step S617, the second user terminal 102 plays the image generated in step S632 on the display 202.

[0087] According to the above-mentioned embodiment 1, the first user can change the way the avatar of the first user is displayed when the other user is a counselor and when the other user is a patient by setting the way the avatar is displayed via the setting UI 401 shown in Fig. 4. That is, the server 101 can control so that not only the avatar of the first user but also the face image is displayed in the virtual space seen by the counselor, and only the avatar of the first user is displayed in the virtual space seen by the other patients. Therefore, when the server 101 displays the avatar of the first user on the user terminal 102 of the second user participating in the virtual space, the server 101 can display the avatar of the first user in an appropriate display form according to the position of the second user.

[0088] The process of step S614 in FIG. 6A and FIG. 6B does not have to be performed for each second user. The server 101 can reuse the 3D scene generated in step S614 between second users who have a common way of displaying the avatar of the first user. For example, in step S614, the server 101 may reuse the 3D scene when the second user is a main counselor in the process when the second user is a sub-counselor. If the first user has the same way of displaying the avatar, the server 101 can make the process more efficient by reusing the 3D scene of the virtual space generated in S614.

[0089] 4 is assumed to be displayed on an HMD serving as the first user terminal 102, the user terminal 102 is not limited to an HMD and may be a terminal such as a PC, a smartphone, a tablet, etc. The terminal such as a PC, a smartphone, or a tablet serving as the user terminal 102 may display the setting UI 401 on the display 202 and accept an input of settings for how the avatar is to be displayed from the first user.

[0090] <Embodiment 2> In the first embodiment, the communication system 100 is constructed as a client-server system. However, the present invention can also be realized as a serverless system. The communication system according to the second embodiment is constructed as a serverless system.

[0091] FIG. 7 is a diagram showing an example of the configuration of a communication system 700 according to the second embodiment. The communication system 700, which is an example of an information processing system, includes a plurality of user terminals 701 connected in a peer-to-peer manner via a network such as the Internet. The user terminal 701 is the same as the user terminal 102 described in the first embodiment, and therefore a detailed description thereof will be omitted. In addition, the hardware configuration of the user terminal 701 is the same as the user terminal 102 of the first embodiment described in FIG. 2, and therefore a description thereof will be omitted. In the second embodiment, a user The setting UI for setting how the avatar is to be displayed is similar to the setting UI 401 according to the first embodiment described with reference to FIG.

[0092] 8 is a flowchart illustrating the processing of the communication system 700 according to the second embodiment. In step S801, the user terminal 701 of the first user (hereinafter, referred to as the first user terminal 701) receives an instruction from the first user and sets the display form in which the avatar of the first user is to be shown to other users. As described in FIG. 4, the first user terminal 701 can receive the setting from the first user via the setting UI 401. The setting process for the way the avatar is shown is executed by the user terminal 701 of the user participating in the virtual space.

[0093] In step S802, the first user terminal 701 records the settings received from the first user in step S801 in the RAM 204, the storage 212, or the like.

[0094] Steps S803 to S810 are processes performed when a first user participates in a virtual space. Fig. 8 shows processes performed between a first user terminal 701 and a user terminal 701 of a second user (hereinafter, referred to as a second user terminal 701), but the first user terminal 701 performs the same processes with the user terminals 701 of each user participating in the virtual space.

[0095] In step S803, the first user terminal 701 accepts an instruction to participate in a virtual space from the first user. The first user terminal 102 acquires, from the storage 212 or the like, identification information of the virtual space instructed to participate by the first user.

[0096] In step S804, the first user terminal 701 notifies the second user terminal 701 that the first user will participate in the virtual space identified by the identification information by transmitting the identification information of the virtual space acquired in step S803 to the second user terminal 701. In step S805, the second user terminal 701 records that the first user has participated in the virtual space.

[0097] In step S806, the second user terminal 701 acquires information about the second user, and transmits the acquired information about the second user to the first user terminal 701. The method of acquiring the information about the second user is the same as the process of step S612 in Fig. 6A. In step S807, the first user terminal 701 receives information about the second user from the second user terminal 701.

[0098] In step S808, the first user terminal 701 determines the avatar of the first user to be shown to the second user and how to display it, based on the information of the second user received in step S807 and the way to display the avatar of the first user recorded in step S802. The process of step S808 is the same as the process of step S613 in Fig. 6A, so a detailed description will be omitted.

[0099] In step S809, the first user terminal 701 transmits to the second user terminal 701 the avatar of the first user to be shown to the second user and information on how to display the avatar, based on the result determined in step S808. The first user terminal 701 generates, for example, a 3D model of the avatar of the first user to be shown to the second user in a data format capable of describing three-dimensional computer graphics, such as X3D. The first user terminal 701 transmits the generated 3D model of the avatar of the first user to the second user terminal 701. In addition, when displaying a facial image of the first user together with the avatar of the first user to the second user, the first user terminal 701 also transmits information for displaying the facial image of the first user to the second user terminal 701. In step S810, the second user terminal 701 transmits to the first user terminal 701 the avatar of the first user to be shown to the second user and information on how to display the avatar. Receive from 01.

[0100] Steps S811 to S820 are loop processes that are repeated until all users, including the first user, leave the virtual space. In step S812, the first user terminal 701 acquires various information from the first user. Examples of information acquired from the first user are the same as those in step S608 in FIG. 6A, and therefore will not be described.

[0101] The processing from steps S813 to S819 is repeated the number of times corresponding to the number of second user terminals 701 that communicate with the first user terminal 701 in the virtual space. In step S814, the first user terminal 701 transmits the information of the first user acquired in step S812 to the second user terminal 701. The processing in step S814 is the same as step S609 in FIG. 6A, and therefore description thereof will be omitted. In step S815, the second user terminal 701 receives the information of the first user from the first user terminal 701.

[0102] In step S816, the second user terminal 701 determines a 3D scene in the virtual space to be shown to the second user, and generates data of the 3D scene in a data format capable of describing three-dimensional computer graphics, such as X3D. The second user terminal 701 can generate data of the 3D scene in the virtual space to be shown to the second user, using the avatar and presentation of the first user received in step S810 and the information of the first user received in step S815.

[0103] In step S817, the second user terminal 102 renders the data of the 3D scene of the virtual space generated in step S816 to generate a frame of the video seen from the viewpoint of the second user. In step S818, the second user terminal 102 plays the video generated in step S817 on the display 202.

[0104] According to the above embodiment 2, the serverless communication system 700 described in FIG. 7 can control the display of the first user's avatar in a display form that corresponds to the position of other users, based on the first user's settings for how the avatar will be displayed.

[0105] For example, Patient A receiving counseling can set the virtual space displayed on the counselor's user terminal 701 to display an image of Patient A's face, allowing the counselor to check the actual facial expression of Patient A. On the other hand, Patient A can set the virtual space displayed on the user terminal 701 of another Patient B not to display an image of Patient A's face, so that Patient B cannot see Patient A's face, thereby protecting his / her privacy.

[0106] When displaying the avatar of patient A on the user terminal 701 of another user participating in the virtual space, the communication system 700 can display the avatar in an appropriate display form depending on the position of the other user (psychiatrist, counselor, other patient, etc.). That is, when displaying the avatar of a first user on the user terminal 102 of a second user participating in the virtual space, the server 101 can display the avatar of the first user in an appropriate display form depending on the position of the second user.

[0107] (Other embodiments) Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be appropriately combined.

[0108] In addition, a software program for implementing the functions of the above-described embodiments can be implemented in a system having a computer that can execute the program directly from a recording medium or via wired / wireless communication. The present invention also includes cases where the program is supplied to a system or device and the program is executed. Therefore, the program code itself that is supplied to and installed in a computer to realize the functional processing of the present invention can also realize the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In that case, the form of the program is not important, so long as it has the function of the program, such as object code, a program executed by an interpreter, script data supplied to an OS, etc.

[0109] The recording medium for supplying the program may be, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory. The method for supplying the program may be, for example, a method in which a computer program for realizing the present invention is stored in a server on a computer network, and a client computer connected to the server downloads and executes the computer program.

[0110] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0111] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) A first acquisition means for acquiring information on a plurality of display forms of an avatar of a first user; A second acquisition means for acquiring information of a second user in a virtual space in which the first user participates; a determining means for determining a display form of the avatar of the first user to the second user, from among the plurality of display forms of the avatar of the first user, based on information of the second user; 13. An information processing device comprising: (Configuration 2) The display mode of the avatar of the first user is determined by the determining means, and a transmitting means is configured to generate an image of the virtual space displaying the avatar of the first user in the display mode of the avatar of the first user determined by the determining means, and transmit the image to a user terminal of the second user. 2. The information processing device according to configuration 1. (Configuration 3) The display mode of the avatar of the first user is determined by the determining means, and a notification means is further provided for determining a scene of the virtual space in which the avatar of the first user is displayed, and notifying a user terminal of the second user of the determined scene. 2. The information processing device according to configuration 1. (Configuration 4) The display mode of the avatar of the first user for the second user is set by a first setting unit. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) The first setting means accepts the setting of the display mode of the avatar of the first user by the first user selecting an avatar of the second user while participating in the virtual space. 5. The information processing device according to configuration 4. (Configuration 6) The plurality of display forms of the avatar of the first user includes a form that displays a 3D shape obtained from a face of the first user. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) The plurality of display forms of the avatar of the first user include a form in which a part or all of a facial image of the first user is displayed together with the avatar of the first user. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) The plurality of display forms of the avatar of the first user include a form in which the avatar of the first user is processed and displayed. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) The plurality of display forms of the avatar of the first user include a form in which a face image of the avatar of the first user is enlarged and displayed together with the image of the virtual space. 9. The information processing device according to any one of configurations 1 to 8. (Configuration 10) The voice recognition device further includes a second setting means for receiving a setting from the first user as to whether or not the voice of the first user to be reproduced for the second user is to be processed. 10. The information processing device according to any one of configurations 1 to 9. (Configuration 11) The second setting means receives the setting of whether or not to process the voice of the first user by selecting an avatar of the second user while the first user is participating in the virtual space. 11. The information processing device according to configuration 10. (Configuration 12) The determining means determines a display form of the avatar of the first user to the second user based on information of the second user and a purpose of communication between the first user and the second user. 12. The information processing device according to any one of configurations 1 to 11. (Configuration 13) The purpose of the communication is set based on information about the virtual space or is set by the first user. 13. The information processing device according to configuration 12. (Configuration 14) The determining means determines a display form of the avatar of the first user to the second user based on information about the second user and information about the virtual space. 12. The information processing device according to any one of configurations 1 to 11. (Configuration 15) The determining means determines a display form of the avatar of the first user to the second user based on information of the second user and a frequency or duration of communication between the first user and the second user. 12. The information processing device according to any one of configurations 1 to 11. (Configuration 16) The information processing device of any one of configurations 1 to 11, characterized in that the determination means determines the display form of the avatar of the first user to the second user based on information of the second user and information of the avatar of the second user. (Configuration 17) The determining means determines a display form of the avatar of the first user to the second user based on information of the second user and an action of the second user. 12. The information processing device according to any one of configurations 1 to 11. (Configuration 18) The determining means determines a display form of the avatar of the first user to the second user based on information of the second user and a stay time of the second user in the virtual space. Decide 12. The information processing device according to any one of configurations 1 to 11. (Configuration 19) The plurality of display forms of the avatar of the first user can be set for each of the second users or for each category of the second users. 19. The information processing device according to any one of configurations 1 to 18. (Configuration 20) a display control means for controlling the display of an avatar of a first user in a different display form for each user who participates in the same virtual space as a first user; The display control means a user terminal of a second user who participates in the same virtual space as the first user displays an avatar of the first user in a first display form; In a user terminal of a third user who participates in the same virtual space as the first user, an avatar of the first user is displayed in a second display form different from the first display form. 23. An information processing apparatus comprising: (Configuration 21) An information processing device according to any one of configurations 1 to 19, a user terminal of the second user; and Equipped with The user terminal, a playback means for playing back an image of the virtual space in which the avatar of the first user is displayed in the display mode of the avatar of the first user determined by the determination means; An information processing system comprising: (method) A first acquisition step of acquiring information on a plurality of display forms of an avatar of a first user; A second acquisition step of acquiring information of a second user in the virtual space in which the first user participates; a determining step of determining a display form of the avatar of the first user to the second user from among the plurality of display forms of the avatar of the first user based on information of the second user; 13. An information processing method comprising: (program) A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 20. [Explanation of symbols]

[0112] 101: Server (information processing device), 306: CPU 701: user terminal (information processing device), 201: CPU

Claims

1. The system includes a display control means that controls the display of the first user's avatar for each user participating in the same virtual space as the first user. The display control means is On the user terminal of the second user who is participating in the same virtual space as the first user, the avatar of the first user is displayed in the first display mode among multiple display modes. On the user terminal of a third user participating in the same virtual space as the first user, the avatar of the first user is displayed in a second display mode, which is different from the first display mode among the multiple display modes. An information processing device characterized by the following:

2. A first acquisition means for acquiring information of the plurality of display forms, The second acquisition means for acquiring information of the second user, A determination means for determining the first display form among the plurality of display forms based on the second user's information. The information processing apparatus according to claim 1, characterized by having the following features.

3. The system further includes a transmission means that generates an image of the virtual space displaying the avatar of the first user in the first display mode determined by the determination means, and transmits it to the user terminal of the second user. The information processing apparatus according to feature 2.

4. The system further includes a notification means that determines a scene in the virtual space displaying the avatar of the first user in the first display mode determined by the determination means, and notifies the user terminal of the second user. The information processing apparatus according to feature 2.

5. The determination means determines the first display format based on the information of the second user and the purpose of communication between the first user and the second user. The information processing apparatus according to feature 2.

6. The purpose of the aforementioned communication is set based on information regarding the virtual space. , or set by the first user The information processing apparatus according to feature 5.

7. The determination means determines the first display mode based on the information of the second user and the information of the virtual space. The information processing apparatus according to feature 2.

8. The determination means determines the first display format based on the information of the second user and the number or duration of communications between the first user and the second user. The information processing apparatus according to feature 2.

9. The determination means determines the first display form based on the information of the second user and the information of the second user's avatar. The information processing apparatus according to feature 2.

10. The determination means determines the first display form based on the information of the second user and the actions of the second user. The information processing apparatus according to feature 2.

11. The determination means determines the first display mode based on the information of the second user and the time the second user spends in the virtual space. The information processing apparatus according to feature 2.

12. The system further includes a first setting means for receiving the setting of the first display format from the first user. The information processing apparatus according to feature 1.

13. The first setting means accepts the setting for the display format of the first user's avatar when the first user selects the second user's avatar while participating in the virtual space. The information processing apparatus according to feature 12.

14. The plurality of display modes include a mode that displays a 3D shape acquired from the face of the first user. The information processing apparatus according to feature 1.

15. The plurality of display modes include a mode in which part or all of the first user's face video is displayed together with the first user's avatar. The information processing apparatus according to feature 1.

16. The plurality of display forms include a form in which the avatar of the first user is processed and displayed. The information processing apparatus according to feature 1.

17. The plurality of display modes include a mode in which the face image of the first user's avatar is enlarged and displayed together with the image of the virtual space. The information processing apparatus according to feature 1.

18. The system further includes a second setting means for receiving a setting from the first user indicating whether or not to process the first user's voice, which is to be played back to the second user. The information processing apparatus according to feature 1.

19. The second setting means accepts the setting of whether or not to process the first user's voice by having the first user select the second user's avatar while participating in the virtual space. The information processing apparatus according to feature 18.

20. The plurality of display modes can be set for each user or for each user category. The information processing apparatus according to feature 1.

21. An information processing device according to any one of claims 1 to 20, The user terminal of the second user, The user terminal of the third user and Equipped with, The user terminal of the second user is The system has a playback means for playing back the video of the virtual space displaying the avatar of the first user in the first display mode, The user terminal of the third user is The second display mode includes a playback means for playing back the video of the virtual space displaying the avatar of the first user. An information processing system characterized by the following:

22. A display control step that controls the display of the first user's avatar for each user who participates in the same virtual space as the first user, In the aforementioned display control step, On the user terminal of the second user who is participating in the same virtual space as the first user, the avatar of the first user is displayed in the first display mode among multiple display modes. An information processing method characterized in that, on the user terminal of a third user who participates in the same virtual space as the first user, the avatar of the first user is displayed in a second display mode, which is different from the first display mode, among the plurality of display modes.

23. A program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 20.

Images