Information processing device, information processing system, information processing method, and program

Abstract

An information processing apparatus includes: a first acquisition unit configured to acquire information in a plurality of display forms of an avatar of a first user; a second acquisition unit configured to acquire information of a second user in a virtual space in which the first user participates; and a determination unit configured to determine a display form of the avatar of the first user for the second user among a plurality of display forms of the avatar of the first user based on the information of the second user.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. Background Art

[0002] With the development and widespread use of virtual reality (VR) technology, research is underway to utilize virtual spaces for various purposes, including distribution, commerce, and medical care. For example, applications in the medical field have made it possible for patients who have difficulty traveling due to physical or mental disabilities, or who live in areas with limited access to hospitals, such as remote islands, to receive medical care and consultations in virtual spaces. Furthermore, by using avatars (the user's virtual self) in virtual spaces, even patients whose faces are widely known or who are concerned about being watched can communicate openly without being seen by other users, making it easier for them to participate in medical examinations and consultations.

[0003] Group counseling is a method of counseling that brings together multiple patients. In group counseling, patients can more easily see themselves through the reactions and interpersonal relationships of others, and can view things more flexibly by incorporating other people's thoughts and perspectives.

[0004] Previous studies have shown that using an avatar that does not resemble the user is more likely to prompt self-disclosure. In group counseling sessions held in a virtual space, it is expected that the patient can have more effective counseling sessions using an avatar that does not look like him / her. [Prior art literature] [Patent Document]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-104482 Summary of the Invention [Technical problem to be solved by the invention]

[0006] In group counseling, for privacy reasons and because patients can express their true feelings without being noticed, it's desirable for them to use an avatar that closely resembles their actual facial image. However, since psychiatrists and counselors can assess a patient's mental state and the severity of their symptoms by observing their actual facial expressions, using an avatar is discouraged. Specifically, when a patient's avatar is displayed on other users' devices, the appropriate display format varies depending on whether the other user is a psychiatrist, counselor, or other patient.

[0007] Even for purposes other than group consultation, when displaying a user's avatar on other users' terminals in a virtual space, the appropriate display form of the avatar may differ depending on the other user's perspective. For example, when a user conducts business negotiations in a virtual space, the user may want to display their usual avatar to colleagues, while displaying an avatar that resembles the businessperson's appearance to business partners, to create a better impression. As another example, when conducting a live event in a virtual space, the organizer may want to display a realistic avatar that faithfully mimics the performer's actual appearance to users who have purchased tickets, while displaying a deformed avatar to users who have not yet purchased tickets. By switching avatars, the organizer of the live event can host the live event for fans and promote the event to users who are not fans.

[0008] As a technology for changing the display of avatars, Patent Document 1 discloses a technology for identifiably displaying the avatars of users with more common activity history (such as adding balloons, etc.) However, even considering the common activity history, it may be difficult to appropriately switch the avatars according to the user's position.

[0009] An object of the present invention is to provide a technology for displaying a user's avatar in an appropriate display form according to the stance of another user who participates in a virtual space when the avatar of the user is displayed on the terminal of the other user. [Solutions to solve the problem]

[0010] According to the present invention, an information processing device includes: a first acquisition unit, which is configured to acquire information on multiple display forms of an avatar of a first user; a second acquisition unit, which is configured to acquire information on a second user in a virtual space in which the first user participates; and a determination unit, which is configured to determine the display form of the first user's avatar for the second user among the multiple display forms of the first user's avatar based on the information of the second user. [Advantageous Effects of the Invention]

[0011] According to the present invention, when displaying the user's avatar on the terminal of another user participating in the virtual space, the avatar can be displayed in an appropriate display form according to the position of the other user. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a diagram illustrating a configuration example of a communication system according to the first embodiment. Figure 2 This is a diagram illustrating the hardware configuration of a user terminal. Figure 3 This is a diagram illustrating the hardware configuration of a server. Figure 4is a diagram illustrating an example of a user interface for setting a display method of an avatar. Figure 5 A to Figure 5 D is a diagram illustrating a specific example of changing the way an avatar is displayed. Figure 6A is a flowchart illustrating the processing of the system according to the first embodiment. Figure 6B is a flowchart illustrating another process of the system according to the first embodiment. Figure 7 is a diagram illustrating a configuration example of a communication system according to the second embodiment. Figure 8 is a flowchart illustrating the processing of the system according to the second embodiment. DETAILED DESCRIPTION

[0013] Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings.

[0014] <First embodiment> Figure 1 1 is a diagram illustrating a configuration example of a communication system 100 according to the first embodiment. The communication system 100, which is an example of an information processing system, is a system configured as a client-server system. The communication system 100 is applied to a remote medical assistance system, for example.

[0015] The communication system 100 includes a server 101 and a plurality of user terminals 102 connected to the server 101 via a network such as the Internet. The server 101 is, for example, a personal computer (PC). The user terminals 102 are, for example, electronic devices such as a PC, a smartphone, a tablet, a head-mounted display (HMD), and a controller capable of communicating with the HMD. In the following description, the user terminal 102 is the HMD. In addition to the HMD as the user terminal 102, the communication system 100 may also include other electronic devices, and may include different types of electronic devices. In the following description, the HMD is assumed to have a function of being able to directly connect to the network. Note that the HMD can also be connected to the network via another PC or smartphone, etc.

[0016] Figure 2 2 is a diagram illustrating the hardware configuration of the user terminal 102. The CPU 201 comprehensively controls various functions of the user terminal 102 via the internal bus 206 using programs stored in the read-only memory (ROM) 203. Various processes executed by the user terminal 102 are implemented by the CPU 201. The CPU 201 can project the results of program execution as a video through the display 202 and display it in the user's field of view.

[0017] The ROM 203 is, for example, a flash memory or the like, and stores various setting information, application programs, etc. A random access memory (RAM) 204 is used as a memory and a work area of ​​the CPU 201. A network interface (I / F) 205 is a module for connecting to a network.

[0018] Despite Figure 2 Although briefly illustrated in FIG, the sensor unit 209 includes one or more sensors. Specifically, the sensor unit 209 includes at least one sensor such as a global positioning system (GPS), a gyro sensor, an acceleration sensor, a proximity sensor, and a blood pressure / heart rate / brain wave measurement sensor. The sensor unit 209 may include a sensor capable of detecting biometric information for implementing fingerprint authentication, vein authentication, iris authentication, and the like.

[0019] The camera 210 is, for example, a fisheye camera mounted inside the HMD serving as the user terminal 102, and has a function of capturing the user's face. The captured image data is stored in the RAM 204 after distortion due to the fisheye lens is removed. The storage device 212 is a storage medium that stores various data, such as applications.

[0020] The short-range communication I / F 213 is an interface used for communication with the controller. The user can perform gesture input to the user terminal 102 by moving the controller they are holding, or can instruct the user terminal 102 by operating buttons or joysticks included in the controller. The controller may include sensors for measuring the user's heart rate, pulse, perspiration, etc. The user terminal 102 can communicate with a wearable device worn by the user via the short-range communication I / F 213 to obtain the user's heart rate, pulse, perspiration, etc. In addition, the user terminal 102 can communicate with a camera and sensor array installed in the user's room via the short-range communication I / F 213 to obtain information related to the room and the user.

[0021] The microphone 208 captures the voice uttered by the user, and the speaker 211 reproduces the voices of other users participating in the communication system 100, sound effects, background music, and the like.

[0022] Figure 3 1 is a diagram illustrating the hardware configuration of server 101 (information processing device). Server 101 includes a display unit 301, video random access memory (VRAM) 302, a bit shift unit (BMU) 303, a keyboard 304, and a pointing device (PD) 305. Furthermore, server 101 includes a CPU 306, a storage device 307, RAM 308, ROM 309, a memory card 310, a network interface 311, and a bus 312.

[0023] The display unit 301 displays, for example, live view images, icons, messages, menus, and other user interface information. The VRAM 302 stores information about moving images to be displayed on the display unit 301. Data generated in the VRAM 302 is transferred to the display unit 301 according to predetermined standards and displayed on the display unit 301.

[0024] The BMU 303 controls, for example, data transfer between memories (for example, between the VRAM 302 and other memories) and data transfer between memories and various I / O devices (for example, the network I / F 313 ).

[0025] The keyboard 304 includes various keys for inputting characters, etc. The PD 305 is used to select and indicate an icon, menu, or other content displayed on the display unit 301, or to drag and drop an object, for example.

[0026] The CPU 306 controls each device of the server 101 based on control programs such as an OS and various programs for implementing the functions of the server 101 stored in the storage device 307, the ROM 309, or the memory card 310. Various processes executed by the server 101 are implemented by the CPU 306.

[0027] The storage device 307 includes, for example, an HDD and an SSD. The storage device 307 stores control programs and various data to be temporarily stored. The RAM 308 is used as a work area for the CPU 306, a data storage area for error processing, and a load area for the control program. The ROM 309 is a nonvolatile memory and stores the control program executed by the server 101. The memory card 310 stores the control program and various data to be temporarily stored.

[0028] The network I / F 311 is an interface for communicating with the user terminal 102 and the like via a network. The bus 312 includes an address bus, a data bus, and a control bus. The CPU 306 can retrieve and execute a control program from the storage device 307, the ROM 309, or the memory card 310, or can retrieve and execute a control program from another information processing device and the like via the network I / F 311.

[0029] Figure 4 This is a diagram illustrating an example of a user interface (UI) for setting a presentation method (display form) of an avatar. Figure 4 This is a UI for setting the presentation method for the user Taro Suzuki. Information related to the presentation method of the avatar includes information such as which of the user's avatars to present and how to present the avatar. The presentation method of the user's avatar may include, for example, a display format that also displays a video of the actual user along with the avatar.

[0030] The setting UI 401 is displayed on the display 202 of the user terminal 102. The user can set how to display his / her avatar to other users or to each position (category) of other users using the setting UI 401. The user can pre-register multiple display forms regarding the display method of his / her avatar.

[0031] The server 101 may set a priority order for the display methods of the avatars set for each category of other users in the settings UI 401. For example, the server 101 may set the priority order to be higher from the category illustrated at the top of the screen of the settings UI 401. The server 101 may determine other users whose display methods belong to multiple categories based on the category with the higher priority.

[0032] The other user category 402 is a UI component for setting the category of other users whose avatars are displayed. For example, the user can set a specific user name, qualification, predetermined group, etc. to the other user category 402.

[0033] Category 404 shows an example of directly specifying a specific user name (e.g., Suzuki Hanako). Category 405 shows an example of specifying the qualifications of other users (e.g., consultant). Category 406 and category 407 show examples of specifying a predetermined group to which other users belong (e.g., colleagues, employees of X Co., Ltd.). The predetermined group is a group that is specified by the user who sets the display method through the setting UI 401 (in Figure 4 In the example of , a group created by Taro Suzuki) or a pre-registered group, etc. The predetermined group may be a group used to indicate the relationship between the user and other users (such as family, friends, or elders). Category 408 illustrates an example of specifying other users other than categories 404 to 407.

[0034] The avatar display mode 403 is a UI component for setting the display mode of the avatar for each category of other users. For example, the user can set which avatar to display and how to display the avatar as the avatar display mode 403.

[0035] The presentation method 409 corresponding to the category 404 shows an example in which a setting is performed so that the facial image of the user (Taro Suzuki) captured by the camera 210 is displayed together with the avatar X. In other words, the server 101 controls so that the avatar X and the facial image of Taro Suzuki are displayed on the user terminal 102 of another user (Hanako Suzuki) in the category 404.

[0036] The presentation method 410 corresponding to the category 405 shows an example in which the facial image of the user (Taro Suzuki) captured by the camera 210 is set to be displayed together with the avatar Y. In other words, the server 101 controls so that the avatar Y and the facial image of Taro Suzuki are displayed on the user terminal 102 of the other user (counselor) in the category 404. Note that the avatar X and the avatar Y are avatars of Taro Suzuki and are different from each other.

[0037] Display mode 411 corresponding to category 406 and display mode 413 corresponding to category 408 show examples in which avatar Y is set to be displayed. Display mode 412 corresponding to category 407 shows an example in which a real avatar having a 3D shape acquired from the actual face of the user (Taro Suzuki) who set the display mode is set to be displayed.

[0038] According to the above configuration, when a user (Taro Suzuki) participates in group counseling in a virtual space, the server 101 can present an avatar Y having a facial image to other users who are counselors through presentation method 410. On the other hand, the server 101 can present an avatar Y without a facial image to other users who are patients other than Taro Suzuki through presentation method 413.

[0039] Furthermore, when a user (Taro Suzuki) participates in a virtual space with colleagues and business partners (e.g., employees of X Co., Ltd.) to discuss business, server 101 may present avatar Y, which is familiar to the user as a colleague, to the user as a colleague through presentation method 411. On the other hand, server 101 may present a real avatar to the user as a business partner through presentation method 412.

[0040] Although examples of avatar display methods (display forms) have been described above, various forms can be used as avatar display methods. For example, the avatar display method can be a form in which the user's expression is reflected on the expression of a real avatar. The server 101 can reflect the user's expression on the expression of the real avatar by analyzing the expression from the user's facial image using expression analysis technology. Based on this display form, the server 101 can send the user's (patient's) expression to the counselor without displaying the facial image.

[0041] The avatar may be displayed in a manner that displays a portion of the user's facial image along with the user's avatar, rather than the entire user's facial image. For example, server 101 may hide or display the eyes of a facial image, or display only the eyes or mouth, depending on the purpose of consultation. Displaying only a portion of the facial image can make it difficult to identify the individual.

[0042] The avatar display method can be a method in which the user's voice is reproduced as is on the other user's user terminal 102 along with the avatar display, or a method in which the user's voice is processed and reproduced along with the avatar display so that the individual cannot be identified. Alternatively, the avatar display method can be a method in which the other user's user terminal 102 displays the avatar and displays the user's voice as subtitles without reproducing the voice. By processing the voice or displaying the voice as subtitles, the risk of individual identification from the voice can be reduced.

[0043] The server 101 can use the purpose of communication, along with information about other users, as a condition for determining how the avatar is displayed. The purpose of communication can be set by the user, or based on information related to the virtual space. Information related to the virtual space is pre-set for the virtual space and includes information such as the purpose of communication within the virtual space and the type of virtual space (e.g., a gaming space or a consulting space) that can be used to infer the purpose of communication.

[0044] For example, if the other user is a counselor, the server 101 controls the user's (patient's) avatar to be displayed with a facial image if the purpose of the communication is consultation. On the other hand, if the purpose of the communication is chatting, the server 101 controls the user's avatar to be displayed without a facial image. By controlling whether to display a facial image based on the purpose of the communication, it is possible to suppress the display of facial images where they are not needed.

[0045] The server 101 can use information related to the virtual space together with information about other users as conditions for determining how the avatar is displayed. For example, when the other user is a counselor, the server 101 controls the user's (patient's) avatar to be displayed with a facial image when the user (patient) communicates with the counselor in a counseling location. On the other hand, when the server 101 communicates with the counselor in a gaming location, the server 101 controls the avatar to be displayed without a facial image. By controlling whether to display a facial image based on information related to the virtual space (for example, information about the place where the avatar exists), it is possible to suppress the display of facial images in places where facial images are not needed.

[0046] As a condition for determining how to display an avatar, server 101 may use the number or duration of past communications along with information about other users. For example, if the other user is a counselor, and the number or duration of consultations exceeds a threshold, server 101 will consider the other user a trusted counselor and control the display of the avatar with a facial image. By considering the number or duration of communications, it is possible to achieve the effect of displaying facial images for counselors who have become trustworthy through repeated consultations.

[0047] Server 101 can use information about the appearance and clothing of other users' avatars, along with information about other users, as a condition for determining how to display their avatars. For example, when conducting business negotiations with business partners in a virtual space, server 101 controls the display of the user's formal avatar to other users (business partner users) using formal avatars. On the other hand, server 101 controls the display of the user's casual avatar to other users (business partner users) using casual avatars. By considering information about other avatars, it is possible to automatically select an avatar based on TPO.

[0048] Server 101 can use information related to other users' behavior, along with other user information, as a condition for determining how to display an avatar. This information can include, for example, whether a user has purchased a live broadcast ticket or paid for a product. For example, when an artist hosts a live event in a virtual space, server 101 can control the display so that the real avatar is displayed to users who have purchased tickets, while an avatar that morphs into the artist's image is displayed to users who have not yet purchased tickets.

[0049] Server 101 can use information about other users' stay in the virtual space, along with other users' information, as a condition for determining how their avatars are displayed. For example, when an artist hosts a live event in the virtual space, server 101 controls the display so that users who haven't purchased tickets are shown their real avatars before a predetermined time has passed, and are shown an avatar that morphs into the artist's avatar after the predetermined time has passed. In other words, server 101 can change the display of their avatars based on how long other users have been in the virtual space.

[0050] Regarding the display method of the avatar, the server 101 can perform processing such as changing the resolution when displaying the avatar, applying effects such as mosaic and blur, changing the color to a single color, or changing the shader to a simpler shader. For example, when a user who is an artist hosts a live event in a virtual space, the server 101 controls the avatar to be displayed as is on the user terminal 102 of the user who has purchased a ticket. On the other hand, the server 101 controls the user terminal 102 of the user who has not purchased a ticket to display the avatar with a lower resolution, mosaic, or blur.

[0051] Although it is assumed that the setting UI 401 is used before the user makes the avatar participate in the virtual space, the server 101 may enable the setting of the presentation mode (display form) of other avatars while the user's avatar is participating in the virtual space. Figure 5 C describes a specific example of a UI for changing the presentation method to other avatars when the user's avatar is participating in a virtual space.

[0052] Figure 5 A to Figure 5 D is a diagram illustrating a specific example of changing the way an avatar is displayed. Figure 5 A to Figure 5 D illustrates the state of group consultation in virtual space. There are four participants in group consultation, namely, the main counselor, the deputy counselor, the patient A and the patient B. Figure 4 As in the described example, patient A makes settings so that avatar Y with a facial image is shown to other users who are counselors, and avatar Y without a facial image is shown to other users who are patients (other users who are “others”).

[0053] Figure 5 A illustrates the state of the virtual space viewed by the sub-counselor on his / her user terminal 102. On the sub-counselor's user terminal 102, the display 202 shows the main counselor's avatar 501 and the patient A's avatar 502. Because patient A has set their avatar to appear to the counselor with a facial image, the display 202 also shows patient A's facial image 503. Similarly, on the main counselor's user terminal 102, the display 202 displays patient A's facial image 503 along with the sub-counselor's avatar and patient A's avatar 502.

[0054] Figure 5 B illustrates the state of the virtual space viewed by patient B on his / her user terminal 102. Since patient B is in a different position from the sub-counselor in the virtual space, the angle from which he / she views the main counselor and patient A is different from that of the sub-counselor. Since patient B is not in the sub-counselor's field of view, Figure 5As shown in FIG. 1 , the avatar of patient B is not displayed on the display 202 of the user terminal 102 of the sub-counselor. In addition, since there is no sub-counselor in the field of view of patient B, Figure 5 As illustrated in B, the avatar of the sub-counselor is not displayed on the display 202 of the user terminal 102 of patient B.

[0055] Furthermore, since patient A is set to present the avatar to patient B in a state without a facial image, Figure 5 As illustrated in FIG. 1 , the facial image of patient A is not displayed on the display 202 of the user terminal 102 of patient B. That is, the facial image of patient A is not displayed in the virtual space viewed by patient B.

[0056] When displaying the avatar of patient A for each of other users participating in the same virtual space, the server 101 may perform display control to display the avatar of patient A in different display forms on the user terminals 102 of different categories of users based on the settings of patient A.

[0057] When a PC or the like is used as the user terminal 102, Figure 5 The size of the facial image 503 of the patient A illustrated in FIG. 503 may cause the assistant consultant to ignore the detailed changes in the patient's expression. Figure 5 As shown in FIG. 3 , the user terminal 102 can use the facial images of other users (patient A and patient B) as a comparison Figure 5 The size of A is displayed in a large size so that the user in the counselor's position can easily observe the expression. Figure 5 The C example is as follows: Figure 5 The facial image 503 of patient A and the facial image 505 of patient B are combined with the facial image obtained by the screen of A. Figure 5 A is displayed in an enlarged manner. Figure 5 The layout change of the screen of C can be performed by the user terminal 102 based on information such as the user's position, or can be performed by an instruction of the server 101. Note that the layout change of the screen of C can be performed according to an instruction of a user such as a sub-consultant, patient A, or patient B. Figure 5 The layout of the C screen changes.

[0058] Figure 5 D illustrates an example of setting up a UI for how a user presents his / her avatar when participating in a virtual space. For example, when patient B is Figure 5When patient B uses a controller or the like to select the primary counselor's avatar 501 on the screen and gives instructions to set the presentation method for their avatar, the display 202 displays a settings screen 504 in the virtual space. On settings screen 504, patient B can set how their avatar will be presented to the primary counselor. The user terminal 102 receives the user's settings and records or updates the presentation method for patient B's avatar to the primary counselor. In this way, the user can change the presentation method for their avatar even while participating in the virtual space.

[0059] Note that the user terminal 102 can receive settings regarding whether to process the user's own voice, regardless of the presentation method of the avatar when the user is participating in a virtual space. For example, patient B can prevent his / her actual voice from being heard by patient A by selecting the avatar of another patient A and processing patient B's own voice for patient A.

[0060] Figure 6A and Figure 6B is a flowchart illustrating the processing of the communication system 100 according to the first embodiment. Figure 6A The illustrated process uses a method called remote rendering in which the server 101 renders an image displayed on each user terminal 102 . Figure 6B The illustrated processing is processing using a method called local rendering for rendering an image in the user terminal 102. The processing of the communication system 100 according to the first embodiment can be realized by using any method.

[0061] First, the use of Figure 6A Remote rendering processing. Figure 6A The following illustrates the process between the user terminal 102 of the first user (hereinafter referred to as the first user terminal 102) and the server 101. The server 101 also performs a process similar to the process with the first user terminal 102 with the user terminals 102 of users other than the first user participating in the virtual space.

[0062] Steps S601 to S603 are the process of the first user setting the display mode of the avatar to other users. In step S601, the first user terminal 102 receives the instruction from the first user and sets the display mode of the first user's avatar to other users. The setting process of the display mode of the avatar is as shown in FIG. Figure 4 In step S602, the first user terminal 102 sends information related to the display mode of the avatar set in S601 to the server 101. In step S603, the server 101 records the received information related to the display mode of the first user's avatar.

[0063] Steps S604 to S606 are the process of causing the first user to join the virtual space. In step S604, the first user terminal 102 receives an instruction from the first user to join the virtual space. The first user terminal 102 obtains the identification information of the virtual space that the first user has instructed to join from the storage device 212 or the like.

[0064] In step S605, the first user terminal 102 sends the virtual space identification information obtained in step S604 to the server 101 and requests the first user to join the virtual space. In step S606, the server 101 allows the first user to join the virtual space corresponding to the identification information received from the first user terminal 102.

[0065] Steps S607 to S619 are a loop process and are repeated until all users including the first user leave the virtual space. In step S608, the first user terminal 102 acquires various types of information from the first user.

[0066] The user terminal 102 can obtain the following information, for example. The user terminal 102 can use the camera 210 to capture the user's face and obtain information about the user's expression from the captured user's face using expression analysis technology. The user terminal 102 can use the microphone 208 to obtain the voice uttered by the user. In the case where the user terminal 102 is an HMD, or in the case where the user terminal 102 is communicatively connected to the HMD worn by the user, the user terminal 102 can use a gyroscope sensor to detect the movement of the HMD and obtain information related to the movement of the user's head. The user terminal 102 communicates with the controller held by the user via the short-range communication I / F 213 and can obtain information about the avatar operation instructions input by the user using the controller. The user terminal 102 can obtain from the controller information related to the movement of the user's arm detected by the gyroscope sensor built into the controller.

[0067] In step S609, the first user terminal 102 transmits the user information acquired in step S608 to the server 101. The first user terminal 102 selects which of the user information acquired in step S608 to transmit to the server 101 based on the avatar display method configured in step S601. For example, if the first user's avatar and the first user's facial image are displayed on the second user's terminal 102 (hereinafter referred to as the second user terminal 102), the first user terminal 102 transmits the first user's facial image along with the avatar information. In step S610, the server 101 receives the first user's information from the first user terminal 102.

[0068] Steps S611 to S618 are a loop process, repeated for each number of second users who join the virtual space and view videos in the virtual space. In step S612, for example, server 101 obtains information about the second user as follows. Server 101 may refer to the second user's account information and obtain the second user's username, information related to their qualifications, and information about the groups to which the second user belongs.

[0069] Server 101 can obtain the second user's position (such as qualifications and categories of all groups) from an external system. For example, server 101 queries the hospital's electronic medical record system to determine whether the second user is registered as a consultant. By coordinating with the external system to obtain information related to the second user's position, server 101 can reduce the risk of impersonation.

[0070] In addition, the server 101 can classify the second user based on the group information set by the first user, and change the display method of the avatar for each group to which it belongs. The server 101 only needs to be able to set the group set by the first user as the position (category) of other users. For example, the first user who is a patient classifies the second user who is a counselor into the "trusted counselor" or "untrusted counselor" group. The server 101 can control to display the facial image of the first user to users in the "trusted counselor" group, and not to display the facial image of the first user to users in the "untrusted counselor" group. By allowing the first user to set the group and classify the second user, the server 101 can display the avatar of the first user in different ways to users who are objectively in the same position.

[0071] In step S613 , the server 101 determines the avatar of the first user and the display method of the avatar to be displayed to the second user based on the information of the display method of the first user's avatar recorded in step S603 .

[0072] In step S614, server 101 determines a 3D scene in the virtual space to be presented to the second user and generates data for the 3D scene in a data format capable of describing three-dimensional computer graphics, such as X3D. Server 101 generates a 3D model of the first user's avatar in the virtual space based on the presentation method of the first user's avatar determined in step S613.

[0073] In step S615, the server 101 renders the data of the 3D scene of the virtual space generated in step S614 and generates a video viewed from the second user's point of view in a data format such as MP4. In step S616, the server 101 transmits the video generated in step S615 to the second user terminal 102. In step S617, the second user terminal 102 reproduces the video received from the server 101 on the display 202.

[0074] Note that, as referenced Figure 4 As described above, when the purpose of communication is used as a condition for determining the presentation method of the avatar, the server 101 performs processing to obtain the purpose of communication after executing step S612. For example, the server 101 may receive the setting of the purpose of communication input by any user participating in the virtual space on the user terminal 102.

[0075] Furthermore, in the case where the purpose of communication is associated as information related to the virtual space (such as “virtual space used for consultation”), the server 101 can set the purpose of communication based on the information related to the virtual space.

[0076] Furthermore, server 101 can estimate the purpose of communication from users participating in the virtual space. For example, if a user with a counselor account is participating in the virtual space, server 101 can estimate the purpose of the communication to be "consultation." Furthermore, server 101 can analyze the appearance of avatars and, for example, if an avatar is wearing a white coat, estimate the purpose of the communication to be "medical consultation or consultation."

[0077] As reference Figure 4 As described above, when using information related to the virtual space as a condition for determining the avatar display method, server 101 performs processing to obtain information related to the virtual space after executing step S612. Examples of information related to the virtual space include the name of the virtual space registered with communication system 100, a description of the virtual space entered by the administrator of communication system 100 or a user participating in the virtual space, and furniture present in the virtual space. Based on this information related to the virtual space, server 101 can use, for example, AI technology to identify the status of the virtual space and change the avatar display method accordingly.

[0078] As reference Figure 4 As described above, when the number or duration of communications is used as a condition for determining the display mode of the avatar, the server 101 obtains the number and duration of communications between the first user and the second user after executing step S612. To obtain the number and duration of communications, the server 101 records the number and duration of communications between the first user and the second user.

[0079] The number of communications may be, for example, the number of times the first user and the second user have participated in the same virtual space. Alternatively, the number of communications may be the number of times the first user and the second user have participated in the same virtual space for a predetermined purpose. Furthermore, the number of communications may be the number of times the first user and the second user have participated in a specific virtual space together.

[0080] The time of communication may be, for example, when the first user and the second user were engaged in a conversation via their avatars. Furthermore, the time of communication may be when the first user and the second user were already participating in the same virtual space together. Furthermore, the time of communication may be when the first user and the second user were already participating in the same virtual space together for a predetermined purpose. Furthermore, the time of communication may be when the first user and the second user were already participating in a specific virtual space together.

[0081] As reference Figure 4 As described above, when information such as the appearance of another user's avatar is used as a condition for determining the display method of the avatar, server 101 performs processing to obtain information about the second user's avatar after executing step S612. Server 101 can obtain information about the appearance of the second user's avatar by, for example, analyzing the 3D model of the second user's avatar using AI. In addition, server 101 can obtain the name of the second user's avatar or a description added by the second user to the avatar as information related to the second user's avatar, and determine the display method of the first user's avatar based on the obtained information related to the second user's avatar.

[0082] As reference Figure 4 As described above, when information related to the behavior of another user is used as a condition for determining the display method of the avatar, the server 101 performs a process of obtaining information related to the behavior of the second user (the other user) after executing step S612. For example, the server 101 can obtain information related to the behavior of the second user by storing the behavior history of the second user in the storage device 307 or the like. In addition, the server 101 can query an external system to obtain information related to the behavior of the second user.

[0083] Information related to user behavior, for example, includes information about whether the user has purchased a live broadcast ticket. In the example of a live broadcast in a virtual space, server 101 can obtain information about whether the second user has purchased a ticket from the ticketing system. Server 101 can change the presentation of the first user's avatar based on whether the user has performed a predetermined action, such as purchasing a ticket.

[0084] Next, the use of Figure 6B Since the processing of steps S601 to S614 is the same as Figure 6A The processes denoted by the same reference numerals are the same, and thus their description will be omitted.

[0085] After determining the 3D scene of the virtual space to be presented to the second user in step S614, in step S631, the server 101 notifies the second user terminal 102 of the determined 3D scene. The 3D scene is represented in a data format capable of describing three-dimensional computer graphics, such as X3D.

[0086] In step S632, the second user terminal 102 renders the 3D scene of the virtual space notified from the server 101 and generates a video viewed from the second user's viewpoint. In step S617, the second user terminal 102 reproduces the video generated in step S632 on the display 202.

[0087] According to the first embodiment, the first user can Figure 4 The illustrated settings UI 401 sets the display mode of the avatar to change the display mode of the first user's avatar between when the other user is a counselor and when the other user is a patient. Specifically, the server 101 can control the display of not only the first user's avatar but also their facial image in the virtual space viewed by the counselor, but only the first user's avatar in the virtual space viewed by the other patient. Therefore, when the first user's avatar is displayed on the user terminal 102 of a second user participating in the virtual space, the server 101 can display the first user's avatar in an appropriate display mode based on the second user's perspective.

[0088] Note that it is not necessary to execute the Figure 6A and Figure 6B The server 101 can reuse the 3D scene generated in step S614 between the second user who has the same avatar display method as the first user. For example, in step S614, the server 101 can reuse the 3D scene when the second user is a primary consultant in the process when the second user is a secondary consultant. For users who have the same avatar display method as the first user, the server 101 can improve processing efficiency by reusing the 3D scene of the virtual space generated in S614.

[0089] In addition, assuming Figure 4 The setting UI 401 illustrated in FIG2 is displayed on an HMD as the first user terminal 102, but the user terminal 102 is not limited to an HMD and may be a terminal such as a PC, a smartphone, or a tablet. The terminal such as a PC, a smartphone, or a tablet as the user terminal 102 may display the setting UI 401 on the display 202 and receive an input of setting of the presentation method of the avatar from the first user.

[0090] <Second embodiment> In the first embodiment, the communication system 100 is configured as a client-server system. However, the present invention can also be implemented as a serverless system. The communication system according to the second embodiment is configured as a serverless system.

[0091] Figure 7 1 is a diagram illustrating a configuration example of a communication system 700 according to the second embodiment. The communication system 700, which is an example of an information processing system, includes a plurality of user terminals 701 connected point-to-point via a network such as the Internet. Since the user terminals 701 are the same as the user terminals 102 described in the first embodiment, their detailed description will be omitted. In addition, since the hardware configuration of the user terminals 701 is similar to that of the reference 1 embodiment, the hardware configuration of the user terminals 701 is similar to that of the reference 1 embodiment. Figure 2 The hardware structure of the user terminal 102 of the first embodiment described is the same, so its description is omitted. The setting UI for the user to set the display mode of the avatar in the second embodiment is similar to that of the reference embodiment. Figure 4 The setting UI 401 according to the first embodiment is described.

[0092] Figure 8 7 is a flowchart illustrating the processing of the communication system 700 according to the second embodiment. In step S801, the user terminal 701 of the first user (hereinafter, described as the first user terminal 701) receives an instruction from the first user and sets the display form in which the avatar of the first user is displayed to other users. Figure 4 As described, the first user terminal 701 may receive settings from the first user via the setting UI 401. The process of setting the presentation mode of the avatar is performed by the user terminal 701 of the user participating in the virtual space.

[0093] In step S802 , the first user terminal 701 records the settings received from the first user in step S801 in the RAM 204 or the storage device 212 .

[0094] Steps S803 to S810 are processes when the first user participates in the virtual space. Figure 8 Although the processing between the first user terminal 701 and the user terminal 701 of the second user (hereinafter referred to as the second user terminal 701 ) is illustrated, the first user terminal 701 and the user terminal 701 of each user participating in the virtual space perform the same processing.

[0095] In step S803, the first user terminal 701 receives an instruction from the first user to join the virtual space. The first user terminal 102 obtains identification information of the virtual space that the first user has instructed to join from the storage device 212 or the like.

[0096] In step S804, the first user terminal 701 sends the virtual space identification information obtained in step S803 to the second user terminal 701, thereby notifying the first user to join the virtual space specified by the identification information. In step S805, the second user terminal 701 records that the first user has joined the virtual space.

[0097] In step S806, the second user terminal 701 obtains the information of the second user and sends the obtained information of the second user to the first user terminal 701. The method for obtaining the information of the second user is the same as Figure 6A The process is the same as that in step S612 in FIG. In step S807 , the first user terminal 701 receives information related to the second user from the second user terminal 701 .

[0098] In step S808, the first user terminal 701 determines the first user's avatar to be displayed to the second user and the display method of the avatar based on the second user's information received in step S807 and the display method of the first user's avatar recorded in step S802. Figure 6A The processing of step S613 is the same as that of step S614, so its detailed description will be omitted.

[0099] In step S809, based on the result determined in step S808, the first user terminal 701 transmits information regarding the first user's avatar to be displayed to the second user and the manner in which the avatar is displayed to the second user to the second user terminal 701. For example, the first user terminal 701 generates a 3D model of the first user's avatar to be displayed to the second user in a data format capable of describing three-dimensional computer graphics, such as X3D. The first user terminal 701 transmits the generated 3D model of the first user's avatar to the second user terminal 701. Furthermore, if the first user's facial image is to be displayed to the second user along with the avatar, the first user terminal 701 also transmits information for displaying the first user's facial image to the second user terminal 701. In step S810, the second user terminal 701 receives information regarding the first user's avatar to be displayed to the second user and the manner in which the avatar is to be displayed from the first user terminal 701.

[0100] Steps S811 to S820 are loop processing and are repeated until all users including the first user leave the virtual space. In step S812, the first user terminal 701 obtains various types of information from the first user. Examples of the information obtained from the first user are as follows: Figure 6A The information in step S608 in is the same, so its description is omitted.

[0101] The processing from step S813 to step S819 is repeated as many times as the number of second user terminals 701 communicating with the first user terminal 701 in the virtual space. In step S814, the first user terminal 701 sends the information of the first user acquired in step S812 to the second user terminal 701. Figure 6A The processing in step S609 of FIG. 8 is the same, so its description will be omitted. In step S815, the second user terminal 701 receives the information of the first user from the first user terminal 701.

[0102] In step S816, the second user terminal 701 determines a 3D scene in the virtual space to be presented to the second user, and generates data for the 3D scene in a data format capable of describing three-dimensional computer graphics, such as X3D. The second user terminal 701 may generate data for the 3D scene in the virtual space to be presented to the second user by using the first user's avatar and avatar presentation method received in step S810 and the first user's information received in step S815.

[0103] In step S817, the second user terminal 102 renders the data of the 3D scene of the virtual space generated in step S816 and generates a frame of a video viewed from the second user's viewpoint. In step S818, the second user terminal 102 reproduces the video generated in step S817 on the display 202.

[0104] According to the second embodiment described above, referring to Figure 7 The described serverless communication system 700 can control to display the first user's avatar in a display form corresponding to the standpoints of other users based on the setting of the presentation method using the first user's avatar.

[0105] For example, patient A receiving counseling sets the facial image of patient A to be displayed in the virtual space displayed on the counselor's user terminal 701, so that the counselor can check the actual expression of patient A. On the other hand, by setting the facial image of patient A not to be displayed in the virtual space displayed on the user terminal 701 of another patient B, patient A's face cannot be seen by patient B, and thus patient A can protect his privacy.

[0106] When displaying the avatar of patient A on user terminal 701 of another user participating in the virtual space, communication system 700 can display the avatar in an appropriate display format based on the perspective of the other user (psychiatrist, counselor, other patient, etc.). In other words, when displaying the avatar of the first user on user terminal 102 of a second user participating in the virtual space, server 101 can display the avatar of the first user in an appropriate display format based on the perspective of the second user.

[0107] (Other embodiments) Although the present invention has been described in detail based on the preferred embodiments of the present invention, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention are also included in the present invention. Some of the above embodiments can be appropriately combined.

[0108] In addition, the present invention also includes the following situation: the program of the software that realizes the functions of the above-mentioned embodiment is supplied directly from a recording medium or by using wired / wireless communication to a system or device having a computer capable of executing the program, and the program is executed. Therefore, in order to realize the functional processing of the present invention by a computer, the program code itself supplied and installed in the computer can also realize the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In this case, as long as the program has the function of the program, the form of the program is not limited, such as object code, a program executed by an interpreter, and script data supplied to the OS.

[0109] The recording medium used to supply the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory. A program supply method is, for example, a method in which a computer program for implementing the present invention is stored in a server on a computer network, and a client computer connected to the server downloads and executes the computer program.

[0110] The present invention can also be implemented by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. Furthermore, the present invention can also be implemented by a circuit (e.g., an ASIC) for implementing one or more of the functions.

[0111] The present invention is not limited to the above embodiments, and various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention. Therefore, the following claims are added to disclose the scope of the present invention.

[0112] This application claims priority based on Japanese Patent Application No. 2023-041630 filed on March 16, 2023, and all the contents disclosed therein are incorporated by reference. [Explanation of Reference Numerals] 101: Server (information processing device) 306: CPU 701: User terminal 201: CPU

Claims

1. An information processing device, comprising: a first acquiring unit configured to acquire information of a plurality of display forms of an avatar of a first user; a second acquiring unit configured to acquire information of a second user in the virtual space in which the first user participates; as well as A determination unit is configured to determine a display form of the first user's avatar for the second user among a plurality of display forms of the first user's avatar based on the information of the second user.

2. The information processing device according to claim 1 further includes a sending unit, which is configured to generate a video of the virtual space displaying the avatar of the first user in the display form of the avatar of the first user determined by the determination unit, and send the video to the user terminal of the second user.

3. The information processing device according to claim 1 further includes a notification unit, which is configured to determine a scene in the virtual space in which the avatar of the first user is displayed in the display form of the avatar of the first user determined by the determination unit, and notify the user terminal of the second user of the scene. 4 . The information processing apparatus according to claim 1 , further comprising a first setting unit configured to receive, from the first user, a setting of a display form of the first user's avatar with respect to the second user.

5. The information processing apparatus according to claim 4, wherein: The first setting unit receives setting of a display form of the avatar of the first user by selecting the avatar of the second user when the first user participates in the virtual space.

6. The information processing device according to any one of claims 1 to 5, wherein: The plurality of display forms of the first user's avatar include a form displaying a 3D shape acquired from the first user's face.

7. The information processing device according to any one of claims 1 to 6, wherein: The plurality of display forms of the first user's avatar include a form in which a part or all of the first user's facial image is displayed together with the first user's avatar.

8. The information processing device according to any one of claims 1 to 7, wherein: The multiple display forms of the first user's avatar include forms in which the first user's avatar is processed and displayed.

9. The information processing device according to any one of claims 1 to 8, wherein: The plurality of display forms of the first user's avatar include a form in which a facial image of the first user's avatar is enlarged and displayed together with an image of the virtual space. 10 . The information processing apparatus according to claim 1 , further comprising a second setting unit configured to receive, from the first user, a setting regarding whether to process the first user's voice reproduced to the second user. The information processing apparatus according to claim 10 , wherein: The second setting unit receives a setting regarding whether to process the voice of the first user by selecting the avatar of the second user when the first user participates in the virtual space.

12. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar with respect to the second user based on the information of the second user and a purpose of communication between the first user and the second user.

13. The information processing apparatus according to claim 12, wherein: The purpose of the communication is set based on information related to the virtual space or is set by the first user.

14. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar with respect to the second user based on the information of the second user and information related to the virtual space.

15. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar with respect to the second user based on the information of the second user and the number or time of communications between the first user and the second user.

16. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar with respect to the second user based on the information of the second user and the information of the second user's avatar.

17. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar with respect to the second user based on the information of the second user and the behavior of the second user.

18. The information processing apparatus according to any one of claims 1 to 11, wherein: The determination unit determines a display form of the first user's avatar for the second user based on the information of the second user and a stay time of the second user in the virtual space.

19. The information processing apparatus according to any one of claims 1 to 18, wherein: A plurality of display forms of the first user's avatar can be set for each second user or for each category of the second user.

20. An information processing device comprising: a display control unit configured to control display of an avatar of the first user for each user participating in the same virtual space as the first user, Wherein, the display control unit: displaying the avatar of the first user in a first display form on a user terminal of a second user participating in the same virtual space as the first user, and The avatar of the first user is displayed in a second display format different from the first display format on a user terminal of a third user participating in the same virtual space as the first user.

21. An information processing system comprising: The information processing device according to any one of claims 1 to 19; as well as a user terminal of the second user, The user terminal of the second user includes a reproduction unit configured to reproduce a video of the virtual space in which the avatar of the first user is displayed according to the display form of the avatar of the first user determined by the determination unit.

22. An information processing method, comprising: A first acquisition step is used to acquire information of multiple display forms of the first user's avatar; A second obtaining step is used to obtain information of a second user in the virtual space participated by the first user; as well as A determining step for determining a display form of the first user's avatar for the second user among a plurality of display forms of the first user's avatar based on the information of the second user.

23. A program causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 20.

Images