System and method for controlling the same

Abstract

To more appropriately control reflection of user information on an avatar.SOLUTION: A system for achieving communication between a first user and a second user comprises: acquisition means for acquiring real-time information of the first user; and control means for controlling reflection of real-time information of the first user on a first avatar in a display device that is owned by the second user and displays a virtual space including the first avatar of the first user based on a purpose of the communication.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a system for controlling the display of a user's avatar and a method for controlling the system. [Background technology]

[0002] As virtual reality (VR) technology develops and spreads, its use for various purposes (such as distribution, business, and medicine) is being considered. In VR, users generally communicate in a virtual space using an avatar (the user's alter ego in the system).

[0003] On the other hand, there are already technologies that use cameras or wearable devices to obtain a user's physical information (such as the user's facial expression, movements, vital signs, or brain waves). It is therefore conceivable to reflect the user's physical information in the facial expression or movements of an avatar.

[0004] Patent document 1 describes a technology that detects the movements, facial expressions, or five senses of a user who is a distributor, and if it is determined that the detection results satisfy certain conditions, changes the facial expression of an avatar to a specified facial expression and changes the avatar's pose to a specified pose. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-189674 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, when it is determined that the detection result of the movement or the like satisfies a predetermined condition, it may not be appropriate to uniformly change the facial expression and pose of the avatar. For example, in a typical meeting (business talks or negotiations, etc.), it is not necessarily desirable to change the facial expression of the avatar to clearly convey the patient's symptoms such as tics or negative emotions to the other party.

[0007] Therefore, an object of the present invention is to provide a technique for more appropriately controlling the reflection of a user's information in an avatar. [Means for solving the problem]

[0008] One aspect of the present invention is a method for producing a composition comprising the steps of: A system for realizing communication between a first user and a second user, comprising: An acquisition means for acquiring real-time information of the first user; a control means for controlling, on the basis of a purpose of communication, reflection of the real-time information of the first user in the first avatar of the display device, the display device being owned by the second user and displaying a virtual space including the first avatar of the first user; The system is characterized by having:

[0009] One aspect of the present invention is a method for producing a composition comprising the steps of: A method for controlling a system for realizing communication between a first user and a second user, comprising: acquiring real-time information of the first user; a control step of controlling, in a display device owned by the second user and displaying a virtual space including a first avatar of the first user, reflection of the real-time information of the first user in the first avatar based on a purpose of the communication; The present invention relates to a method for controlling a system comprising the steps of: Effect of the Invention

[0010] According to the present invention, it is possible to more appropriately control the reflection of user information in an avatar. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of a communication system according to a first embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a user terminal according to the first embodiment. [Diagram 3] FIG. 2 is a configuration diagram of a server PC according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing a setting UI according to the first embodiment. [Diagram 5] FIG. 1 is a diagram for explaining group counseling according to the first embodiment. [Figure 6A] 4 is a flowchart of a process using remote rendering according to the first embodiment. [Figure 6B] 4 is a flowchart of a process using local rendering according to the first embodiment. [Figure 7] FIG. 11 is a diagram showing a setting UI according to the second embodiment. [Figure 8] FIG. 11 is a diagram showing the appearance of an avatar in a virtual space in accordance with the second embodiment. [Figure 9] 11 is a flowchart of a process according to the second embodiment. [Figure 10] FIG. 11 is a configuration diagram of a communication system according to a third embodiment. [Figure 11] 11 is a flowchart of a process according to the third embodiment. [Figure 12] FIG. 13 is a diagram showing the appearance of an avatar in a virtual space in accordance with embodiment 4. [Figure 13] 13 is a flowchart of a process according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0013] <Embodiment 1> In the first embodiment, a communication system constructed as a client-server system will be described.

[0014] 1 is a diagram showing the overall configuration of a communication system according to embodiment 1. The communication system includes a server PC 101 and a plurality of user terminals 102 (terminals connected to the server PC 101 via a network such as the Internet).

[0015] The user terminal 102 is a display device such as a PC, a smartphone, a tablet, or an HMD (head mounted display). The user terminal 102 may be a controller (control device) capable of controlling these display devices. The following describes a case where the user terminal 102 is an HMD. Note that, in the first embodiment, a case where the HMD can be directly connected to a network is described, but the HMD may be connected to a network via another device (such as a PC or a smartphone).

[0016] 2 shows an example of a hardware configuration diagram of the user terminal 102 in the first embodiment. The user terminal 102 includes a CPU 201, a display 202, a ROM 203, a RAM 204, a network I / F 205, and an internal bus 206. The user terminal 102 also includes a microphone 208, a sensor unit 209, a camera 210, a speaker 211, a storage device 212, a proximity sensor 213, a video camera 214, a video camera 215, a video camera 216, a video camera 217, a video camera 218, a video camera 219, a video camera 220, a video camera 221, a video camera 222, a video camera 223, a video camera 224, a video camera 225, a video camera 226, a video camera 227, a video camera 228, a video It has a distance communication I / F 213.

[0017] The CPU 201 is a control unit that performs overall control of various functions of the user terminal 102 via an internal bus 206 using a program stored in a ROM 203. The results of execution of the program by the CPU 201 are displayed on a display 202 so that the user can visually confirm the results.

[0018] The ROM 203 is a flash memory, etc. The ROM 203 stores various setting information and, as described above, application programs, etc. The RAM 204 functions as a memory and a work area for the CPU 201.

[0019] A network I / F (interface) 205 is a module for connecting to a network. A microphone 208 captures the voice uttered by the user.

[0020] The sensor unit 209 includes one or more sensors. Specifically, the sensor unit 209 includes at least one of a GPS, a gyro sensor, an acceleration sensor, a proximity sensor, and a measurement sensor (a sensor for measuring blood pressure, heart rate, or brain waves). The sensor unit 209 may also be equipped with a sensor for detecting physical information (information about the body; biometric information) for realizing authentication (fingerprint authentication, vein authentication, iris authentication, etc.).

[0021] The camera 210 is a fisheye camera (imaging unit) mounted inside the HMD. The camera 210 can acquire a captured image of the user's face. The captured image is stored in the RAM 204 after distortion of the fisheye lens is removed.

[0022] The speaker 211 reproduces the voices of users participating in the communication system, sound effects, background music, and the like.

[0023] The storage device 212 is a storage medium and is a device for storing various data such as applications.

[0024] The short-range communication I / F 213 is an interface used for communication with a controller held by the user. The user can input gestures to the user terminal 102 by moving the controller held by the user. The user can also give instructions to the user terminal 102 by operating a button or a joystick provided on the controller. The controller may have a sensor that measures the user's heart rate, pulse, sweat, and the like. The short-range communication I / F 213 may also communicate with a wearable device worn by the user to obtain the user's heart rate, pulse, sweat, and the like. The short-range communication I / F 213 may also communicate with a device (such as a camera or a group of sensors) installed in a room where the user is present.

[0025] FIG. 3 shows an example of a hardware configuration diagram of the server PC 101 in the first embodiment.

[0026] 3, the server PC 101 includes a display unit 301, a VRAM 302, a BMU 303, a keyboard 304, a PD 305, a CPU 306, a storage 307, a RAM 308, a ROM 309, and a flexible disk 310. The server PC 101 also includes a microphone 311, a speaker 312, a network I / F 313, and a bus 314.

[0027] The display unit 301 displays, for example, a live view video, an icon, a message, a menu, or other user interface information.

[0028] In the VRAM 302, moving images to be displayed on the display unit 301 are drawn. Data of the moving images generated in the VRAM 302 is transferred to the display unit 301 in accordance with a predetermined rule, and is thereby displayed on the display unit 301.

[0029] BMU (Bit Move Unit) 303, for example, controls data transfer between multiple memories (for example, between VRAM 302 and other memories). BMU (Bit Move Unit) 303 also controls data transfer between memories and each I / O device (for example, network I / F 313).

[0030] The keyboard 304 has various keys for the user to input characters and the like.

[0031] A pointing device (PD) 305 is used, for example, to point to content (such as an icon or a menu) displayed on the display unit 301, or to drag and drop an object.

[0032] The CPU 306 is a control unit that controls each component based on an OS and a program (control program) stored in the storage 307 , the ROM 309 , or the flexible disk 310 .

[0033] The storage 307 is a hard disk drive (HDD) or a solid state drive (SSD). The storage 307 stores each control program, various data to be temporarily stored, and the like.

[0034] The RAM 308 includes a work area for the CPU 306, an area for saving data during error processing, and an area for loading control programs.

[0035] The ROM 309 stores various control programs used in the server PC 101, data to be temporarily stored, and the like.

[0036] The flexible disk 310 stores various control programs and various data (data that needs to be temporarily stored).

[0037] The microphone 311 picks up audio around the server PC 101. The speaker 312 outputs audio included in the video data.

[0038] The network I / F 313 communicates with the user terminal 102 via the network. The bus 314 includes an address bus, a data bus, and a control bus.

[0039] The control program can be provided to the CPU 306 from a storage 307, a ROM 309, or a flexible disk 310, or can be provided from another information processing device via a network I / F 313 over a network.

[0040] Fig. 4 is a diagram showing a setting UI (user interface) 401 of the communication system according to the first embodiment. The setting UI 401 is displayed on the display 202 of the user terminal 102. The user uses the setting UI 401 to set (reflection setting) how physical information (information about the body; biometric information) including his / her symptoms is reflected in his / her avatar and communicated to other users. In Fig. 4, five settings with priorities of 1 to 5 are displayed. In the setting UI 401, UI areas 402 to 404 each indicate a condition, and a UI area 405 indicates processing when the condition is met.

[0041] In the following, the user for whom the reflection setting is performed is called the "first user", and other users (users other than the first user who participate in the same virtual space community as the first user) are called the "other users". Also, the avatar of the first user is called the "avatar in use".

[0042] UI area 402 is a UI area for setting the purpose of communication (eg, counseling or conducting business negotiations).

[0043] The UI area 403 is a UI area for setting the role of the other user (for example, counselor or patient). Note that if nothing is input in the UI area 403, it indicates that the role of the other user may be any role.

[0044] The UI area 404 is a UI area for setting the type of physical information (physical information type) to be reflected in the avatar. The physical information type is, for example, a tic, a smile, or tension. A tic is a quick body movement or vocalization that occurs involuntarily.

[0045] UI area 405 is a UI area for setting the degree to which the physical information indicated by the physical information type is reflected in the avatar (reflection method). When reflecting the user's facial expression, etc. in the avatar, the user can select one of the options "reflect as is," "reflect with emphasis," "reflect with suppression," and "do not reflect."

[0046] For example, assume that group counseling regarding tic disorder is conducted in a virtual space. "Group counseling" is a counseling method in which multiple patients gather together. In this case, multiple users with different roles (positions), such as counselor and patient, participate in a community (group) in the virtual space. In this case, a first user who is a patient may be concerned about his or her tic disorder and may not want other patients to see the symptoms of tic disorder. In this case, the first user sets, for example, as in setting group 406, "When the purpose of communication is to provide counseling, tics are not reflected in the avatar used to be shown to the other user whose role is a patient" (a setting with a priority of 2 is made). Then, the first user sets, "When the purpose of communication is to provide counseling, tics are reflected in the avatar used to be shown to the other user whose role is a counselor" (a setting with a priority of 1 is made).

[0047] In other words, when the first user reflects his / her own symptoms in the used avatar, the first user can set whether or not to show the symptoms (and to what extent the symptoms are to be shown) depending on the role of the other user who will be viewing the used avatar.

[0048] Also, for example, assume that when conducting business negotiations in a virtual space, the first user wishes to emphasize and reflect a friendly expression (such as a smile) in the avatar to be used in order to ease the other party's guard, and conversely, to suppress and reflect a nervous appearance in the avatar to be used. In this case, the first user can set, as in setting group 407, "When the purpose of communication is to conduct business negotiations, the avatar to be used shown to the other user is to emphasize and reflect the first user's smile, and suppress and reflect the first user's nervousness."

[0049] In addition, in the setting UI 401, the type of physical information and the degree of reflection of the physical information in the avatar may be set in advance for each purpose of communication.

[0050] In addition, the communication system preferentially uses the setting with the smaller assigned priority number among the multiple settings input in the setting UI 401. For example, in the example of FIG. 4, the purpose of communication is to provide counseling, and the other user Let us assume that the role of the first user is a counselor and that the communication system controls the reflection of tics in the avatar being used. In this case, settings with priorities of 1 and 5 can be used, but the communication system uses the setting with the lower priority (i.e., the setting with priority of 1). Therefore, the communication system controls the tic symptoms of the first user to be reflected "as is" in the avatar being used.

[0051] Furthermore, the communication system may reflect not only visible information such as the facial expression or movement of the first user, but also other information in the avatar used. For example, when remote medical treatment or counseling is performed in a virtual space, the user terminal 102 acquires the patient's body temperature or heart rate with the sensor unit 209, and acquires eye movements with the camera 210. The user terminal 102 may estimate the degree of tension or calmness of the first user based on the acquired information, and reflect the estimation result in the avatar.

[0052] The above setting UI is intended to be used before the first user joins a community (group) in the virtual space, but a UI for setting while the first user is viewing the virtual space may also exist. Such a UI will be described later with reference to FIG. 5C.

[0053] 5A to 5C are diagrams showing the state of group counseling in a virtual space. Here, it is assumed that Patient A has set that "when the purpose of communication is to provide counseling, tics are not reflected in the avatar to be used that is shown to the other user who plays the role of a patient." It is also assumed that Patient A has set that "when the purpose of communication is to provide counseling, tics are reflected in the avatar to be used that is shown to the other user who plays the role of a counselor." It is also assumed that four people, a main counselor, a sub-counselor, Patient A, and Patient B, are participating in the group counseling. It is to be noted that avatar 501 is the avatar of the main counselor, and avatar 502 is the avatar of Patient A.

[0054] Fig. 5A shows the virtual space displayed on the sub-counselor's user terminal 102, and Fig. 5B shows the virtual space displayed on the patient B's user terminal 102. The virtual space displayed on the sub-counselor's user terminal 102 represents the space (field of view) seen from the sub-counselor's avatar. The virtual space displayed on the patient B's user terminal 102 represents the space seen from the patient B's avatar.

[0055] Here, the sub-counselor's avatar and Patient B's avatar are located at different locations in the virtual space. Therefore, the range of the avatars (main counselor's avatar 501 and Patient A's avatar 502) displayed on the sub-counselor's user terminal 102 and on Patient B's user terminal 102 is different. Note that the display on the sub-counselor's user terminal 102 does not include Patient B's avatar. The display on Patient B's user terminal 102 does not include the sub-counselor's avatar.

[0056] Here, assume that patient A develops a motor tic that causes him to grimaces during counseling. Since information about this motor tic is necessary for the counselor, as shown in Fig. 5A, in the virtual space seen by the sub-counselor, patient A's avatar 502 is frowning (patient A's motor tic is reflected in patient A's avatar 502). Similarly, patient A's avatar 502 is also frowning in the virtual space seen by the main counselor.

[0057] On the other hand, as shown in Fig. 5B, in the virtual space seen by patient B, the avatar 502 of patient A does not change its facial expression. This is because, as shown in the setting group 406, patient A has set "tics are not reflected in the avatar shown to other users whose role is patient."

[0058] FIG. 5C is another example of a UI for a first user to set "how the avatar to be used is to be displayed to the other user in the virtual space." For example, patient B uses a controller or the like to select avatar 501 of the main counselor, and then issues an instruction to set how his / her avatar is to be displayed. Then, a setting screen 503 is displayed in the virtual space. Patient B uses this setting screen 503 to set how his / her avatar is to be displayed to the main counselor. This allows the first user to easily change the setting even when the virtual space is displayed.

[0059] The flowcharts in Figures 6A and 6B show the processing of the communication system according to the first embodiment. The flowchart in Figure 6A shows processing using a method in which the server PC 101 renders the video to be displayed on each user terminal 102 (a method called remote rendering). On the other hand, Figure 6B shows processing using a method in which the user terminal 102 renders the video (a method called local rendering). The communication system according to the first embodiment is capable of executing either of these two types of processing.

[0060] A process using remote rendering will be described with reference to the flowchart in FIG. 6A.

[0061] Steps S601 to S603 are processing in which the first user sets the reflection of his / her own physical information in the avatar to be used. This processing is executed between the user terminals 102 of all users participating in the community in the virtual space and the server PC 101. In the following, the user terminal 102 of the first user will be referred to as the "user terminal 102A", and each component of the user terminal 102A will be suffixed with "A". For example, the display 202 of the user terminal 102A will be referred to as the "display 202A", and the CPU 201 of the user terminal 102A will be referred to as the "CPU 201A".

[0062] In step S601, CPU 201 (CPU 201A) of user terminal 102A accepts a reflection setting (a setting as to how the first user's physical information is reflected in the avatar to be used and communicated to the other user) from the first user. Specifically, CPU 201A acquires the setting (setting that associates the purpose of communication, the role of the other user, the type of physical information, and the reflection degree) input by the user in the setting UI shown in Fig. 4 as the reflection setting.

[0063] In step S 602 , the CPU 201 A transmits the accepted reflection setting to the server PC 101 .

[0064] In step S603, the CPU 306 of the server PC 101 records (stores) the received reflection setting in the storage 307 or the like.

[0065] The following steps S604 to S606 are a process in which the first user participates in the community in the virtual space. Although omitted in Fig. 6A, this process is executed between the user terminals 102 of all users who will participate in the community in the virtual space and the server PC 101.

[0066] In step S604, the CPU 201A receives an instruction from the first user to participate in a community of a virtual space. At this time, the CPU 201A acquires identification information of the virtual space of the community in which the first user wishes to participate (hereinafter, referred to as a "desired space").

[0067] In step S605, CPU 201A transmits identification information of the desired space to server PC 101, and requests participation in the community of the desired space.

[0068] In step S606, the CPU 306 allows the first user to participate in the community of the desired space corresponding to the obtained identification information.

[0069] The following steps S607 to S616 are a process (loop process) that is repeatedly executed until all users including the first user have left the community of the desired space.

[0070] In step S607, CPU 201A acquires real-time (current) physical information of the first user. CPU 201A acquires at least one of the physical information, for example, voice, emotion, facial expression, blood pressure, heart rate, stress level, body temperature, amount of sweat, brain waves, pulse, posture, and movement (including eye movement). CPU 201A may acquire, for example, a captured image of the first user as the physical information.

[0071] For example, CPU 201A controls microphone 208A to acquire the voice uttered by the first user. In this case, CPU 201A may further estimate the emotion of the first user from the acquired voice using an existing voice emotion analysis technique.

[0072] CPU 201A may control camera 210A to acquire a captured image of the user's face. In this case, CPU 201A may analyze (acquire) the facial expression of the first user based on the captured image using facial expression analysis technology. CPU 201A may also analyze the eye movement of the first user based on the captured image to estimate the psychology of the first user. CPU 201A may also estimate the blood pressure, heart rate, and / or stress level of the first user using vital data analysis technology.

[0073] Furthermore, the CPU 201A may control the sensor unit 209A to measure at least one of the blood pressure, heart rate, body temperature, amount of sweat, and brain waves of the first user.

[0074] CPU 201A may communicate with a controller held by the first user or a wearable device worn by the first user using short-range communication I / F 213. CPU 201A may acquire information acquired by the controller or the wearable device (any of the first user's heart rate, body temperature, amount of sweat, brain waves, etc.). CPU 201A may communicate with a camera installed in a room or the like using short-range communication I / F 213 to acquire an image of the first user captured by the camera. CPU 201A may then acquire posture information or movement information of the first user based on the captured image.

[0075] The CPU 201A may use the short-range communication I / F 213 to connect to a group of sensors installed in a room where the first user is located, and acquire vital information of the user.

[0076] In step S608, CPU 201A transmits to server PC 101 the physical information of the first user acquired in step S607.

[0077] In step S607, if CPU 201A determines that the first user with a tic disorder has grimaced, CPU 201A may determine that the first user has exhibited motor tics. CPU 201A may also determine whether the first user with a dizziness disorder has exhibited dizziness based on the eye movement of the first user. That is, CPU 201A may determine whether the first user has exhibited symptoms by referring to information indicating the disease the first user has (disease information). CPU 201A may then control the reflection of physical information such as motor tics or dizziness to the used avatar based on the determination result of whether the first user has exhibited symptoms.

[0078] For example, when the CPU 201A determines that the disease indicated by the disease information of the first user has appeared in the first user, the CPU 201A may transmit physical information related to the symptom of the first user (for example, information of a grimacing motion) in step S608. On the other hand, when the CPU 201A determines that the disease indicated by the disease information of the first user has not appeared in the first user, the CPU 201A may not transmit physical information related to the symptom of the first user in step S608. According to this, when the CPU 201A determines that the symptom indicated by the disease information of the first user has appeared in the first user, the physical information related to the symptom can be reflected in the avatar used in step S612 described later. On the other hand, when the CPU 201A determines that the symptom indicated by the disease information of the first user has not appeared in the first user, the physical information related to the symptom can be not reflected in the avatar used in step S612 described later. In other words, even if the same facial expression, a frown, appears, it is possible to change whether or not to reflect the facial expression in the avatar used and convey it to the other user depending on the disease the first user has.

[0079] In step S609, the CPU 306 of the server PC 101 receives the physical information of the first user.

[0080] In step S610, the CPU 306 acquires (determines) the purpose of the communication to be performed in the desired space.

[0081] For example, the CPU 306 acquires the purpose of communication that is input by a given user (any user participating in the community of the desired space) using a UI displayed on the user terminal 102.

[0082] Furthermore, if the purpose of communication is linked to the information constituting the desired space (for example, if information constituting the desired space is linked to information such as "virtual space for counseling"), CPU 306 may determine the purpose from that information. CPU 306 may estimate (determine) the purpose of communication based on information on the account of at least one of multiple users participating in the community of the desired space. For example, if a user with a counselor account participates in the community of the desired space, CPU 306 may estimate that the purpose of communication is "to provide counseling."

[0083] The CPU 306 may analyze the appearance of each avatar in the desired space to estimate the purpose of the communication. For example, if an avatar wearing a white coat is present in the desired space, the CPU 306 may estimate that the purpose of the communication is "to provide medical examination or counseling."

[0084] The subsequent processing of steps S611 to S615 is a loop processing, which is repeated for each user (users other than the first user) who joins the community and views the video of the desired space (virtual space). One of the users who views the video of the desired space is hereinafter referred to as the "second user." Hereinafter, the user terminal 102 of the second user is referred to as the "user terminal 102B," and each component of the user terminal 102B is suffixed with "B." For example, the display 202 of the user terminal 102B is referred to as the "display 202B," and the CPU 201 of the user terminal 102B is referred to as the "CPU 201B."

[0085] In step S611, the CPU 306 determines (confirms) the role (position) of the second user.

[0086] The CPU 306 may, for example, determine whether the second user's account is valid based on information about the second user's account. For example, when the purpose of communication is to provide counseling, the CPU 306 determines whether the role of the second user is a counselor based on the account information of each user managed by the communication system.

[0087] The CPU 306 may determine the role of the second user based on information acquired from an external system. For example, the CPU 306 queries an electronic medical record system in a hospital, and determines whether the role of the second user is a counselor (registered as a counselor) based on the query result.

[0088] For example, if the first user can set the role of the other user who will participate in the community of the desired space, the CPU 306 may refer to the setting information to determine the role of the second user. With this method, for example, the first user who is a patient may classify the second user who is a counselor into either a "trusted counselor" or an "untrusted counselor." In this case, the CPU 306 may perform control so that the tic disorder of the first user is reflected in the avatar used by the "trusted counselor" and so that the tic disorder of the first user is not reflected in the avatar used by the "untrusted counselor."

[0089] In step S612, the CPU 306 controls the usage avatar of the first user based on the physical information of the first user, the purpose of communication, and the role of the second user.

[0090] The process of step S612 will be described taking as an example a case where the first user has produced a motor tic (a grimacing motor tic).

[0091] For example, assume that in step S607, CPU 201A controls camera 210 to acquire a captured image of the face of the first user. In this case, in step S607, CPU 201A performs facial expression analysis from the captured image to acquire information on whether the first user has grimaced. Furthermore, CPU 201A acquires information indicating that motor tics have appeared as physical information of the first user because the user has a tic disorder. Then, in step S608, CPU 201A transmits the physical information of the first user to server PC 101 via network I / F 205.

[0092] Then, in step S612, CPU306 refers to the reflection setting of the first user recorded in step S603. Here, as shown in FIG. 4, it is assumed that the first user has set "in counseling, tics are reflected in the avatar seen by the user who is a counselor, but tics are not reflected in the avatar seen by the user who is a patient." Also, information indicating that motor tics have appeared has been acquired as physical information of the first user. Therefore, CPU306 controls the avatar of the first user to frown in accordance with the reflection setting when the purpose of communication is to provide counseling and the role of the second user is a counselor. On the other hand, CPU306 controls the avatar of the first user not to frown in cases where the purpose of communication is not to provide counseling or when the role of the second user is a patient.

[0093] In step S613, CPU 306 generates a 3D scene of the desired space including the avatar of the first user controlled in step S612. CPU 306 generates the 3D scene in, for example, a data format (such as X3D) capable of describing three-dimensional computer graphics.

[0094] In step S614, CPU 306 renders a 3D scene of the desired space to generate an image of the desired space as seen from the second user's avatar (the avatar's viewpoint). The CPU 306 generates video in a data format such as MP4.

[0095] In step S615, CPU 306 transmits the video generated in step S614 to user terminal 102B.

[0096] In step S616, the CPU 201B receives the video and displays it on the display 202B.

[0097] Next, the process using local rendering will be described with reference to the flowchart in Fig. 6B. When the process using local rendering is performed (when the process shown in Fig. 6B is performed), steps S614 and S615 shown in Fig. 6A are replaced with steps S631 and S632. On the other hand, the processes of other steps (steps S601 to S613, S616) when using local rendering are performed in the same way as when using remote rendering (when shown in Fig. 6A). Therefore, only steps S631 and S632 will be described below.

[0098] In step S631, the CPU 306 transmits the 3D scene generated in step S613 to the user terminal 102B.

[0099] In step S632, CPU 201B renders the received 3D scene of the desired space, and then generates a frame of an image of the desired space as seen from the second user's avatar.

[0100] As described above, by operating the communication system according to the reflection settings described in Fig. 4, it is possible to realize, for example, the avatar of Patient A frowning only in the virtual space seen by the counselor. In other words, it becomes possible to more appropriately control the avatar to be used based on the role of the other user and the purpose of communication.

[0101] In the above, for simplicity of explanation, the process of generating the 3D scene in the virtual space in steps S612 to S613 is executed in each case where the second user is the main counselor and the sub-counselor. However, in reality, the process can be made more efficient by reusing the 3D scene in the virtual space generated in steps S611 to S612 for multiple other users with the same role.

[0102] <Embodiment 2> In the first embodiment, an example was shown in which a first user sets the reflection degree of physical information in the avatar to be used by performing a reflection setting that associates the purpose of communication with the reflection degree of physical information. However, depending on the purpose of communication, a user with a specific role (position) may wish to determine the reflection degree of physical information of other users. For example, in a group counseling session in which a patient with a severe tic disorder and a patient with a mild tic disorder are participating, the counselor may want to make the symptoms of the patient with the mild tic disorder more noticeable. Therefore, in the second embodiment, a communication system in which a certain user can set the reflection degree of physical information in the avatar of another user will be described.

[0103] As in the first embodiment, it is assumed that four people, a main counselor, a sub-counselor, patient A, and patient B, participate in group counseling (community) in a virtual space. It is further assumed that patient A is a severely ill patient with a large motor tic reaction, and patient B is a mildly ill patient with a small motor tic reaction. In the second embodiment, the communication system (CPU 306) increases the size of the user's avatar if the user's motor tic reaction is large (according to the magnitude of the motor tic reaction).

[0104] 7A to 7C are diagrams showing an example of a setting UI according to embodiment 2. The setting UI 701 is displayed on the counselor's user terminal 102. The counselor uses the setting UI 701 to set (performs reflection setting) how the patient's physical information (including the patient's symptoms) is to be displayed on his / her own user terminal 102.

[0105] UI area 702 is an area that displays types of physical information. In Figures 7A to 7C, motor tics and vocal tics are selected as types of physical information.

[0106] UI area 703 is a UI area for setting the degree to which the physical information is reflected in the avatar. In Figures 7A to 7C, a slide bar 704 is displayed in UI area 703 as an example.

[0107] Slide bar 704 is a UI area for setting whether to "suppress" or "emphasize" the user's tic reaction when reflecting the user's tic reaction in the avatar. Moving pointer 706 on slide bar 704 to the left "suppresses" the user's tic reaction. Moving pointer 706 to the right "emphasizes" the user's tic reaction.

[0108] Figure 7A shows an example where the degree to which Patient A's motor tic reactions are reflected in the avatar is set to neither "suppressed" nor "emphasized." Figure 7B shows an example where the degree to which Patient A's motor tic reactions are reflected in the avatar is set to "suppressed." Figure 7C shows an example where the degree to which Patient B's motor tic reactions are reflected in the avatar is set to "emphasized."

[0109] Instead of the counselor setting the degree of reflection using the setting UI 701 as described above, the CPU 306 may automatically set the degree of reflection according to the disease information of the patient. For example, the milder the symptoms indicated by the disease information of the patient, the greater the degree to which the physical information related to the disease information is reflected in the avatar of the patient. This reduces the effort required of the user to set the degree of reflection.

[0110] 8A to 8C are diagrams showing the appearance of an avatar of patient A and an avatar of patient B displayed on the main counselor's user terminal 102. Avatar 801 is the avatar of patient A, and avatar 802 is the avatar of patient B.

[0111] FIG. 8A shows the appearance of two avatars displayed when neither Patient A nor Patient B is experiencing motor tics.

[0112] Figure 8B shows the appearance of the two avatars when motor tics occur simultaneously in Patient A and Patient B, when the degree to which the reactions of Patient A and Patient B are reflected in the avatars is set to neither "suppress" nor "emphasize," as in Figure 7A. In this case, Figure 8B shows that Patient A's motor tic reaction is large, while Patient B's motor tic reaction is small.

[0113] Figure 8C shows the two avatars with the degree of reflection of the avatar's reactions set to "suppressed" in the UI for patient A in Figure 7B, and with the degree of reflection of the avatar's reactions set to "emphasis" in the UI for patient B in Figure 7C. Compared to the case of Figure 8B, the two avatars are displayed with the size of patient A's avatar made smaller and the size of patient B's avatar made larger. This makes it easier to see the occurrence of motor tics in patient A and patient B.

[0114] Next, referring to the flowchart in FIG. 9, the process according to the second embodiment will be described using FIG. In the flowchart of Fig. 9, steps S601 and S602 in the flowchart of Fig. 6A are replaced with steps S901 and S902, and step S603 and onwards are the same as in the flowchart of Fig. 6A. Therefore, only steps S901 and S902 will be described.

[0115] In step S901, CPU 201B of user terminal 102B accepts a reflection setting indicating the degree to which the first user's physical information (including symptoms) is reflected in the first user's avatar. Here, when the second user uses the UI shown in Fig. 7A to set the degree to which the first user's physical information is reflected in the first user's avatar to "emphasis" or "suppression," CPU 201B accepts the setting as the reflection setting.

[0116] In step S 902 , the CPU 201 B transmits the reflection setting information to the server PC 101 .

[0117] In this way, a user who actually sees an avatar can set the degree to which physical information of other users is reflected in the avatar. In addition, when reflecting the movements or voices of other users in the avatar, it is also possible to set the degree to which each movement or voice is emphasized or suppressed for each user.

[0118] <Embodiment 3> In the first and second embodiments, a communication system has been described that is a client-server system in which a server PC 101 and a plurality of user terminals 102 are connected. However, the communication system can also be realized by a system that does not have a server PC 101. Therefore, in the third embodiment, a case will be described in which the communication system described in the first embodiment is constructed by a system that does not involve the server PC 101. It should be noted that the communication system described in the second embodiment can also be realized by a system that does not involve the server PC 101.

[0119] Fig. 10 is a system configuration diagram of a communication system according to the third embodiment. The communication system has a plurality of user terminals 102 connected by P2P (Peer to Peer) via a network such as the Internet. Each of the plurality of user terminals 102 shown in Fig. 10 has the same configuration as the user terminal 102 according to the first embodiment, and therefore detailed description will be omitted. Also, the setting UI according to the third embodiment is assumed to be the same as the setting UI 401 described in Fig. 4 of the first embodiment.

[0120] FIG. 11 is a flowchart showing the process of the communication system according to the third embodiment.

[0121] In step S1101, CPU 201A of user terminal 102A of the first user accepts the reflection setting from the first user, similarly to step S401.

[0122] The following steps S1102 to S1106 are processes in which the first user participates in the community of the desired space (virtual space). Although omitted in Fig. 11, these processes are executed between the user terminals 102 of all other users who participate in the community of the desired space.

[0123] In step S1102, similarly to step S604, CPU 201A accepts an instruction to participate in the community of the desired space from the first user, and acquires identification information of the desired space.

[0124] In step S1103, the CPU 201A executes the process of the user terminal 1 in the same manner as in step S605. 02B (the other user's user terminal 102), CPU 201A thereby notifies the other user that the first user will participate in the community of the desired space.

[0125] In step S1104, the CPU 201B records that the first user has joined the community of the desired space.

[0126] In step S1105, CPU 201B transmits information about the second user to user terminal 102A. The information about the second user includes information about the role of the second user. The role of the second user can be acquired in the same manner as in step S611.

[0127] In step S1106, CPU 201A receives information about the second user.

[0128] In step S1107, CPU 201A acquires the purpose of communication to be performed in the desired space, similarly to step S610.

[0129] The subsequent processes of steps S1110 to S1117 are loop processes that are repeated until all users, including the first user, have left the virtual space.

[0130] In step S1110, CPU 201A acquires real-time (current) physical information of the first user, similarly to step S607.

[0131] The process of steps S1112 to S1113 is repeated the number of times corresponding to the number of second users other than the first users who participate in the community of the desired space (the number of user terminals 102B that communicate with user terminal 102A).

[0132] In step S1112, CPU 201A controls the avatar used by the first user based on the physical information of the first user, the purpose of communication, and the information of the second user, similarly to step S612.

[0133] In step S1113, CPU 201A generates a 3D model of the avatar used under control in step S1112, and transmits the generated 3D model to user terminal 102B.

[0134] In step S1114, the CPU 201B receives a 3D model of the first user's avatar.

[0135] In step S1115, the CPU 201B generates a 3D scene of the desired space including the avatar of the first user. The CPU 201B generates the 3D scene in a data format capable of describing three-dimensional computer graphics, such as X3D.

[0136] In step S1116, CPU 201B renders the 3D scene of the desired space generated in step S1115 to generate a frame of an image of the desired space seen from the viewpoint of the second user.

[0137] In step S1117, CPU 201B displays the generated image of the desired space on display 202B.

[0138] As described above, according to the third embodiment, even in a communication system that does not include the server PC 101 (server), it is possible to more appropriately control the reflection of physical information in an avatar. become.

[0139] <Embodiment 4> In the fourth embodiment, the communication system changes (updates) information required for controlling an avatar according to the virtual space of the community in which the user has joined.

[0140] In the fourth embodiment, the communication system has the configuration described with reference to Figs. 1 to 3, as in the first embodiment. In the fourth embodiment, the communication system estimates the user's emotion based on the acquired physical information, and reflects the estimated emotion in the facial expression of the user's avatar. Note that the "physical information" in the fourth embodiment is information other than emotion.

[0141] 12A and 12B are diagrams showing the avatars of participants participating in a communication system. As an example, assume that multiple participants are conducting business negotiations using a virtual space. Business partners A and B, and user C, who is the presenter, are participating in this business negotiation. The scope of the virtual space displayed on the user terminals 102 of business partners A and B, and user C, are different from each other. FIG. 12A and 12B show, for example, the virtual space displayed on the user terminal 102 of business partner B. Here, avatar 1201 is the avatar of business partner A, and avatar 1202 is the avatar of user C.

[0142] Here, Figs. 12A and 12B show an example in which a presenter, user C, is estimated to be anxious and nervous through emotion estimation based on physical information.

[0143] 12A is a diagram for explaining the display of user terminal 102 in the case where the change process of physical information according to embodiment 4 is not performed (the acquired physical information is used for emotion estimation as is). In the virtual space displayed on user terminal 102 of business negotiation partner B, as shown in FIG. 12A, since user C is feeling anxious and tense in the real space, the emotions are also reflected in the facial expression of avatar 1202.

[0144] 12B is a diagram for explaining the display of the user terminal 102 when the physical information change process according to the fourth embodiment is performed (when the acquired physical information is changed and then the changed physical information is used to estimate the emotion). In the virtual space displayed on the user terminal 102 of the business negotiation partner B, the facial expression of the avatar 1202 of the user C does not reflect (is suppressed by) the anxiety and tension of the user C.

[0145] Fig. 13 is a flowchart showing the processing of the communication system according to the embodiment 4. The processing of the flowchart in Fig. 13 is processing using the remote rendering method, similar to Fig. 6A. Note that the communication system according to the embodiment 4 can also be realized by using the local rendering method described in Fig. 6B.

[0146] The processing of the flowchart in FIG. 13 is realized by the CPU 201 controlling each unit of the user terminal 102 in accordance with a program stored in the ROM 203.

[0147] Steps S1301 to S1303 are a process in which a first user participates in a community in a virtual space. This process is executed between the server PC 101 and the user terminals 102 of all users participating in this virtual space.

[0148] In step S1301, the CPU 201A receives an instruction to participate in a virtual space from a first user. At this time, the CPU 201A acquires identification information of the virtual space in which the first user wants to participate (desired space).

[0149] In step S1302, CPU 201A transmits identification information of the desired space to server PC 101, thereby requesting server PC 101 to participate in the community of the virtual space.

[0150] In step S1303, the CPU 306 of the server PC 101 allows the first user to participate in the community of the desired space corresponding to the identification information.

[0151] The subsequent processing of steps S1305 to S1315 is a loop processing, which is repeatedly executed until all users including the first user have withdrawn from the community in the virtual space.

[0152] In step S1305, CPU 306 transmits account information of all users participating in the community of the desired space to user terminal 102A.

[0153] In step S1306, CPU 201A determines the purpose of the communication. For example, CPU 201A determines the purpose of the communication based on information on the tool used by the first user in the desired space. For example, if the first user is using the presenter tool, CPU 201A can determine that the purpose of the communication is to hold a meeting (presentation).

[0154] Furthermore, in step S1306, CPU 201A further acquires (determines) information related to the communication (related information). Note that related information may be understood to be included in the purpose of the communication.

[0155] (1) For example, CPU 201A determines the role (position) of a first user based on information about a tool used by the first user in a desired space. If the first user uses a presenter tool, CPU 201A can determine that the first user is a presenter.

[0156] (2) For example, when the purpose of communication is to hold a conference, CPU 201A determines the type of conference (relationship between multiple users participating in the conference) based on the accounts of users participating in the community of the desired space. CPU 201A determines, for example, whether the type of conference to be held is a conference with business partners, an internal conference, or a conference with friends. To determine the type of conference, identification information of a user account registered in user terminal 102 or server PC 101 (virtual space) is used. In step S1308 described later, CPU 201A changes the physical information of the first user acquired in step S1307 described later according to the type of conference (relationship between multiple users participating in the conference).

[0157] (3) For example, CPU 201A determines the nationality of a participant based on the account of the user participating in the community of the desired space. For this determination, identification information of the user account registered in user terminal 102 or server PC 101 (virtual space) is used. In step S1308 described later, CPU 201A changes the physical information of the first user acquired in step S1307 according to the nationality of the participant.

[0158] (4) For example, the CPU 201A determines the content of the event based on the information of the event held in the desired space (within the platform of the desired space). If the content of the event is a speech meeting or a business negotiation event, the CPU 201A changes the physical information of the first user acquired in S1037 in step S1308 described later.

[0159] In step S1307, CPU 201A acquires physical information of the first user.

[0160] Furthermore, CPU 201A determines emotion information of the first user based on the acquired physical information (emotion determination). For example, the correspondence between the output result of the physical information and emotions is verified in advance, and a table showing the correspondence is stored in ROM 203 or server PC 101. CPU 201A determines emotion information of the first user by determining whether the acquired physical information matches a specific emotion pattern described in the table.

[0161] In step S1308, CPU 201A controls the physical information of the first user acquired in step S1307, based on the purpose of communication and related information determined in step S1306.

[0162] (1) For example, assume that CPU 201A determines that the purpose of communication is to hold a conference and that the first user is a presenter (the purpose of communication is to hold a conference with the first user as a presenter). In this case, CPU 201A changes the physical information of the first user so that "facial expressions or gestures showing nervousness in emotion determination are suppressed." For example, when CPU 201A has acquired blood pressure or heart rate information as the physical information, it reduces the blood pressure or heart rate by a predetermined value.

[0163] (2) For example, assume that the CPU 201A determines that the purpose of communication is to hold a conference and that the type of the conference is a conference with a business partner (the purpose of communication is to hold a conference with a business partner). In this case, the CPU 201A changes the physical information of the first user so that "facial expressions or gestures showing nervousness in emotion determination are suppressed."

[0164] (3) For example, assume that the CPU 201A determines that the purpose of communication is to hold a meeting and that the type of the meeting is a meeting with friends (the purpose of communication is to hold a meeting with friends). In this case, the CPU 201A does not change the physical information of the first user.

[0165] (4) For example, assume that CPU 201A determines that the purpose of communication is to hold a conference and that users of multiple nationalities will participate in the conference (the purpose of communication is to hold a conference in which users of multiple nationalities will participate). In this case, CPU 201A changes the physical information of the first user so that "in the emotion determination, tense facial expressions or gestures are suppressed and smiles are emphasized." For example, when CPU 201A has acquired a stress level as the physical information, CPU 201A reduces the stress level by a predetermined value so that smiles are emphasized in the emotion determination. Alternatively, when CPU 201A has acquired information on a movement as the physical information, CPU 201A emphasizes information on the movement of opening the mouth so that smiles are emphasized in the emotion determination.

[0166] (5) For example, assume that CPU 201A determines that the purpose of communication is to hold a conference and the content of the conference is to hold a lecture or a business meeting (the purpose of communication is to hold a lecture or a business meeting). In this case, CPU 201A changes the physical information of the first user so that "facial expressions or gestures showing nervousness in emotion determination are suppressed."

[0167] (6) For example, assume that CPU 201A determines that the purpose of communication is to play a game and that the type of game is a game that requires a poker face (the purpose of communication is to play a specific game). In this case, CPU 201A changes the physical information of the first user so that "tense facial expressions or gestures are suppressed in emotion determination."

[0168] In step S1309, the CPU 201A determines the emotion of the first user based on the physical information of the first user controlled in step S1308 (performs emotion determination again). The CPU 201A transmits the information on the emotion of the first user to the server PC 101.

[0169] In step S1310, the CPU 306 receives information on the emotion of the first user.

[0170] The subsequent processing of steps S1311 to S1314 is a loop processing, which is repeated the same number of times as the number of second users who participate in the desired space and view the video of the desired space.

[0171] In step S1311, CPU 306 controls the avatar used by the first user based on the information on the emotion of the first user received in step S1310. For example, CPU 306 controls the facial expression or movement (gestures) of the avatar used to reflect the emotion of the first user.

[0172] Steps S1312 to S1315 are similar to steps S613 to S616 in the first embodiment, and therefore a detailed description thereof will be omitted.

[0173] In this way, according to the fourth embodiment, the facial expression of the avatar can be adjusted to match the purpose of communication.

[0174] In step S1308, CPU 201A may issue a warning (notification) to the first user if the physical information acquired in step S1307 satisfies a specific condition. For example, CPU 201A may issue a warning (notification) to the first user if it determines that a negative emotion value (e.g., emotion value such as anxiety) indicated by an emotion corresponding to the physical information exceeds a threshold. Alternatively, CPU 201A may issue a warning (notification) to the first user if it determines that a value indicated by the physical information acquired in step S1307 (e.g., blood pressure, heart rate, or body temperature) exceeds a threshold. In this case, CPU 201A issues a warning to the first user before transmitting the physical information to server PC 101 in step S1309, for example, to inquire of "whether the physical information may be transmitted" or "whether the physical information may be changed". CPU 201A may display a display item indicating a warning on display 202, or may output a sound indicating a warning from speaker 312.

[0175] Here, it is determined whether or not to issue a warning based on the physical information before it is changed (controlled) in step S1308, but a warning may be issued based on the physical information after it is changed in step S1308.

[0176] In step S1308, user terminal 102A changes the user's physical information so that emotion determination can be performed based on the purpose of communication, etc., but emotion information may be changed directly based on the purpose of communication and related information.

[0177] In the fourth embodiment, the user terminal 102A sends emotion information determined based on the user's physical information to the server PC 101. However, the server PC 101 may determine the user's emotion based on the user's physical information transmitted by the user terminal 102A and control the facial expression of the avatar.

[0178] As described above, according to the fourth embodiment, emotion information is controlled based on the purpose of communication in the virtual space. As a result, for example, as shown in FIG. 12B, the avatar 1202 of user C in the virtual space seen by business negotiation partner B does not express anxiety or tension, but instead shows a calm expression. In other words, the communication system is able to more appropriately express emotion in business negotiations. It is possible to display an avatar with a different facial expression.

[0179] Also, in the above, "If A is equal to or greater than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2." Conversely, "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2." For this reason, unless a contradiction occurs, the expression "equal to or greater than A" may be read as "A or greater (high; long; many)," or may be read as "greater than A (high; long; many)." On the other hand, the expression "equal to or less than A" may be read as "A or smaller (low; short; few)," or may be read as "smaller than A (low; short; few)." Furthermore, "bigger (higher; longer; more) than A" may be read as "A or greater," and "smaller (lower; shorter; fewer) than A" may be read as "A or less."

[0180] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0181] Each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.

[0182] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0183] The disclosure of the above embodiments includes the following configurations, methods, and programs. [Configuration 1] A system for realizing communication between a first user and a second user, comprising: An acquisition means for acquiring real-time information of the first user; a control means for controlling, on the basis of a purpose of communication, reflection of the real-time information of the first user in the first avatar of the display device, the display device being owned by the second user and displaying a virtual space including the first avatar of the first user; A system comprising: [Configuration 2] The real-time information includes at least information related to the body. 2. The system according to configuration 1, [Configuration 3] The real-time information includes voice, facial expression, blood pressure, heart rate, stress level, body temperature, sweating, Including at least one of information on brainwaves, pulse, posture, and movement; 3. The system according to configuration 2. [Configuration 4] The control means emphasizes or suppresses the real-time information of the first user based on the purpose of the communication, and reflects the information in the first avatar. 4. The system according to any one of configurations 1 to 3. [Configuration 5] the control means generates an image of a virtual space including the first avatar, and controls the display device to display the image of the virtual space. 5. The system according to any one of configurations 1 to 4. [Configuration 6] the control means controls the reflection of the real-time information of the first user in the first avatar based on a reflection setting made by the first user or the second user and a purpose of the communication; The reflection setting is a setting for associating a purpose of the communication with a degree of reflection of the real-time information of the first user in the first avatar. 6. The system according to any one of configurations 1 to 5. [Configuration 7] The control means controls the reflection of the real-time information of the first user in the first avatar based on the disease information of the first user and the purpose of the communication. 7. The system according to any one of configurations 1 to 6. [Configuration 8] The control means Based on the disease information of the first user and the real-time information of the first user, determine whether the first user has experienced a symptom indicated by the disease information; controlling reflection of the real-time information of the first user in the first avatar based on a result of determining whether or not the symptom has occurred; 8. The system of claim 7. [Configuration 9] The control means determines a purpose of the communication based on information constituting the virtual space. 9. The system according to any one of configurations 1 to 8. [Configuration 10] The control means determines a purpose of the communication based on information or an avatar of at least one of the accounts of a plurality of users participating in the community in the virtual space. 10. The system according to any one of configurations 1 to 9. [Configuration 11] The control means determines a purpose of the communication based on information of a tool used by the first user. 11. The system according to any one of configurations 1 to 10. [Configuration 12] The purpose of the communication is a purpose input by at least one of a plurality of users participating in the community of the virtual space. 9. The system according to any one of configurations 1 to 8. [Configuration 13] the control means controls the reflection of the real-time information of the first user in the first avatar based on the nationalities of a plurality of users participating in the community in the virtual space and the purpose of the communication; 13. The system of any one of configurations 1 to 12. [Configuration 14] the control means controls the reflection of the real-time information of the first user in the first avatar based on relationships between a plurality of users participating in the community of the virtual space and a purpose of the communication; 14. The system of any one of configurations 1 to 13. [Configuration 15] The control means controls the reflection of the real-time information of the first user in the first avatar based on the purpose of the communication and the role of the second user. 15. The system of any one of configurations 1 to 14. [Configuration 16] The control means acquires information about the role of the second user based on information about the account of the second user or an operation of the first user. 16. The system according to claim 15, [Configuration 17] The control means acquires information on the role of the second user from an external system. 16. The system according to claim 15, [Configuration 18] The system further includes a warning means for warning the first user before the control means reflects the real-time information of the first user in the first avatar when the real-time information of the first user satisfies a specific condition. 18. The system of any one of configurations 1 to 17. [method] A method for controlling a system for realizing communication between a first user and a second user, comprising: acquiring real-time information of the first user; a control step of controlling, in a display device owned by the second user and displaying a virtual space including a first avatar of the first user, reflection of the real-time information of the first user in the first avatar based on a purpose of the communication; 13. A method for controlling a system comprising: [program] A program for causing a computer to function as each means of the system according to any one of configurations 1 to 18. [Explanation of symbols]

[0184] 101: Server PC, 102: User terminal 201:CPU, 306:CPU

Claims

1. A system for realizing communication between a first user and a second user, comprising: acquiring means for acquiring real-time information of the first user; a control means for controlling reflection of the real-time information of the first user in a facial expression of the first avatar in a display device owned by the second user, the display device displaying a virtual space including the first avatar of the first user; and The control means emphasizes or suppresses the real-time information of the first user based on at least one of the purpose of the communication and the information of the second user, and reflects this in the facial expression of the first avatar. A system characterized by:

2. The real-time information includes information about the body.

2. The system of claim 1.

3. The real-time information includes at least one of information on voice, facial expression, blood pressure, heart rate, stress level, body temperature, sweating, brain waves, pulse, posture, and movement.

3. The system of claim 2.

4. the control means generates an image of a virtual space including the first avatar and controls the display device to display the image of the virtual space.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

5. the control means controls reflection of the real-time information of the first user in a facial expression of the first avatar based on a reflection setting made by the first user or the second user and a purpose of the communication; the reflection setting is a setting that associates a purpose of the communication with a degree of reflection of the real-time information of the first user in a facial expression of the first avatar; 4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

6. the control means controls reflection of the real-time information of the first user in a facial expression of the first avatar based on disease information of the first user.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

7. The control means determining whether the first user has experienced a symptom indicated by the disease information based on the disease information of the first user and the real-time information of the first user; controlling reflection of the real-time information of the first user in a facial expression of the first avatar based on a determination result of whether or not the symptom has occurred; The system of claim 6 .

8. the control means determines the purpose of the communication based on information constituting the virtual space.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

9. the control means determines the purpose of the communication based on account information or avatar of at least one of a plurality of users participating in the community in the virtual space; 4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

10. the control means determines the purpose of the communication based on information about a tool used by the first user.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

11. the purpose of the communication is a purpose input by at least one of a plurality of users participating in the community of the virtual space; 4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

12. the control means controls reflection of the real-time information of the first user in a facial expression of the first avatar based on the nationalities of a plurality of users participating in the community of the virtual space.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

13. the control means controls reflection of the real-time information of the first user in a facial expression of the first avatar based on relationships between a plurality of users participating in the community of the virtual space.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

14. the control means controls reflection of the real-time information of the first user in a facial expression of the first avatar based on a role of the second user.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

15. the control means acquires information about the role of the second user based on information about the account of the second user; 15. The system of claim 14.

16. The control means acquires information about the role of the second user based on an operation of the first user.

15. The system of claim 14.

17. the control means acquires information about the role of the second user from an external system; 15. The system of claim 14.

18. The system further comprises a warning means for warning the first user before the control means reflects the real-time information of the first user in the facial expression of the first avatar when the real-time information of the first user satisfies a specific condition.

4. The system according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

19. A method for controlling a system that realizes communication between a first user and a second user, comprising: acquiring real-time information of the first user; a control step of controlling reflection of the real-time information of the first user in a facial expression of the first avatar in a display device owned by the second user, the display device displaying a virtual space including the first avatar of the first user; and In the control step, the real-time information of the first user is emphasized or suppressed based on at least one of the purpose of the communication and the information of the second user, and is reflected in the facial expression of the first avatar. A method for controlling a system.

20. A program for causing a computer to function as each means of the system according to any one of claims 1 to 3.

Images