Communication support program, communication support method, and communication support system

Abstract

A communication support program supports communication of a plurality of characters in a virtual space, in which the communication support program is configured to cause an object in which one or more characters are located to be displayed integrally with the one or more characters so as to enable a session between the characters among the plurality of characters located in the one object.

Description

Technical Field

[0001] One aspect of the present disclosure relates to a communication support program, a communication support method, and a communication support system.

[0002] This application claims the benefit of priority based on Japanese Application No. 2023-081344 filed on May 17, 2023, and incorporates by reference all the contents described in the aforementioned Japanese Application. Background Art

[0003] In remote communication, Teams (registered trademark), Zoom (registered trademark), etc. have become mainstream in recent years. In these remote communications, participants are pre-set and invited, and when the specified time comes, the participants access the WEB (network) and the like to conduct meetings and other communications. In non-patent document 1, Teams is disclosed as a remote communication.

[0004] Prior art literature

[0005] Non-patent literature

[0006] Non-Patent Literature 1: “Microsoft Teams”, [Online], Microsoft, [retrieved on May 10, 2023], Internet, <URL: https: / / www.microsoft.com / ja-jp / microsoft-teams / group-chat-software> Summary of the invention

[0007] A communication support program of one embodiment of the present disclosure supports communication between multiple characters in a virtual space, wherein the communication support program is configured to: display an object where one or more characters are located together with the one or more characters, so that multiple characters located on one object can have conversations with each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram showing an example of a schematic configuration of a display system according to one embodiment.

[0009] Figure 2 This is a diagram showing an example of functional blocks of a terminal and a server.

[0010] Figure 3 It is a perspective view of the operating device.

[0011] Figure 4 Yes means Figure 3 Figure 2 shows an example of functional blocks of an operating device.

[0012] Figure 5 This is a diagram showing an example of the movement of the ear animation.

[0013] Figure 6 This is a diagram showing an example of a screen displayed in a virtual space.

[0014] Figure 7 This is a diagram showing an example of a screen displayed in a virtual space.

[0015] Figure 8 This is a diagram showing an example of a screen displayed in a virtual space.

[0016] Fig. 9 This is a diagram showing an example of a screen displayed in a virtual space.

[0017] Fig.10 This is a diagram showing an example of a screen displayed in a virtual space.

[0018] Fig.11 This is a diagram showing an example of a screen displayed in a virtual space.

[0019] Fig.12 This is a diagram showing an example of a screen displayed in a virtual space.

[0020] Fig.13 This is a diagram showing an example of a screen displayed in a virtual space.

[0021] Fig.14 This is a diagram showing an example of a screen displayed in a virtual space.

[0022] Fig.15 This is a diagram showing an example of a screen displayed in a virtual space.

[0023] Fig.16 This is a diagram showing an example of a screen displayed in a virtual space.

[0024] Fig.17 This is a diagram showing an example of a screen displayed in a virtual space.

[0025] Fig.18 This is a diagram showing an example of a screen displayed in a virtual space.

[0026] Fig.19 This is a diagram showing an example of a screen displayed in a virtual space.

[0027] Fig. 20 This is a diagram showing an example of a screen displayed in a virtual space.

[0028] Fig.21 This is a diagram showing an example of a screen displayed in a virtual space.

[0029] Fig. 22 This is a diagram showing an example of a screen displayed in a virtual space.

[0030] Fig.23 This is a diagram showing an example of a screen displayed in a virtual space. DETAILED DESCRIPTION

[0031] [Problems to be Solved by the Present Disclosure]

[0032] In the above-mentioned remote communication, it is possible to seek to have conversations with members set as participants, but it is difficult to generate conversations without predetermined communication partners, so-called small talk, etc. in an accidental / natural manner. For example, in fact, in places where people gather (party venues, smoking rooms, free spaces, tea rooms (hot water supply rooms), lounges, cafeterias, corridors, etc.), people in the places will naturally chat with each other. In fact, in places where people gather, the interlocutors look at each other's faces while nodding or responding, so it is possible to grasp the conversation status of the interlocutors by observing their subtle changes in expression and identify the conversation group. Therefore, it is possible to observe the interlocutors' expressions to find the right time to talk to the interlocutors, etc., so small talk can be generated naturally.

[0033] In virtual spaces such as the Metaverse, which is one of the remote communication methods, characters (avatars) can move freely in the virtual space to communicate with other characters. Therefore, it is possible to generate small talk between characters. However, it is not easy to have small talk in the Metaverse. This is because in the Metaverse, non-verbal information such as the character's line of sight and conversation reaction is difficult to accurately express, so it is difficult to grasp the status of each other and difficult to identify which characters are having a conversation.

[0034] An object of the present disclosure is to provide a communication support program, a communication support method, and a communication support system that can easily generate communication between characters in a virtual space.

[0035] [Effects of the present disclosure]

[0036] According to the present disclosure, communication between characters can be easily generated in a virtual space.

[0037] [Description of Embodiments of the Present Disclosure]

[0038] First, the contents of the embodiments of the present disclosure will be described by listing them. At least a part of the embodiments described below may be arbitrarily combined.

[0039] (1) A communication support program according to one aspect of the present disclosure supports communication among multiple characters in a virtual space, wherein the communication support program is configured to: cause an object where one or more characters are located to be displayed as one with the one or more characters, so that multiple characters located on one object can have conversations with each other.

[0040] In a communication support program of one aspect of the present disclosure, when a plurality of characters are located in one object, the characters can have a conversation with each other. That is, a conversation is established between characters located in one object. Therefore, it is possible to understand at a glance which characters are having a conversation with each other. In other words, it is possible to understand at a glance which characters are not having a conversation. In this way, in the communication support program, the state of communication can be visualized. Therefore, in the communication support program, for example, when no communication is taking place, small talk can be asked to a character. Therefore, in the communication support program, small talk, etc. can be generated naturally. As a result, in the communication support program, communication between characters can be easily generated in a virtual space.

[0041] (2) In the communication support program of (1) above, the communication support program may be configured to: when an operation of associating any one of one or more characters with another character different from the character is accepted, an object where the any character is located and an object where the other character is located are integrated, so that the any character and the other character are located in the integrated object. In this configuration, multiple characters can be located in one object by performing an operation of associating any character with the other characters. In addition, by integrating the objects, the sense of unity of the conversation is improved.

[0042] (3) In the communication support program of (2) above, the communication support program may be configured to: when the above-mentioned operation of establishing association is accepted, the user of the other role can select whether to allow the conversation; when the user of the other role allows the conversation, the object where the arbitrary role is located is integrated with the object where the other role is located, so that the arbitrary role and the other role are located in the integrated object; when the user of the other role refuses the conversation, the operation of establishing association is invalidated. In this configuration, the user of the role can select whether to conduct the conversation.

[0043] (4) In the communication support program of (2) or (3) above, the operation of establishing association may be an operation of dragging and dropping any character or the object where the character is located to another character or the object where the character is located. In this configuration, multiple characters can be located on one object by a simple operation such as dragging and dropping. In addition, by providing a transition process such as dragging and dropping, the psychological preparation of the user who operates the character can be completed, so that it can be closer to natural feeling.

[0044] (5) In any of the communication support programs in (2) to (4) above, the communication support program may be configured to: when a plurality of characters are located in one object and an operation to end a conversation of one or more characters is accepted, the object is redisplayed and the one or more characters that have ended the conversation are located in the redisplayed object. In this configuration, when the conversation ends, the characters that have ended the conversation can be located in the object again.

[0045] (6) In the communication support program of (5) above, the operation for ending the conversation may be an operation of dragging and dropping one or more characters from the object to the outside. In this configuration, the conversation can be ended by a simple operation such as dragging and dropping.

[0046] (7) In any one of the communication support programs (1) to (6) above, the object may be movable in the virtual space. In this configuration, the character located in the object can be moved in the virtual space.

[0047] (8) In any of the communication support programs of (1) to (7) above, the communication support program may be configured to enable the character located at the object to communicate with the character not located at the object. In this configuration, for example, it is possible to confirm in advance whether the character located at the object can participate in the conversation.

[0048] (9) In any one of the communication support programs of (1) to (8) above, the communication support program may be configured to display information related to participation in a conversation on an object. In this configuration, whether or not one can participate in a conversation at the object, the degree of participation in the conversation, etc. can be understood at a glance.

[0049] (10) In any of the communication support programs in (1) to (9) above, the communication support program may be configured to: when one or more characters are located at one object, when a specified time comes or a specified operation is received, automatically move the characters to a specified location. In this configuration, a conversation at one object can be ended and the characters can be automatically gathered at a specified location (such as a gathering place).

[0050] (11) In any one of the communication support programs (1) to (10) above, at least one of the plurality of characters located in one object may be operated by AI.

[0051] (12) In the communication support program of (1) above, the communication support program may be configured to automatically integrate an object where any one of the one or more characters is located with an object where other characters different from the character are located, so that the arbitrary character and the other characters are located in the integrated object. In this configuration, multiple characters can be automatically located in one object. That is, the characters can be automatically matched. As a result, not only communication with a specific character (user) can be achieved, but also communication with a new character can be achieved.

[0052] (13) In the communication support program of (12) above, the communication support program may be configured to: when a predetermined time has passed while multiple characters are located on one object, the object is redisplayed, and one or more characters that have ended the conversation are located on the redisplayed object. In this configuration, when the predetermined time has passed, the conversation ends. In this way, the communication time can be limited, so that communication with the same character will not be prolonged. Therefore, communication with many characters can be achieved.

[0053] (14) In the communication support program of (12) or (13) above, the communication support program may be configured to display a display unit on an object where the plurality of characters are located, the display unit displaying information related to the content of the conversation. In this configuration, for example, the topic of the conversation can be displayed (provided) on the display unit as information related to the content of the conversation, thereby enabling smooth communication.

[0054] (15) In the communication support program of (13) above, the communication support program may be configured to display information related to a specified time on an object where a plurality of characters are located. In this configuration, for example, the remaining time of the specified time can be confirmed, so that communication can be performed in consideration of time allocation (end time).

[0055] (16) In any of the communication support programs of (12) to (15) above, the communication support program may be configured to display a character operated by a host user in a virtual space. In this configuration, when applied to a gathering attended by many users (e.g., a training session, an information exchange meeting, a party, etc.), the gathering can be promoted by the host, thereby enabling the gathering to proceed smoothly.

[0056] (17) In any one of the communication support programs of (12) to (16) above, the communication support program may be configured to enable the exchange of cards when a plurality of characters are located in one object, wherein the card at least includes information that can identify the user of the character. In this configuration, the character (user) who conducted the conversation can be confirmed after the conversation ends.

[0057] (18) In any one of the communication support programs (12) to (17) above, at least one of the plurality of characters located in one object may be operated by AI.

[0058] (19) A communication support method according to one aspect of the present disclosure supports communication among multiple characters in a virtual space. In the communication support method, an object where one or more characters are located is displayed as one with the one or more characters, so that multiple characters located on one object can have conversations with each other.

[0059] The communication support method according to one aspect of the present disclosure includes the same features as the communication support program described above, and thus can achieve the same functions and effects as the communication support program described above.

[0060] (20) A communication support system according to one aspect of the present disclosure is a communication support method for supporting communication among multiple characters in a virtual space, wherein the communication support system is configured to: cause an object where one or more characters are located to be displayed as one with the one or more characters, so that multiple characters located on one object can have conversations with each other.

[0061] The communication support system according to one aspect of the present disclosure includes the same features as the communication support program described above, and thus can achieve the same functions and effects as the communication support program described above.

[0062] [Details of the embodiments of the present disclosure]

[0063] Hereinafter, specific examples of the embodiments of the present disclosure are described with reference to the accompanying drawings. The present disclosure is not limited to these examples, but is shown by the claims, and is intended to include all changes within the meaning and scope equivalent to the claims. In the description of the drawings, the same reference numerals are marked on the same elements, and repeated descriptions are omitted.

[0064] Figure 1 1 is a diagram showing an example of a schematic configuration of a communication support system (hereinafter, sometimes referred to as a "system"). The system 100 supports a virtual space R (see Figure 6 ) wherein the communication uses a character C (refer to Figure 5 Mode A), the ear animation E of the character C (refer to Figure 5 Mode A) performs actions based on user operations.

[0065] The operation of the character C may be performed by human users U1, U2, and U3, or by subjects other than humans. The operation of the character C may be performed by, for example, AI (Artificial Intelligence) (ChatGPT, Siri (registered trademark), Google Assistant (registered trademark), Alexa (registered trademark), Cortana (registered trademark)). That is, the user who operates the character C may also be AI. The user refers to an operator who operates the character C, etc.

[0066] The character C represents the user. The character C is a dynamic character. The character C is generated by an image material that is independent of the actual image (photographed data, etc.) of the user. The image material is a material used to generate an animated character. The character C can also be drawn by two-dimensional or three-dimensional computer graphics (CG: Computer Grapgic).

[0067] As an example, character C is composed of body B (see Figure 5 Mode A) and ear animation E. In character C, body B and ear animation E are displayed as one. Being displayed as one means that when character C is seen, body B and ear animation E are visually recognized as a whole. That is, being displayed as one means that it is possible to recognize at a glance that body B and ear animation E are related (corresponding). Therefore, being displayed as one can be a state in which body B and ear animation E are in contact or a state in which body B and ear animation E are separated.

[0068] The body B may also include at least one of an image representing the user and text representing the user. A shadow may also be displayed on the character C. The body B may also be spherical and include text ("A", "B", "C", etc.) that can identify the user. The character C may be configured to include a speech animation (not shown) or an animation that simulates arms, legs, etc.

[0069] Ear animation E is an animation that expresses non-verbal information. Ear animation E is an animation that expresses the user's state (action) and emotion. Ear animation E is an animation that expresses the user's state in a form different from the user's actual action part. Ear animation E may or may not include dynamically changing facial expressions. Ear animation E is displayed as protruding from the display area of ​​body B to the outside of the display area. Ear animation E is an animation that simulates the ears of animals (for example, rabbits, etc.), and moves according to changes in the user's state. Ear animation E moves according to the user's actions. Ear animation E performs actions such as stretching, bending, and swinging. Ear animation E can be deformed freely, and performs visual deformation corresponding to the user's state so as to remind people of the user's state.

[0070] In the system 100 , the user's state, emotion, reaction, etc. can be confirmed through the ear animation E of the character C.

[0071] <System Configuration>

[0072] System 100 includes terminal 10, terminal 20 and terminal 30. Figure 1 In the example shown, terminal 10 is a laptop personal computer (PC) used by user U1. Terminal 20 is a portable telephone (smartphone, etc.) used by user U2. Terminal 30 is a desktop PC used by user U3. Terminal 10, terminal 20, and terminal 30 can connect to network N. In this example, terminal 10 is connected to network N via access point AP, terminal 20 is connected to network N via base station BS, and terminal 30 is connected to network N via a wire (not shown). For terminal 10, terminal 20, and terminal 30, in addition to the above-mentioned PCs and portable telephones, tablet terminals, wearable terminals, head-mounted display (HMD) terminals, and various other terminals can also be used. It should be noted that the number of terminals included in system 100 is not limited to Figure 1 Example.

[0073] The system 100 includes a server 40. The server 40 is connected to a network N. Therefore, the terminal 10, the terminal 20, the terminal 30 and the server 40 can communicate with each other via the network N. It should be noted that the number of servers included in the system 100 is not limited to Figure 1 Example.

[0074] There is no limitation on the configuration of the network N. For example, the network N may be configured to include the Internet (public network), a communication carrier network, an operator network, and the like.

[0075] Figure 2 1 is a diagram showing an example of functional blocks of the terminal 10, the terminal 20, the terminal 30, and the server 40. The functional blocks of the terminal 20 and the terminal 30 are the same as the functional blocks of the terminal 10, so the terminal 10 and the server 40 will be described below. The terminal 10 will be described first, and then the server 40 will be described.

[0076] <Terminal Configuration>

[0077] The terminal 10 includes an input unit 11. The input unit 11 is a part for inputting voice data and image data of the user U1. The input unit 11 may also be used to input image data of the user U1. The image includes the voice data and image data of the user U1.

[0078] The terminal 10 includes a control unit 12. The control unit 12 is a part that generates control data of the user U1. The control data of the user U1 is generated based on the action data output from the operating device 50. In the present embodiment, for example, as control data, data for causing the ear animation E of the character C to perform six actions of "nodding", "agreeing", "appreciating (clapping)", "asking questions", "raising hands" and "surprised" can be generated. The control unit 12 can also generate control data related to other actions (for example, actions indicating "yes" and "no").

[0079] The control data and the action data are associated with each other. In the present embodiment, for example, the action data and the control data of "nod", "agree", "applaud (clap)", "ask", "raise hand", and "surprise" are associated with each other. The action data is data sent from the operating device 50. When the action data and the control data are associated with each other, the control unit 12 generates the control data.

[0080] The control data may also be generated based on the voice data and image data of the user U1 input to the input unit 11. The image data is composed of a plurality of continuous unit frames. The unit frame generally includes the image information of the user U1. For example, image data of 60 frames / second includes 60 unit frames in one second. The details of the generation of the control data by the control unit 12 will be described later.

[0081] The control unit 12 is also a part that controls the motion of the character C based on the control data. The control unit 12 causes the character C to be displayed in the virtual space R and controls the motion of the ear animation E of the character C based on the generated control data and the received control data.

[0082] The control unit 12 generates image data of the virtual space R. The image data of the virtual space R includes image data of the character C controlled based on the control data. In addition, the image data of the virtual space R may also include various objects that are to appear in the virtual space R. The method for generating the virtual space R based on the control unit 12 is not particularly limited, and various well-known methods can be used. The control unit 12 may also generate the image data of the virtual space R based on a captured image obtained by a camera, for example. Alternatively, the control unit 12 may also generate the image data of the virtual space R based on three-dimensional CAD (Computer Aided Design) data.

[0083] The terminal 10 includes a storage unit 13. The storage unit 13 is a part that stores information necessary for the control (processing) of the terminal 10. Figure 21 shows a terminal program 13P as information stored in the storage unit 13. The terminal program 13P is a program for operating the terminal 10, and operates the computer in a manner such as to execute the control performed by the control unit 12. Since the system 100 is a communication support system, the terminal program 13P may also be called a communication support program.

[0084] The terminal 10 includes a communication unit 14. The communication unit 14 communicates with the network N (see Figure 1 ) is a part that communicates with an external device of the terminal 10. Examples of the external device of the terminal 10 are the terminal 20, the terminal 30, the server 40, and the operating device 50. The communication unit 14 sends the voice data and the control data generated by the control unit 12 to the server 40, or receives the voice data and the control data from the server 40. The communication unit 14 is a part that communicates with the operating device 50. The communication unit 14 receives the action data from the operating device 50.

[0085] The terminal 10 includes an output unit 15 . The output unit 15 outputs an image of the virtual space R. The image of the virtual space R is an image based on image data of the virtual space R generated by the control unit 12 .

[0086] <Server Configuration>

[0087] The server 40 includes a control unit 42. The control unit 42 controls the operation of the server 40. The server 40 includes a storage unit 43. The storage unit 43 is a part that stores information necessary for controlling the server 40. Figure 3 4 shows a server program 43P and received data 43S as information stored in the storage unit 43. The server program 43P is a program for causing the server 40 to operate, and causes the computer to operate in a manner such as executing the control performed by the control unit 42. The received data 43S is data sent from the terminal 10, the terminal 20, and the terminal 30 to the server 40, and may include control data generated by the terminal 10, the terminal 20, and the terminal 30.

[0088] The server 40 includes a communication unit 44. The communication unit 44 communicates with the server 40 via a network N (see Figure 1 ) communicates with an external device of the server 40. Examples of the external device of the server 40 are the terminal 10, the terminal 20, and the terminal 30. The communication unit 44 receives control data of the user U1, the control data of the user U2, and the control data of the user U3 from the terminal 10, the terminal 20, and the terminal 30, respectively, or transmits control data generated by the terminal 10, the terminal 20, and the terminal 30 to each of the terminals 10, the terminal 20, and the terminal 30.

[0089] <Hardware Configuration>

[0090] An example of the hardware configuration related to each functional part of the terminal 10 and the server 40 is described. The communication unit 44 of the server 40 can be configured using a network card or a wireless communication device so that it can access the network N. The control unit 42 of the server 40 can be configured using a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a clock, and a built-in memory. The control unit 42 can also be configured as a hardware (SoC: System On a Chip) obtained by integrating a processor, a clock, a built-in memory, a storage unit 43, and a communication unit 44. The control unit 42 operates based on the server program 43P, thereby causing the server computer to operate as the server 40. The storage unit 43 of the server 40 can be configured using a non-volatile storage medium such as a flash memory, a hard disk, or an SSD (Solid State Disk).

[0091] The input unit 11 of the terminal 10 can be configured using a microphone, a camera, and the like. The microphone can acquire the voice of the user U1. That is, the microphone is a type of voice sensor that acquires voice. The camera can acquire the image of the user U1. That is, the camera is a type of image sensor that acquires images. Moreover, the input unit 11 can also be configured using operating devices such as a keyboard, a mouse, and a touch panel. The control unit 12 of the terminal 10 can be configured in the same manner as the control unit 42 of the server 40. The control unit 12 operates based on the terminal program 13P, thereby causing a general-purpose computer to operate as the terminal 10. The storage unit 13 of the terminal 10 can be configured in the same manner as the storage unit 43 of the server 40. The communication unit 14 of the terminal 10 can be configured in the same manner as the communication unit 44 of the server 40. The output unit 15 of the terminal 10 can be configured using a display device such as a liquid crystal panel, an organic EL (Electro Luminescent) panel, or the like (it can also be a touch panel). Moreover, the output unit 15 can also be configured using a speaker.

[0092] <Configuration of the operating device>

[0093] Figure 3 It is a perspective view of the operating device 50 . Figure 3 1 is a diagram showing a state where the operating device 50 is being operated by the finger F of the user U1, the user U2, and the user U3. The operating device 50 is a device operated by the user U1, the user U2, and the user U3. The ear animation E moves by the operation of the operating device 50 by the user U1, the user U2, and the user U3. The operating device 50 is a touch panel. As the touch panel, an electrostatic capacitance method, a pressure-sensitive touch method, an ultrasonic method, etc. can be used.

[0094] The operating device 50 is provided with a main body 51 as a hardware structure. The main body 51 has an operating surface 51A. The operating device 50 is operated by a finger F of a hand H of the user U1, the user U2, and the user U3. The operating device 50 may be provided separately or may be provided as part of a remote controller for operating a television display or the like. That is, the operating device 50 may also be part of a device.

[0095] Figure 4 is a diagram showing an example of a functional block of the operating device 50. Figure 4 As shown, the operating device 50 includes a communication unit 52. The communication unit 52 is a part that communicates with an external device by wireless communication or wired communication. Examples of wireless communication are LAN (Local Area Network), Bluetooth (registered trademark), Wifi, etc. Examples of external devices are the terminal 10, the terminal 20, and the terminal 30. The communication unit 52 sends the action data output from the control unit 54 to the terminal 10, the terminal 20, and the terminal 30.

[0096] The operation device 50 includes an input unit 53. The input unit 53 is a part that inputs the operation of the user U1, the user U2, and the user U3 on the operation surface 51A. The input unit 53 detects the operation (gesture) of the operation surface 51A performed by the user U1, the user U2, and the user U3. Examples of the operation include click, tap, slide, and scroll. The input unit 53 outputs an operation signal to the control unit 54.

[0097] The operating device 50 includes a control unit 54. The control unit 54 generates motion data based on the operation signal of the input unit 53. The control unit 54 outputs the motion data to the communication unit 52.

[0098] The control unit 54 may be configured using a processor such as a CPU or a GPU, a clock, and a built-in memory. The control unit 54 may also be configured as a single hardware (SoC) in which the processor, the clock, the built-in memory, and the communication unit 52 are integrated.

[0099] The operating device 50 includes a battery 55 . The battery 55 supplies power to each part of the operating device 50 .

[0100] Next, the operation of the operating device 50 performed by the user U1, user U2, and user U3 in making the ear animation E of the character C move, for example, in a manner of "nodding", "agreeing", "appreciating (clapping)", "asking a question", "raising hands", or "surprised" is described. That is, the relationship between the movement of the ear animation E of the character C and the movement of the user U1, user U2, and user U3 detected by the operating device 50 is described. The movement of the ear animation E is similar to (related to) the movement of the finger F of the user U1, user U2, and user U3 used to move the ear animation E. For example, in the case of moving the ear animation E of "nodding", a movement similar to the nodding movement is performed on the operating surface 51A of the operating device 50 with the finger F. That is, the user U1, user U2, and user U3 can make the ear animation E move by performing a movement similar to the ear animation E that they want to move at the operating device 50.

[0101] Figure 5 : is a diagram showing an example of the movement of the ear animation E. Figure 5 Mode A shows the ear animation E as it normally would.

[0102] Figure 5 Mode B of FIG. 4 shows the “nodding” action in the ear animation E. Figure 5 As shown in the pattern B, for the "nod" action, in the ear animation E of the character C, the upper 1 / 2 of the ear is bent forward. In order to make the ear animation E perform the "nod" action, the user U1, the user U2, and the user U3 touch the operation surface 51A of the main body 51 of the operation device 50 with two fingers (for example, the index finger and the middle finger). Alternatively, the user U1, the user U2, and the user U3 slide the two fingers F while touching the operation surface 51A. The sliding amount of the fingers F is, for example, 1 cm to 2 cm.

[0103] When the above-mentioned action is performed, the operating device 50 detects the action through the input unit 53 and sends the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for making the ear animation E move in a "nodding" manner based on the action data.

[0104] Figure 5The "agree" action in the ear animation E is shown in mode C. For the "agree" action, in the ear animation E of the character C, the ear is bent forward as a whole. In order to make the ear animation E perform the "agree" action, the user U1, the user U2, and the user U3 slide two fingers F (for example, the index finger and the middle finger) while the user U1, the user U2, and the user U3 touch the operation surface 51A. The sliding amount of the finger F is, for example, longer than 2 cm, for example, 3 cm to 4 cm. That is, in order to make the ear animation E perform the "agree" action, the user U1, the user U2, and the user U3 slide the finger F longer than the sliding amount of the finger F when the ear animation E performs the "nod" action.

[0105] When the above-mentioned action is performed, the operating device 50 detects the action through the input unit 53 and sends the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for causing the ear animation E to move in an "agree" manner based on the action data.

[0106] Figure 5 The “clapping (appreciation)” action in the ear animation E is shown in mode D of FIG. For the “clapping (appreciation)” action, in the ear animation E of the character C, the ears cycle closer and farther apart. In order to make the ear animation E perform the “clapping (appreciation)” action, the user U1, the user U2, and the user U3 move the two fingers F (for example, the thumb and the index finger, or the index finger and the middle finger) closer (contact) and farther apart in a cycle while touching the operation surface 51A.

[0107] When the above-mentioned action is performed, the operating device 50 detects the action through the input unit 53 and sends the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for making the ear animation E move in a "clapping (appreciation)" manner based on the action data. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 may also generate control data in the following manner: for the action data related to one action of making two fingers F move closer and apart in a cycle, clap multiple times.

[0108] Figure 5Mode E shows the "ask a question" action in the ear animation E. For the "ask a question" action, in the ear animation E of the character C, the ears are entangled. In order to make the ear animation E perform the "ask a question" action, the user U1, the user U2, and the user U3 slide two fingers F (for example, the index finger and the middle finger) in a manner of drawing a circle (semicircle) while touching the operation surface 51A. It should be noted that in order to make the ear animation E perform the "ask a question" action, the user U1, the user U2, and the user U3 may also slide two fingers F (for example, the index finger and the middle finger) in a manner of drawing ∞ (infinity) while touching the operation surface 51A.

[0109] When the above-mentioned action is performed, the operating device 50 detects the action through the input unit 53 and sends the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for causing the ear animation E to move in the form of "asking questions" based on the action data.

[0110] Figure 5 The "hand-raising" action in the ear animation E is shown in the mode F of FIG. For the "hand-raising" action, the ear of the character C extends in the ear animation E. In order to make the ear animation E perform the "hand-raising" action, the user U1, the user U2, and the user U3 slide only one finger F (middle finger) upward while the user U1, the user U2, and the user U3 touch the operation surface 51A with two fingers F (for example, the index finger and the middle finger).

[0111] When the above-mentioned action is performed, the operating device 50 detects the action through the input unit 53 and sends the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for making the ear animation E move in a "hand-raising" manner based on the action data.

[0112] Figure 5 The "surprise" action in the ear animation E is shown in the mode G of FIG. For the "surprise" action, in the ear animation E of the character C, the top of the ear becomes larger. At this time, the body B of the character C bends backward. In order to make the ear animation E perform the "surprise" action, the user U1, the user U2, and the user U3 touch the operation surface 51A with two fingers F (for example, the index finger and the middle finger) and slide the two fingers F upward while separating them.

[0113] When the above-mentioned action is performed, the operation device 50 detects the action through the input unit 53 and transmits the action in the form of action data to the terminal 10, the terminal 20, and the terminal 30. The control unit 12 of the terminal 10, the terminal 20, and the terminal 30 generates control data for making the ear animation E move in a "surprised" manner based on the action data.

[0114] <How to operate the ear animation>

[0115] Next, the method of operating the ear animation E of the character C is described. When operating the ear animation E, the user U1, the user U2, and the user U3 perform the operation with Figure 5 Mode B Figure 5 Mode C, Figure 5 Mode D, Figure 5 Mode E, Figure 5 Mode F or Figure 5 The operation device 50 generates the operation data in the operation device 50. The operation device 50 transmits the operation data to the terminal 10, the terminal 20, and the terminal 30.

[0116] When the terminal 10, the terminal 20, and the terminal 30 receive the action data, the terminal 10, the terminal 20, and the terminal 30 generate control data corresponding to the action data. As a result, in the virtual space R of the terminal 10, the terminal 20, and the terminal 30, the ear animation E of the character C performs any action of "nodding", "agreeing", "appreciating (clapping)", "asking questions", "raising hands" or "surprised". In this way, the action of the ear animation E of the character C is operated by the operation of the operating device 50 performed by the user U1, the user U2, and the user U3.

[0117] <Methods of communication>

[0118] The method of conversation (dialogue) between characters C in the virtual space R is described. First, the virtual space R is described. In the virtual space R, multiple characters C can exist. The virtual space R is, for example, a metaverse. In the virtual space R, in addition to the characters C, various items and the like can also exist. In the virtual space R, there can also be BGM (BackGroundMusic: background music), broadcasts (guidance), etc. The BGM and broadcasts can also be changed according to the position of the character C. Various activities can be carried out in the virtual space R. Examples of activities include lectures, speeches, training, meetings, reports, consultations, negotiations, entertainment, sightseeing, and the like.

[0119] In the virtual space R, the character C can be configured on a mobile station (object) S. That is, the character C can ride on the mobile station S. The mobile station S is displayed integrally with one or more characters C. The mobile station S forms an independent area where the character C is located. An independent area refers to an area that can be distinguished from other areas. In other words, the area of ​​a certain mobile station S (hereinafter also referred to as the "first mobile station S") and the area of ​​another different mobile station S (hereinafter also referred to as the "second mobile station S") are independent of each other. Thus, it can be recognized at a glance that a certain character C is riding on the first mobile station S and another character C is riding on the second mobile station S that is different from the first mobile station S. It should be noted that an independent area may also include multiple parts.

[0120] The character C is located on a mobile station S. The mobile station S is configured to be movable in a virtual space R. The moving direction and speed of the mobile station S can be operated by the user. That is, the user can move the mobile station S to any position. In the case where multiple characters C are riding on the mobile station S, it can also be configured that: a control seat is provided on the mobile station S, and the user of the character C located at the control seat can preferentially operate the mobile station S. The mobile station S can also be moved by automatic operation. In this case, the mobile station S can automatically move along a set route, for example, or automatically move to a set destination. In addition, the user can arbitrarily change the display viewpoint. For example, the viewpoint can be rotated in 3D (3-Dimension).

[0121] One or more characters C may be configured on the mobile station S. The size of the mobile station S may also vary according to the number of configured characters C. Specifically, the size of the mobile station S may increase in response to an increase in the number of configured characters C, or may decrease in response to a decrease in the number of configured characters C. The size of the mobile station S may change in stages, or the size of the mobile station S may change continuously (gradually) as the number of configured people increases or decreases. It should be noted that the size, design, etc. of the mobile station S may also be appropriately set by the user. Identification information, text information, etc. may also be set on the mobile station S.

[0122] Next, refer to Figure 6 , an example of a method for starting a conversation between role C1 (any role in one or more roles C) and role C2 (another role different from role C1) is described. The user of role C1 can be user U1, user U2, and user U3. The user of role C2 can be user U1, user U2, user U3, and AI. In the following, for example, a scenario in which the user of role C1 is user U1 and the user of role C2 is user U2 is described as an example.

[0123] exist Figure 6 In the example shown, character C1 is on the mobile station S, and character C2 is on the mobile station S. Figure 6 In the state shown, character C1 and character C2 cannot communicate. In other words, character C1 and character C2 cannot have a conversation. Figure 6 In the state shown, both the character C1 and the character C2 can hear the BGM, broadcast, etc. played in the virtual space R.

[0124] When a session is requested from role C1 to role C2, a consent process is performed at role C2 to confirm whether the session can be performed. The consent process is performed when an operation of associating role C1 with role C2 is accepted. In this embodiment, the operation of associating role C1 with role C2 is an operation of dragging and dropping role C1 to the mobile station S of role C2.

[0125] During the consent process, Figure 7 When the character C1 is dragged and dropped onto the mobile station S of the character C2, a message for agreeing to the conversation is automatically sent to the user U2 of the character C2. The message may include, for example, a text including "Is it convenient for me to talk to you now?" and user information (e.g., name, user name, etc.) that can identify the character C1.

[0126] It should be noted that the character C1 can also be dragged and dropped to the character C2, and the mobile station S where the character C1 is located can also be dragged and dropped to the character C2 or the mobile station S where the character C2 is located. Figure 6 As shown in the figure, the operation of establishing association between the role C1 and the role C2 can be performed in a state where the mobile station S where the role C1 is located is separated from the mobile station S where the role C2 is located. Figure 8 As shown, the operation of associating the characters C1 and C2 is performed in a state where the mobile station S where the character C1 is located and the mobile station S where the character C2 is located are close to (in contact with) each other.

[0127] When a message of consent to the conversation is sent to the character C2, the message is displayed on the screen of the user U2 of the character C2. The above text and user information are displayed on the message display screen, and an "OK button" for consenting to the conversation and an "NG button" for rejecting the conversation are displayed. When the user of the character C2 selects the "NG button", the screen of the user U1 of the character C1 displays the content to the effect that the conversation is not consented to. As an example of the display, for example, "I'm sorry, something's up", "Cannot be handled before 00 o'clock", "Can be handled from 00 o'clock onwards", etc. The content of the message sent when the NG button is selected can be appropriately set by the user. When the user of the character C2 selects the "NG button", the operation of dragging and dropping the character C1 to the mobile station S of the character C2 is invalid (invalidated).

[0128] When user U2 of role C2 selects the "OK button", Fig. 9As shown, the animation of the process of combining the mobile station S where the character C1 is located and the mobile station S where the character C2 is located into one is executed, such as Fig.10 As shown, one mobile station S is displayed. That is, the mobile station S of character C1 and the mobile station S of character C2 are integrated. The size of the mobile station S varies according to the number of characters C located at the mobile station S. Specifically, the mobile station S used by the two characters C1 and C2 is larger than the mobile stations S used by the characters C1 and C2. Fig. 9 The animation shown can also be found in Figure 6 or Figure 7 The state shown changes to Fig.10 Status shown.

[0129] Characters C1 and C2 are located at the same mobile station S. Characters C1 and C2 are located at one mobile station S. At the mobile station S, characters C1 and C2 can have a conversation. That is, a conversation can be held (conversation is established) between multiple characters C1 and C2 located at one (the same) mobile station S. Characters outside the mobile station S cannot hear the conversation between characters C1 and C2 at the mobile station S. At the mobile station S, broadcasts played in the virtual space R can be heard.

[0130] The volume of the conversation in the mobile station S can be set appropriately. The volume of the conversation can also be adjusted so that the sound leaks to a certain extent outside the mobile station S. In this case, it is easier to create a natural atmosphere such as some kind of conversation is going on compared to a case where there is no sound. Instead of sound, a changing sound linked to sound can be emitted.

[0131] It should be noted that the function of contacting the characters C1 and C2 from outside the mobile station S can also be provided. For example, it can also be set that users outside the mobile station S can make voice calls and WEB exchanges (email, chat, etc.) with the characters C1 and C2 mounted on the mobile station S. For example, voice calls, etc. can be made on a character basis or on a mobile station basis. When making a voice call to a character, the number set for each character is contacted. When making a voice call to a station, the carrier number set for each mobile station is contacted. Users of the characters C1 and C2 can notify specific users of the carrier number.

[0132] In the above method, when the character C1 is dragged and dropped onto the mobile station S of the character C2, a message for the conversation consent is automatically sent to the user U2 of the character C2. However, it may also be that when the user U1 of the character C1 clicks the character C2 as the conversation object, a message for the conversation consent is automatically sent to the user U2 of the character C2. In addition, it may also be that when the character C2 is clicked, a function selection window is displayed, and a message is sent to the character C2 by selecting (clicking) the "agree to send message" function displayed in the function selection window. In this case, after the user U2 of the character C2 agrees to the conversation, when the character C1 is dragged and dropped onto the mobile station S of the character C2, the mobile station S on which the characters C1 and C2 are riding is displayed.

[0133] In addition, if communication (meeting plan, etc.) is pre-set between character C1 and character C2, the consent process can be omitted. In this case, when character C1 is dragged and dropped onto the mobile station S of character C2, the mobile station S on which characters C1 and C2 are riding is displayed, and characters C1 and C2 can have a conversation.

[0134] When the conversation between the characters C1 and C2 is to be ended, the user U1 performs an operation to end the conversation. In this embodiment, the operation to end the conversation is an operation of dragging and dropping the character C to the outside of the mobile station S. Specifically, Fig.11 As shown, the user U1 drags and drops the character C1 to the outside of the mobile station S. As a result, the mobile station S is redisplayed, and the character C1 is located on the mobile station S (see Figure 6 ). In this state, character C1 and character C2 cannot communicate. That is, character C1 and character C2 cannot have a conversation. Through the above, the conversation between character C1 and character C2 ends.

[0135] In addition, the conversation between character C1 and character C2 can also be automatically ended based on a preset time. The preset time can be any set time or a time set by a timer. Fig.11 The status shown automatically changes to Figure 6 That is, two mobile stations S are displayed, and characters C1 and C2 are located at the two mobile stations S respectively. Alternatively, characters C1 and C2 automatically move to a gathering place (which can be a mobile station S where multiple characters C gather, or any place in the virtual space R). Thus, the conversation between characters C1 and C2 ends. In addition, the conversation between characters C1 and C2 can also be ended by the administrator (the administrator of the virtual space R). In this case, either from Fig.11 The status shown automatically changes to Figure 6 In the state shown, the characters C1 and C2 may automatically move to the gathering place.

[0136] Next, a case where a plurality of characters C are on board a mobile station S and the plurality of characters C apply for a session together will be described. Fig.12 In the example shown, characters C1, C2, C3, and C4 are on a certain mobile station S, and characters C5 and C6 are on another mobile station S different from the mobile station S. Characters C1, C2, C3, and C4 constitute group 1. Characters C5 and C6 constitute group 2. The following describes a case where group 2 applies for a session to group 1.

[0137] An administrator is set in each of group 1 and group 2. The administrator has the authority to control the group's conversations in a unified manner. The administrator has the authority to operate the mobile station S. The administrator can be set as one or more users. Here, an example is used to explain the scenario in which the administrator of group 1 is role C1 and the administrator of group 2 is role C5.

[0138] When group 2 requests a session with group 1, a consent process for confirming whether the session can be conducted is implemented at group 1. In the consent process, when the role C5 as a manager is dragged and dropped to the mobile station S of group 1, a message for session consent is automatically sent to the user of the role C1 as the manager of group 1.

[0139] When the user of role C1 selects the "NG button", the screen of the user of role C5 displays the content that the conversation is not approved. When the user of role C1 selects the "OK button", Fig.13 As shown, one mobile station S is shown. That is, the mobile stations S of group 1 and the mobile stations S of group 2 are integrated.

[0140] Role C1, role C2, role C3, role C4, role C5, and role C6 are located at the same mobile station S. Role C1, role C2, role C3, role C4, role C5, and role C6 are located at one mobile station S. At the mobile station S, roles C1, role C2, role C3, role C4, role C5, and role C6 can have a conversation. That is, it is set to be possible to have a conversation (make a conversation established) between multiple roles C1, C2, C3, C4, C5, and C6 located at one mobile station S. At the mobile station S, roles C1, role C2, role C3, role C4, role C5, and role C6 can all have a conversation, or roles C1, role C2, role C3, role C4, role C5, and role C6 can have a conversation with each other (for example, role C1 and role C2, role C3 and role C4, role C5 and role C6).

[0141] Next, the case of ending a session in a group is described. In the case of ending a session in a group, for example, Fig.13 As shown, the user of role C1, which is the manager of group 1, frames roles C1, C2, C3, and C4 so as to include roles C1, C2, C3, and C4, which are members of group 1. Thus, roles C1, C2, C3, and C4 are surrounded by dotted lines, for example. The user of role C1 drags and drops the portion surrounded by the dotted lines to the outside of mobile station S. As a result, mobile station S is redisplayed, and roles C1, C2, C3, and C4 of group 1 are located on mobile station S. That is, as shown in FIG. Fig.12 As shown, characters C1, C2, C3, and C4 are on a certain mobile station S, and characters C5 and C6 are on another mobile station S different from the mobile station S. As described above, the conversation between group 1 and group 2 ends.

[0142] It should be noted that the designation of the constituent members of group 1 is not limited to the above method, and can also be done by clicking to select role C1, role C2, role C3, and role C4, or by entering information that can identify role C1, role C2, role C3, and role C4 in a specified screen.

[0143] In to be Fig.13 In the state shown, when the conversation is ended in a role unit, for example, the user of role C1 drags and drops role C1 outside the mobile station S. As a result, the mobile station S is redisplayed, role C1 is located at a certain mobile station S, and roles C2, C3, C4, C5, and C6 are located at other mobile stations S different from the mobile station S. In this state, role C1 cannot communicate with role C2, C3, C4, C5, and C6. In other words, role C1 cannot have a conversation with role C2, C3, C4, C5, and C6. Through the above process, the conversation ends.

[0144] It should be noted that, similar to the above case, the conversation between group 1 and group 2 may be automatically terminated based on a preset time or may be terminated by an administrator.

[0145] like Fig.14 As shown, a cover CV can be set on the mobile station S. For example, the cover CV is set on the mobile station S in a manner of covering the characters C1 and C2. The size of the cover CV can be changed according to the size of the mobile station S. The cover CV is, for example, hemispherical. The cover CV can be transparent, semi-transparent, or non-transmissive. Fig.14 In the example shown, the cover CV is transparent.

[0146] The cover CV is information related to the conversation at the mobile station S. The cover CV has a function of notifying the outside of the status related to participating in the conversation. The information related to the conversation is information indicating whether or not the conversation can be participated in, information indicating the degree of possibility of participating in the conversation, and the like. In the present embodiment, the information related to the conversation is information indicating whether or not the conversation can be participated in. In the present embodiment, when the cover CV is displayed, it indicates that it is basically impossible to participate in the conversation. The degree of whether or not the conversation at the mobile station S (the state inside the mobile station S) can be notified to users around the mobile station S by the degree of transparency, color, and the like of the cover CV. For example, when the cover CV is transparent, it can be indicated that the conversation can be participated in by applying for the conversation. When the cover CV is not transparent (opaque), it can be clearly indicated that participation in the conversation from the outside is rejected (denied). In addition, the cover CV can indicate the degree of possibility of participating in the conversation in stages by the degree of transparency (transparent, semi-transparent, opaque, etc.).

[0147] The cover CV can be set by the user to display / not display. Fig.14 When multiple characters C1 and C2 are on the mobile station S, the user set as the administrator can set the display / non-display of the cover CV. The cover CV indicates that only authorized users can participate in the conversation. Fig.12 As shown, even if the user of character C drags and drops character C3 to the mobile stations S of characters C1 and C2, the drag and drop will be rejected.

[0148] Alternatively, when the mobile station S is set with a cover CV, it is possible to contact the characters C1 and C2 in the cover CV. For example, when a meeting is ongoing at the mobile station S, a user of character C who is late for the meeting can make a voice call or communicate on the web with the user or the entire group on the mobile station S. Thus, the cover CV is set to be temporarily hidden by the user of the characters C1 and C2 on the mobile station S, so that, for example, the character C3 can be dragged and dropped to the mobile station S. It should be noted that, when the cover CV is opaque, it can be set to reject contact from the outside.

[0149] Furthermore, the necessity of the consent process may be set by displaying / not displaying the cover CV (the presence or absence of the cover CV). Specifically, the following setting may be made: when the cover CV is displayed, the consent process is required, and when the cover CV is not displayed, the consent process is not required.

[0150] In addition, it can also be that Fig.15 In the case where the mobile station S is provided with an object having a display function, such as a projection board PB, when the cover CV does not display, texts, images, etc. displayed on the projection board PB can be visually confirmed from the outside of the mobile station S. Fig.16When the cover CV is displayed as shown, the text and the like displayed on the projection board PB cannot be visually confirmed from the outside of the moving stage S. Fig.16 In the example shown, the cover CV is transparent, and the character C located at the mobile station S can be visually confirmed from the outside, but the text, etc. displayed on the projection board PB cannot be visually confirmed (the text, etc. displayed on the projection board PB is not displayed). In this case, it can be set that: when the cover CV is displayed, the character C located at the mobile station S can be confirmed from the outside, and the display content of the projection board PB cannot be confirmed from the outside. Thus, for example, when a meeting is being held at the mobile station S, the participants and status of the meeting can be confirmed from the outside, but the content of the meeting cannot be confirmed from the outside.

[0151] In addition, the projection board PB when the cover CV is displayed may not be displayed, or the display may be blurred, etc. In short, it is sufficient as long as the content displayed on the projection board PB cannot be recognized (understood) from the outside of the moving station S.

[0152] [Effect]

[0153] As described above, in the system 100 of the present embodiment, when a plurality of characters C are located on one mobile station S, the characters C can have a conversation with each other. That is, a conversation is established between the characters C located on one mobile station S. Therefore, it is possible to understand at a glance which characters C are having a conversation with each other. In other words, it is possible to understand at a glance which characters C are not having a conversation. In this way, in the system 100, the state of communication can be visualized. Therefore, in the system 100, for example, when there is no communication, it is possible to ask the character C to chat. Therefore, in the system 100, it is possible to naturally generate small talk, etc. As a result, in the system 100, communication between the characters C can be easily generated in the virtual space R.

[0154] In the system 100 of the present embodiment, when an operation of dragging and dropping any character C among one or more characters C or a mobile station S where the character C is located to another character C or a mobile station S where the character C is located is accepted, the mobile station S where the arbitrary character C is located and the mobile station S where the other character C is located are integrated, so that the arbitrary character C and the other character C are located on the integrated mobile station S. In this configuration, multiple characters C can be located on one mobile station S by a simple operation such as dragging and dropping. In addition, by integrating the mobile stations S, the sense of unity of the conversation is improved. Moreover, by setting a transition process such as dragging and dropping, the psychological preparation of the user operating the character C can be completed, so that it can be closer to a natural feeling.

[0155] In the system 100 of this embodiment, when the above-mentioned drag-and-drop operation is accepted, a screen for selecting whether to allow a conversation is displayed to the user of the other role C. If the user of the other role C allows the conversation, the mobile station S where the arbitrary role C is located and the mobile station S where the other role C is located are integrated, so that the arbitrary role C and the other role C are located on the integrated mobile station S. If the user of the other role C refuses the conversation, the above-mentioned drag-and-drop operation is invalidated. In this configuration, the user of the role C can select whether to have a conversation.

[0156] As mentioned above, although embodiment of this disclosure was described, this disclosure is not necessarily limited to the said embodiment, Various changes can be made within the range which does not deviate from the summary.

[0157] In the above-mentioned embodiment, a scheme in which the ear animation E is made to move in any one of the following ways: "nod", "agree", "applaud (clap)", "ask a question", "raise hand" or "surprise" is used as an example for explanation. However, the action of the ear animation E corresponding to the user's action is not limited to this. The action of the ear animation E may also be other actions, and may further include other actions. The bending angle and speed of the ear animation E may be appropriately set (changed). In this way, subtle emotions can be expressed, and the intensity of the emotion expression can also be adjusted (strengthened or weakened). In the ear animation E, when the intensity of the action is set to strong, for example, the non-verbal information expression of a person who is not good at expressing emotions can be supported through the content, intensity and speed of the action of the ear animation E. In addition, the setting of the action scheme can be changed according to the culture of the user U1, user U2, and user U3 in order to express communication between different cultures with respect to non-verbal information.

[0158] In the above embodiment, the scenario of detecting the movement of the hand H or finger F of the user U1, the user U2, and the user U3 by the operating device 50 is described as an example. However, the hand H or finger F of the user U1, the user U2, and the user U3 may also be detected by an image sensor (camera, radar, etc.).

[0159] In addition, the action of the ear animation E is basically performed based on the operation of the user U1, the user U2, and the user U3, but it can also be based on the expression of the user U1, the user U2, and the user U3 (based on the face recognition of the camera), the voice of the user U1, the user U2, and the user U3, the expression control of AI (Artificial Intelligence), etc. The action of the ear animation E can also be controlled by selecting the action of the character C in the operation window displayed on the screen.

[0160] In the above-mentioned embodiment, the operating device can be any device capable of detecting acceleration and angular velocity. The operating device may be, for example, a portable telephone (smartphone, etc.) having a display (display unit) that displays the ear animation E. In the case of a portable telephone, the motion data is generated by operating the portable telephone in the same manner as the operating device 50.

[0161] In the above embodiment, the scheme in which the control data is generated in the terminal 10, the terminal 20, and the terminal 30 is described as an example. However, the control data may be generated by the server 40. In this configuration, the terminal 10, the terminal 20, and the terminal 30 transmit the motion data to the server 40, and the server 40 generates the control data based on the motion data.

[0162] In the above embodiment, the scheme in which the body B and the ear animation E are displayed on the terminal 10, the terminal 20, and the terminal 30 is used as an example for explanation. However, when the terminal is a wearable terminal or a head mounted display (HMD) terminal, the body B and the ear animation E can also be displayed on a lens, etc., and can also be displayed in a holographic manner. That is, the character C is displayed in the cyberspace. The cyberspace can include VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality).

[0163] In the above embodiment, the scheme of using a character C with ear animation E in a virtual space R is described as an example. However, the scheme of the character is not limited to this. The character may also be an avatar. The avatar may include or not include facial expressions. The avatar may also have ear animation E.

[0164] The character C may also be transformed. For example, a display unit may be displayed on the body B of the character C. A conversation text converted from the voice of the character C, a translation of the conversation text, and a translation of the display content of the display unit may be displayed on the display unit. In addition, at least one of a desired picture, article, music, animation, still image, and website may be displayed on the display unit by using AI based on at least one of the voice and text input of the character C.

[0165] In the above embodiment, the character C is described as an example in which the character C is on the moving platform S. However, the object forming the area where the character C is located is not limited to the moving platform S. For example, the object forming the area where the character C is located may be a simple round plate, a flying carpet, a spaceship, an airship, a large bus, or a flying classroom.

[0166] Furthermore, the object may be a band-shaped object surrounding the character C. In this case, the character C being located in the object means that the character C is located in the band-shaped object.

[0167] In the above embodiment, the scenario in which the character C performs actions by operating the operating device 50 is described as an example. However, the actions of the character C may also be operated by other methods. For example, the status of the user U1, the user U2, and the user U3 may be obtained, and the actions of the character C may be controlled based on the status.

[0168] In the above-mentioned embodiment, the operation of establishing association between role C1 and role C2 is an operation of dragging and dropping role C1 to the mobile station S of role C2. However, the operation of establishing association between roles C is not limited to dragging and dropping. For example, in the case of establishing association between role C1 and role C2, the operation of establishing association may be an operation in which the user of role C1 selects (clicks, etc.) role C2, or an operation in which the user of role C1 selects role C2 in a prescribed selection screen, or an operation in which the user of role C1 inputs information to identify role C2. In short, the operation of establishing association can be any operation in which the user of role C1 can identify role C2 as the conversation partner (object).

[0169] In the above embodiment, the following scheme is used as an example for explanation: in the consent process, when a message of consent is sent to the character C as the conversation partner, the message is displayed on the screen of the user of the character C, and the user can select whether to allow the conversation on the message screen. However, the selection of whether to allow the conversation can also be made by the user's voice ("allow", "OK", "do not allow", "NG", etc.).

[0170] In the above embodiment, the operation of ending the conversation between the characters C is an operation of dragging and dropping the character C to the outside of the mobile station S. However, the operation of ending the conversation is not limited to dragging and dropping. For example, the operation of ending the conversation may be an operation of selecting an item to end the conversation in a predetermined screen (a menu screen, etc.), or an operation of clicking an "×" mark displayed on a part of the character C. In short, the operation of ending the conversation can be any operation in which the user of the character who wants to end the conversation indicates the intention to end the conversation.

[0171] In the above embodiment, the cover CV is described as an example in which the cover CV is hemispherical. However, the shape of the cover CV is not limited thereto. In addition, as information related to participating in a session at the mobile station S, the cover CV is described as an example. However, the information related to participating in a session at the mobile station S is not limited thereto.

[0172] For example, information related to participating in a session can also be Fig.17 The cloud CL is shown above the character C in mode A. Fig.17 As shown in the mode B of FIG. 1 , a wall-shaped member WL is arranged on the periphery of the mobile station S. The member WL may be transparent or opaque. In addition, information related to the participation in the session may also be displayed as shown in FIG. Fig.18 The pattern A is shown as an object such as a plant composed of one or more trees TR. In addition, the information related to the participation in the conversation may be an animal (bird, etc.) displayed above the character C, an object such as a curtain, or an object such as fog.

[0173] You can also Fig.18 As shown in mode B, information indicating a specific group such as "XYZ" is displayed on the mobile station S. In this case, the display of "XYZ" can be used to indicate to the outside that participation of users other than users related to "XYZ" is rejected (users other than users related to "XYZ" are not allowed to participate). In addition, it can also be set that in the case of a user related to "XYZ", that is, in the case where information related to "XYZ" is set in the user's attribute information, participation in the session is allowed. In other words, it can also be set that in the case where information related to "XYZ" is not set in the user's attribute information, participation in the session is rejected. It can also be set that even if information related to "XYZ" is not set in the user's attribute information, participation in the session is allowed if allowed by the role C located in the mobile station S.

[0174] In the above embodiment, the following scheme is used as an example for explanation: when an operation of dragging and dropping an arbitrary character C or a mobile station S where the character C is located to another character C or a mobile station S where the character C is located is accepted, the mobile station S where the arbitrary character C is located and the mobile station S where the other character C is located are integrated, and the arbitrary character C and the other character C are located on the integrated mobile station S. However, the process of placing the arbitrary character C and the other character C on one mobile station S may also be performed automatically.

[0175] Reference Fig.19 , an example of a method for role C to automatically start a conversation is described. Fig.19 In the example shown, in the virtual space R, characters C1, C2, C3, C4, C5, and C6 are respectively on the mobile station S. The users of characters C1, C2, C3, C4, C5, and C6 may be users U1, U2, and U3. At least one of characters C1, C2, C3, C4, C5, and C6 may also be an AI user.

[0176] The virtual space R is used for gatherings (conferences, training sessions, exchanges, parties, etc.) where multiple characters gather to have conversations. The conversations between characters in the virtual space R are managed by an administrator. The administrator can set the time of the gathering, the duration of a conversation, the number of people in the conversation, etc. The duration of a conversation can be set to, for example, 3 to 4 minutes. There is no limit on the number of people in the conversation, and it can be set to, for example, two people.

[0177] In the virtual space R, for example, there may be characters C1, C2, C3, C4, C5, and C6 that are allowed to participate by the administrator. For example, the users of characters C1 to C6 apply to the administrator for participation in advance, and when the administrator allows participation, the users receive a URL (Uniform Resource Locator) related to participation from the administrator. The users of characters C1 to C6 can enter the virtual space R by participating from the URL at a specified date and time.

[0178] It can also be, Fig.19 In the situation shown (before the conversation starts), the administrator explains how to proceed with the conversation. For example, a summary of the conversation time is given. In addition, the administrator can explain how to proceed with the conversation. For example, the following method of conducting a conversation can be explained: first introduce yourself by name, etc., and then talk about the topic. In addition, the method of listening to the conversation during the conversation (for example, listening while repeating the words, nodding, expressing surprise or appreciation, etc.) can be explained. While the administrator is explaining, questions from users of roles C1 to C6 can also be accepted.

[0179] When an administrator starts a session, such as Fig. 20 As shown, the two characters are automatically located at a mobile station S. That is, the two characters are automatically matched. Fig. 20 In the example shown, roles C1 and C3 are located at the same mobile station S, roles C2 and C5 are located at the same mobile station S, and roles C4 and C6 are located at the same mobile station S. The roles located at the mobile station S may be selected by an administrator or randomly selected. Preferably, the roles located at the mobile station S are set so as not to match roles that have been matched in the past. That is, preferably, the matching partner is set to a different role each time.

[0180] A character C7 may also exist in the virtual space R. The character C7 is operated by, for example, a facilitator user. The user of the character C7 may also be a manager.

[0181] The characters outside the mobile station S cannot hear the conversation between characters C1 and C3, the conversation between characters C2 and C5, and the conversation between characters C4 and C6 at the mobile station S. The broadcast of character C7 can be heard at each mobile station S. It is also possible that the user of character C7 can hear the conversations at all mobile stations S. In other words, it is also possible to set that the user of character C7 can monitor all mobile stations S. In addition, it is also possible to set that the user of character C7 can record (record) the conversations at all mobile stations S.

[0182] like Fig. 20 As shown, a projection board (display unit) PB may be provided on the mobile station S. The topic of the conversation (information related to the content of the conversation) and the like may also be displayed on the projection board PB. For example, favorite foods, favorite places, things you want to do, favorite activities (interests), things you want, and the like may also be displayed on the projection board PB. In addition, the translation of the characters' conversations may also be displayed on the projection board PB. It should be noted that, in the case where the projection board PB is not provided, the topic of the conversation and the like may also be broadcast. In addition, a clock, a timer, and the like may also be displayed on the mobile station S. In the case of a timer, the remaining time of the conversation is displayed. In addition, it may also be configured such that when the remaining time reaches a specified time (for example, 10 seconds ago), an alarm sounds.

[0183] like Fig.21 As shown, for example, only the mobile stations S where the roles C1 and C3 are located can be displayed on the display screens of the user of role C1 and the user of role C3. The user can switch the display of the screen. By performing a predetermined operation, the user can Fig. 20 As shown in FIG. 1 , a screen (entire screen) showing all characters C1 to C7 (mobile station S) can also be displayed as shown in FIG. Fig.21 As shown, a screen (single screen) showing only the character of the player and the opponent is displayed.

[0184] Roles C1 to C6 can pass on the following information to the other party in the conversation: Fig. 22 The user of roles C1 to C6 can pass the card CD to the conversation partner by performing a predetermined operation (for example, selecting a card sending item in the operation screen). The card CD is prepared in advance by the user of roles C1 to C6.

[0185] The card CD contains at least information that can identify the user of the role. For example, the card CD displays the role. For example, text can be displayed at the role. The name, nickname, identification information, etc. can be displayed as text. A facial photo, etc. can also be displayed at the role. In addition, an image P can be displayed in the display area DA of the card CD. Messages can also be displayed in the display area DA, or music can be embedded, or images can be displayed. Received card CDs are automatically saved. The saved card CDs can be viewed. It can also be set that by clicking on the card CD, the user of the card CD can be contacted (a message creation screen is displayed).

[0186] When Fig. 20 When the specified time has passed since the start of the session, the system will automatically return to the current state. Fig.19 The state shown once, then, as Fig.23 As shown, the conversation partner is automatically changed. That is, in the state where multiple roles are located on one mobile station S ( Fig. 20 When a predetermined time has passed in the state shown in FIG. 1 , the mobile station S is redisplayed, and the character who has ended the conversation is located at the redisplayed mobile station S ( Fig.19 After that, as shown in Fig.23 As shown, the two characters are automatically placed on a mobile station S again.

[0187] exist Fig.23 In the example shown, roles C3 and C5 are located at the same mobile station S, roles C1 and C6 are located at the same mobile station S, and roles C2 and C4 are located at the same mobile station S. That is, the characters are matched with different characters from the previous time.

[0188] When Fig.23 When the specified time has passed since the start of the session in the state shown, the session will automatically return to Fig.19 The state shown is changed once, and then the conversation partner is automatically changed. This process is performed multiple times within the time set by the administrator (for example, 30 minutes). When the time set by the administrator is reached, the state returns to Fig.19 Status shown.

[0189] In the above configuration, the mobile station S where any character is located and the mobile station S where other characters are located are automatically integrated, so that any character and other characters are located on the integrated mobile station S. In this configuration, multiple characters can be automatically located on one mobile station S. That is, the characters can be automatically matched. Thus, not only communication with a specific character (user) can be sought, but also communication with a new character can be sought. Therefore, by contacting a new personality of a user with whom there has been no communication before, the sense of intimacy can be improved. Therefore, subsequent communication can be easily achieved, psychological safety can be improved, and improvement in creativity, productivity, etc. can be expected.

[0190] Furthermore, when a predetermined time has passed while a plurality of characters are located at one mobile station S, the mobile station S is redisplayed, and one or more characters that have ended the conversation are located at the redisplayed mobile station S. In this configuration, when the predetermined time has passed, the conversation is forcibly ended. In this way, the communication time can be limited, so that the communication with the same character will not be prolonged. Therefore, communication with many characters can be realized.

[0191] In addition, a projection board PB is displayed on a mobile station S where a plurality of characters are located, and the projection board PB displays information related to the content of the conversation. In this configuration, the topic of the conversation can be displayed (provided) on the projection board PB, so that communication can be carried out smoothly and efficiently within a limited time. Therefore, a conversation that touches the personality of the user of the character can be carried out efficiently.

[0192] In addition, information related to the predetermined time may be displayed on one mobile station S where a plurality of characters are located. In this configuration, for example, the remaining time of the predetermined time can be confirmed, so that communication can be performed in consideration of time allocation (end time).

[0193] Furthermore, a character operated by a host user may be displayed in the virtual space R. In this configuration, when applied to a gathering attended by many users (e.g., a training session, an information exchange meeting, a party, etc.), the gathering can be promoted by the host, so that the gathering can be promoted smoothly.

[0194] In addition, the card CD containing at least information that can identify the user of the character can be exchanged when multiple characters are located on one mobile station S. In this configuration, the character (user) who conducted the conversation can be confirmed after the conversation ends.

[0195] It should be noted that, in the above scheme, two roles are located in one mobile station S as an example, but there may be three or more roles located in one mobile station S. The number of roles may be appropriately set by the administrator.

[0196] Description of Reference Numerals

[0197] 10: terminal; 11: input unit; 12: control unit; 13: storage unit; 13P: terminal program; 14: communication unit; 15: output unit; 20: terminal; 30: terminal; 40: server; 42: control unit; 43: storage unit; 43P: server program; 43S: receiving data; 44: communication unit; 50: operating device; 51A: operating surface; 52: communication unit; 53: input unit; 54: control unit; 55: battery; 100: system; AP: access point; B: body; BS : Base station; C: role; C1: role (any role); C2: role (other role); C3: role; C4: role; C5: role; C6: role; C7: role; CD: card; CL: cloud; CV: cover; DA: display area; E: ear animation; F: finger; H: hand; N: network; P: image; PB: projection board; R: virtual space; S: mobile station, first mobile station, second mobile station; TR: tree; U1: user; U2: user; U3: user; WL: component.

Claims

1. A communication support program that supports communication between multiple characters in a virtual space, wherein: The communication support program is configured to: causing the object where one or more of the characters are located to be displayed together with the one or more of the characters, It is possible for the plurality of characters located on one of the objects to have conversations with each other.

2. The communication support program according to claim 1, wherein: The communication support program is configured to: When an operation of associating any one of the one or more characters with another character different from the character is accepted, the object where the any character is located and the object where the other character is located are integrated so that the any character and the other character are located in the integrated object.

3. The communication support program according to claim 2, wherein: The communication support program is configured to: When the operation of establishing association is accepted, the user of the other role is allowed to select whether to allow the session, When the user of the other character permits the conversation, the object where any character is located is integrated with the object where the other character is located, so that any character and the other character are located in the integrated object, If the user of the other role refuses the session, the operation of establishing the association is invalidated.

4. The communication support program according to claim 2 or 3, wherein: The operation of establishing association is an operation of dragging and dropping any of the characters or the object where the character is located to other characters or the object where the character is located.

5. The communication support program according to any one of claims 2 to 4, wherein: The communication support program is configured to: When an operation of ending a conversation of one or more of the characters is accepted while a plurality of the characters are positioned on one of the objects, the object is redisplayed and the one or more characters whose conversations have ended are positioned on the redisplayed object.

6. The communication support program according to claim 5, wherein: The operation of ending the session is an operation of dragging and dropping one or more of the characters from the object to the outside.

7. The communication support program according to any one of claims 1 to 6, wherein: The object can move in the virtual space.

8. The communication support program according to any one of claims 1 to 7, wherein: The communication support program is configured to: The character located at the object is enabled to communicate with the character not located at the object.

9. The communication support program according to any one of claims 1 to 8, wherein: The communication support program is configured to: Information related to participating in the session is displayed on the object.

10. The communication support program according to any one of claims 1 to 9, wherein: The communication support program is configured to: When one or more of the characters are located at one of the objects, the characters are automatically moved to a predetermined location when a predetermined time comes or a predetermined operation is received.

11. The communication support program according to any one of claims 1 to 10, wherein: At least one of the plurality of characters located at one of the objects is operated by AI.

12. The communication support program according to claim 1, wherein: The communication support program is configured to: The object where any one of the one or more characters is located is automatically integrated with the object where other characters different from the character are located, so that any character and other characters are located in the integrated object.

13. The communication support program according to claim 12, wherein: The communication support program is configured to: When a predetermined time has elapsed while a plurality of the characters are positioned on one object, the object is redisplayed, and one or more of the characters that have completed the conversation are positioned on the redisplayed object.

14. The communication support program according to claim 12 or 13, wherein: The communication support program is configured to: The display unit displays information related to the content of the conversation on one of the objects where the plurality of characters are located.

15. The communication support program according to claim 13, wherein: The communication support program is configured to: Information related to the specified time is displayed on one of the objects where the plurality of characters are located.

16. The communication support program according to any one of claims 12 to 14, wherein: The communication support program is configured to: The character operated by the host user is displayed in the virtual space.

17. The communication support program according to any one of claims 12 to 16, wherein: The communication support program is configured to: When a plurality of the characters are located on one of the objects, cards can be exchanged, wherein the cards at least contain information that can identify the users of the characters.

18. The communication support program according to any one of claims 12 to 17, wherein: At least one of the plurality of characters located at one of the objects is operated by AI.

19. A communication support method for supporting communication between multiple characters in a virtual space, wherein: causing the object where one or more of the characters are located to be displayed together with the one or more of the characters, It is possible for the plurality of characters located on one of the objects to have conversations with each other.

20. A communication support system, supporting communication between multiple characters in a virtual space, wherein: The communication support system is configured to: The object where one or more of the characters are located is displayed together with the one or more of the characters, so that a conversation between the characters located on one of the objects can be carried out.

Images