COMMUNICATION SUPPORT PROGRAM, COMMUNICATION SUPPORT METHOD, AND COMMUNICATION SUPPORT SYSTEM

Abstract

The communication support program supports communication between multiple characters in a virtual space by displaying an object in which one or more characters are located integrally with the one or more characters, and enabling multiple characters located on a single object to converse with each other.

Description

[Technical field]

[0001] One aspect of the present disclosure relates to a communication assistance program, a communication assistance method, and a communication assistance system. This application claims priority to Japanese Application No. 2023-081344, filed on May 17, 2023, and incorporates by reference all of the contents of said Japanese application. [Background technology]

[0002] In recent years, Teams (registered trademark), Zoom (registered trademark), and the like have become mainstream for remote communication. In these remote communications, participants are set and invited in advance, and when a predetermined time arrives, the participants access the web or the like to carry out communication such as a conference. Non-Patent Document 1 discloses Teams as an example of remote communication. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Microsoft Teams”, [online], Microsoft, [searched May 10, 2023], Internet,<URL:https: / / www.microsoft.com / ja-jp / microsoft-teams / group-chat-software> Summary of the Invention

[0004] A communication support program according to one embodiment of the present disclosure is a communication support program that supports communication between multiple characters in a virtual space, by displaying an object in which one or more characters are located integrally with the one or more characters, and enabling multiple characters located on a single object to converse with each other. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a display system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of the terminal and the server. [Diagram 3] FIG. 3 is a perspective view of the operating device. [Figure 4] FIG. 4 is a diagram illustrating an example of functional blocks of the operation device illustrated in FIG. [Diagram 5] FIG. 5 is a diagram showing an example of the operation of ear animation. [Figure 6] FIG. 6 is a diagram showing an example of a screen displayed in the virtual space. [Figure 7] FIG. 7 is a diagram showing an example of a screen displayed in the virtual space. [Figure 8] FIG. 8 is a diagram showing an example of a screen displayed in the virtual space. [Figure 9] FIG. 9 is a diagram showing an example of a screen displayed in the virtual space. [Figure 10] FIG. 10 is a diagram showing an example of a screen displayed in the virtual space. [Figure 11] FIG. 11 is a diagram showing an example of a screen displayed in the virtual space. [Figure 12] FIG. 12 is a diagram showing an example of a screen displayed in the virtual space. [Figure 13] FIG. 13 is a diagram showing an example of a screen displayed in the virtual space. [Figure 14] FIG. 14 is a diagram showing an example of a screen displayed in the virtual space. [Figure 15] FIG. 15 is a diagram showing an example of a screen displayed in the virtual space. [Figure 16] FIG. 16 is a diagram showing an example of a screen displayed in the virtual space. [Figure 17] FIG. 17 is a diagram showing an example of a screen displayed in the virtual space. [Figure 18] FIG. 18 is a diagram showing an example of a screen displayed in the virtual space. [Figure 19] FIG. 19 is a diagram showing an example of a screen displayed in the virtual space. [Figure 20] FIG. 20 is a diagram showing an example of a screen displayed in the virtual space. [Figure 21] FIG. 21 is a diagram showing an example of a screen displayed in the virtual space. [Figure 22] FIG. 22 is a diagram showing an example of a screen displayed in the virtual space. [Figure 23] FIG. 23 is a diagram showing an example of a screen displayed in the virtual space. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] [Problem that this disclosure aims to solve] In the above remote communication, although communication such as conversation can be performed with members set as participants, conversations without specifying the communication partner in advance, so-called small talk, are difficult to occur accidentally or spontaneously. For example, in places where people actually gather (party venues, smoking rooms, free spaces, tea rooms (kitchen rooms), break rooms, dining rooms, corridors, etc.), small talk naturally occurs among people in the place. In places where people actually gather, conversation participants look at each other's faces and nod or respond, so by observing these subtle changes in facial expression, the conversation situation of the conversation participants can be grasped and the conversation group can be recognized. Therefore, it is possible to look at the conversation participants' behavior and call out to them at the right time, etc., so small talk naturally occurs.

[0007] In a virtual space such as the metaverse, which is one type of remote communication, characters (avatars) can move freely within the virtual space and communicate with other characters. This allows characters to chat with each other. However, it is not easy to chat within the metaverse. This is because the metaverse does not accurately express non-verbal information such as the gaze of characters and conversational reactions, making it difficult to grasp the situation between characters and to recognize conversation groups, such as which characters are talking.

[0008] The present disclosure aims to provide a communication support program, a communication support method, and a communication support system that can easily generate communication between characters in a virtual space.

[0009] [Effects of this disclosure] According to the present disclosure, communication between characters can be easily generated in a virtual space.

[0010] [Description of the embodiments of the present disclosure] First, the contents of the embodiments of the present disclosure will be listed and described. At least a part of the embodiments described below may be arbitrarily combined.

[0011] (1) A communication assistance program according to one aspect of the present disclosure is a communication assistance program that assists communication between multiple characters in a virtual space, by displaying an object in which one or more characters are located integrally with the one or more characters, and enabling multiple characters located on a single object to converse with each other.

[0012] In the communication assistance program according to one aspect of the present disclosure, when a plurality of characters are located in one object, the characters can have a conversation with each other. That is, a conversation is established between the characters located in one object. Therefore, it is possible to grasp at a glance which characters are conversing with each other. In other words, it is possible to grasp at a glance which characters are not conversing. In this way, the communication assistance program can visualize the state of communication. Therefore, in the communication assistance program, for example, when communication is not taking place, it is possible to propose small talk to the characters. Therefore, in the communication assistance program, small talk and the like can be naturally generated. As a result, in the communication assistance program, communication between characters can be easily generated in the virtual space.

[0013] (2) In the communication support program of (1) above, when an operation to associate any one of one or more characters with another character different from the one character is received, the object in which the any one character was located may be integrated with the object in which the other character was located, and the any one character and the other character may be positioned in a single integrated object. In this configuration, the operation to associate the any one character with the other character may cause multiple characters to be positioned in a single object. Furthermore, by integrating the objects, a sense of unity in the conversation is improved.

[0014] (3) In the communication support program of (2) above, when the associating operation is accepted, the user of the other character may be allowed to choose whether or not to allow the conversation, and if the user of the other character allows the conversation, the object in which the given character was located may be integrated with the object in which the other character was located, and the given character and the other character may be positioned in a single integrated object, and if the user of the other character rejects the conversation, the associating operation may be invalidated. In this configuration, the user of the character may choose whether or not to allow the conversation.

[0015] (4) In the communication support program of (2) or (3) above, the associating operation may be an operation of dragging and dropping an arbitrary character or an object on which the character is located onto another character or an object on which the character is located. In this configuration, multiple characters can be placed on one object by a simple operation of dragging and dropping. In addition, by providing a joining transition process of dragging and dropping, the user who operates the character can prepare mentally, making it possible to feel closer to a natural sensation.

[0016] (5) In the communication assistance program of any one of (2) to (4) above, when a plurality of characters are located on one object and an operation to end a conversation of one or more characters is received, the object may be newly displayed and the one or more characters that have ended the conversation may be positioned on the newly displayed object. In this configuration, when the conversation ends, the character that has ended the conversation may be positioned on the object again.

[0017] (6) In the communication support program of (5) above, the operation for ending the conversation may be an operation for dragging and dropping one or more characters from the object to the outside. In this configuration, the conversation can be ended by a simple operation of dragging and dropping.

[0018] (7) In the communication assistance program according to any one of (1) to (6) above, the object may be movable in the virtual space. With this configuration, a character located at the object can be moved in the virtual space.

[0019] (8) In the communication assistance program according to any one of (1) to (7) above, a character located on an object may be allowed to communicate with a character not located on the object. In this configuration, for example, the character located on the object may be asked in advance whether or not he / she can participate in the conversation.

[0020] (9) In the communication assistance program according to any one of (1) to (8), information regarding participation in a conversation may be displayed in the object. With this configuration, it is possible to know at a glance whether the object can participate in a conversation, the degree of participation in the conversation, etc.

[0021] (10) In the communication assistance program according to any one of (1) to (9) above, when one or more characters are located in one object, the characters may be automatically moved to a predetermined location when a predetermined time comes or a predetermined operation is received. In this configuration, a conversation in one object may be ended and the characters may be automatically assembled in a predetermined location (such as a meeting place).

[0022] (11) In the communication assistance program according to any one of (1) to (10) above, at least one of a plurality of characters located on one object may be operated by an AI.

[0023] (12) In the communication assistance program of (1) above, an object in which any one or more characters have been located may be automatically integrated with an object in which another character different from the one or more characters have been located, and the any one character and the other character may be positioned in a single integrated object. In this configuration, multiple characters can be automatically positioned in a single object. In other words, the characters can be automatically matched. This allows communication not only with a specific character (user) but also with new characters.

[0024] (13) In the communication assistance program of (12) above, when a predetermined time has elapsed in a state in which multiple characters are located on one object, the object may be newly displayed and one or multiple characters who have finished their conversation may be positioned on the newly displayed object. In this configuration, the conversation ends when the predetermined time has elapsed. In this way, the communication time is limited, so communication with the same character does not drag on. This makes it possible to communicate with many characters.

[0025] (14) In the communication assistance program of (12) or (13) above, a display unit that displays information related to the content of the conversation may be displayed on an object in which multiple characters are located. In this configuration, the display unit displays (provides) information related to the content of the conversation, such as a topic of conversation, which allows smooth communication.

[0026] (15) In the communication assistance program of (13) above, information related to a predetermined time may be displayed on one object where multiple characters are located. In this configuration, for example, the remaining time of the predetermined time can be confirmed, so that communication can be carried out while taking into consideration time allocation (end time).

[0027] (16) In any one of the communication support programs (12) to (15) above, a character operated by a user who is a facilitator may be displayed in a virtual space. In this configuration, when applied to a meeting in which many users participate (for example, a training session, an information exchange meeting, a social gathering, etc.), the meeting can be smoothly conducted because the meeting is conducted by the facilitator.

[0028] (17) In the communication assistance program according to any one of (12) to (16) above, when a plurality of characters are located in one object, cards including at least information that identifies the user of the characters may be exchanged. In this configuration, after the conversation ends, the character (user) with whom the conversation took place may be identified.

[0029] (18) In the communication assistance program according to any one of (12) to (17) above, at least one of a plurality of characters located on one object may be operated by an AI.

[0030] (19) A communication support method according to one aspect of the present disclosure is a communication support method for supporting communication between multiple characters in a virtual space, comprising: displaying an object in which one or more characters are located integrally with the one or more characters; and enabling multiple characters located on a single object to converse with each other.

[0031] A communication assistance method according to an aspect of the present disclosure includes the same features as the above-described communication assistance program, and therefore can achieve the same effects and advantages as the above-described communication assistance program.

[0032] (20) A communication support system according to one aspect of the present disclosure is a communication support method for supporting communication between multiple characters in a virtual space, comprising: displaying an object in which one or more characters are located integrally with the one or more characters; and enabling multiple characters located on a single object to converse with each other.

[0033] The communication assistance system according to one aspect of the present disclosure includes the same features as the above-described communication assistance program, and can therefore achieve the same effects and advantages as the above-described communication assistance program.

[0034] [Details of the embodiment of the present disclosure] Specific examples of the embodiments of the present disclosure will be described below with reference to the drawings. The present disclosure is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope of the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted.

[0035] Fig. 1 is a diagram showing an example of a schematic configuration of a communication support system (hereinafter, sometimes simply referred to as "system"). The system 100 is a computer system that supports communication in a virtual space R (see Fig. 6) using a character C (see pattern A in Fig. 5) that is operable by a user and whose ear animation E (see pattern A in Fig. 5) moves in response to the user's operation.

[0036] The character C may be operated by human users U1, U2, and U3, or may be operated by a non-human. The character C may be operated by, for example, AI (Chat GPT, Siri (registered trademark), Google Assistant (registered trademark), Alexa (registered trademark), Cortana (registered trademark)). In other words, the user who operates the character C may be AI. The user means an operator who operates the character C, etc.

[0037] Character C represents the user. Character C is a dynamic character. Character C is generated by image material independent of the user's actual image (image data, etc.). Image material is a material for generating an animated character. Character C may be rendered by two-dimensional or three-dimensional computer graphics (CG).

[0038] As an example, character C is composed of body B (see pattern A in FIG. 5) and ear animation E. In character C, body B and ear animation E are displayed as an integrated unit. Displayed as an integrated unit means that when looking at character C, body B and ear animation E are visually recognized as one unit. In other words, displayed as an integrated unit means that it is possible to recognize at a glance that body B and ear animation E are connected (corresponding). Therefore, displayed as an integrated unit may mean that body B and ear animation E are in contact with each other, or may mean that body B and ear animation E are separated from each other.

[0039] Body B may include at least one of an image representing the user and text representing the user. A shadow may be displayed on character C. Body B may be spherical and may include text that identifies the user (such as "A", "B", "C", etc.). Character C may be configured to include a speaking animation (not shown) or may be configured to include animation that resembles arms, legs, etc.

[0040] The ear animation E is an animation that expresses non-verbal information. The ear animation E is an animation that shows the user's state (movement) and emotions. The ear animation E is an animation that expresses the user's state in a form different from the user's actual moving parts. The ear animation E may not include a dynamically changing facial expression, or may include a dynamically changing facial expression. The ear animation E is displayed outside the display area of ​​the body B so as to protrude from the display area. The ear animation E is an animation that imitates the ears of an animal (e.g., a rabbit, etc.), and moves in response to changes in the user's state. The ear animation E moves in response to the user's movement. The ear animation E moves, for example, by stretching, bending, swinging, etc. The ear animation E is freely deformable, and visually deforms in response to the user's state so as to evoke the user's state.

[0041] In the system 100, the ear animation E of the character C allows the user to check the user's state, emotions, reactions, etc.

[0042] <System configuration> The system 100 includes a terminal 10, a terminal 20, and a terminal 30. In the example shown in FIG. 1, the terminal 10 is a laptop personal computer (PC) used by a user U1. The terminal 20 is a mobile phone (smartphone, etc.) used by a user U2. The terminal 30 is a desktop PC used by a user U3. The terminals 10, 20, and 30 can be connected to a network N. In this example, the terminal 10 is connected to the network N via an access point AP, the terminal 20 is connected to the network N via a base station BS, and the terminal 30 is connected to the network N by wire (not shown). The terminals 10, 20, and 30 may be tablet terminals, wearable terminals, head-mounted display (HMD) terminals, and various other terminals in addition to the above-mentioned PCs and mobile phones. The number of terminals included in the system 100 is not limited to the example of FIG. 1.

[0043] The system 100 includes a server 40. The server 40 is connected to a network N. Therefore, the terminals 10, 20, 30, and the server 40 can communicate with each other via the network N. Note that the number of servers included in the system 100 is not limited to the example in FIG.

[0044] There are no limitations on the configuration of the network N. For example, the network N may be configured to include the Internet (public network), a communication carrier network, a business network, and the like.

[0045] 2 is a diagram showing examples of functional blocks of the terminal 10, the terminal 20, the terminal 30, and the server 40. Since the functional blocks of the terminal 20 and the terminal 30 are similar to the functional block of the terminal 10, the following describes the terminal 10 and the server 40. First, the terminal 10 will be described, and then the server 40 will be described.

[0046] <Device configuration> The terminal 10 includes an input unit 11. The input unit 11 is a portion to which voice data and image data of a user U1 are input. The input unit 11 may also receive video data of the user U1. The video includes voice data and image data of the user U1.

[0047] The terminal 10 includes a control unit 12. The control unit 12 is a part that generates control data for the user U1. The control data for the user U1 is generated based on motion data output from the operation device 50. In this embodiment, for example, the control data may be data that causes the ear animation E of the character C to perform six motions, namely, "nodding," "agreement," "praise (clap)," "question," "raise hand," and "surprise." The control unit 12 may also generate control data for other motions (for example, motions indicating "YES" and "NO").

[0048] The control data is associated with the action data. In this embodiment, for example, the action data is associated with each of the control data of "nodding," "agreement," "praise (applause)," "question," "raising hand," and "surprise." The action data is data transmitted from the operation device 50. When the action data corresponds to the control data, the control unit 12 generates the control data.

[0049] The control data may be generated based on the voice data and image data of the user U1 input to the input unit 11. The image data is composed of a plurality of successive unit frames. A unit frame usually contains image information of the user U1. For example, image data of 60 frames / second contains 60 unit frames per second. The generation of the control data by the control unit 12 will be described in detail later.

[0050] The control unit 12 also controls the movement of the character C based on the control data. The control unit 12 displays the character C in the virtual space R and controls the movement of the ear animation E of the character C based on the generated control data and the received control data.

[0051] The control unit 12 generates video data of the virtual space R. The video data of the virtual space R includes video data of the character C controlled based on the control data. In addition, various objects appearing in the virtual space R may be included in the video data of the virtual space R. The method of generating the virtual space R by the control unit 12 is not particularly limited, and various known methods may be used. For example, the control unit 12 may generate the video data of the virtual space R based on an image captured by a camera. Alternatively, the control unit 12 may generate the video data of the virtual space R based on three-dimensional CAD data.

[0052] The terminal 10 includes a storage unit 13. The storage unit 13 is a part that stores information necessary for control (processing) of the terminal 10. In Fig. 2, a terminal program 13P is exemplified as information stored in the storage unit 13. The terminal program 13P is a program for operating the terminal 10, and operates a computer so that control by the control unit 12 and the like are executed. Since the system 100 is a communication support system, the terminal program 13P can also be said to be a communication support program.

[0053] The terminal 10 includes a communication unit 14. The communication unit 14 is a part that communicates with devices external to the terminal 10 via a network N (see FIG. 1 ). Examples of devices external to the terminal 10 are the terminal 20, the terminal 30, the server 40, and the operation device 50. The communication unit 14 transmits voice data and control data generated by the control unit 12 to the server 40, and receives voice data and control data from the server 40. The communication unit 14 is a part that communicates with the operation device 50. The communication unit 14 receives operation data from the operation device 50.

[0054] The terminal 10 includes an output unit 15. The output unit 15 outputs an image of the virtual space R. The image of the virtual space R is an image based on image data of the virtual space R generated by the control unit 12.

[0055] <Server configuration> The server 40 includes a control unit 42. The control unit 42 controls the operation of the server 40. The server 40 includes a storage unit 43. The storage unit 43 is a unit that stores information necessary for controlling the server 40. In FIG. 3, a server program 43P and received data 43S are exemplified as information stored in the storage unit 43. The server program 43P is a program for operating the server 40, and operates a computer so that control by the control unit 42 is executed. The received data 43S is data sent from the terminals 10, 20, and 30 to the server 40, and may include control data generated by the terminals 10, 20, and 30.

[0056] The server 40 includes a communication unit 44. The communication unit 44 is a part that communicates with devices external to the server 40 via a network N (see FIG. 1). Examples of devices external to the server 40 are the terminals 10, 20, and 30. The communication unit 44 receives control data of the user U1, the control data of the user U2, and the control data of the user U3 from the terminals 10, 20, and 30, respectively, and transmits control data generated by the terminals 10, 20, and 30 to the terminals 10, 20, and 30, respectively.

[0057] <Hardware configuration> An example of a hardware configuration of each functional part of the terminal 10 and the server 40 will be described. The communication unit 44 of the server 40 may be configured using a network card or a wireless communication device so as to access the network N. The control unit 42 of the server 40 may be configured using a processor such as a central processing unit (CPU) or a graphics processing unit (GPU), a clock, and an internal memory. The control unit 42 may be configured as a single piece of hardware (SoC: System On a Chip) that integrates the processor, the clock, the internal memory, the storage unit 43, and the communication unit 44. The control unit 42 operates based on a server program 43P to operate the server computer as the server 40. The storage unit 43 of the server 40 may be configured using a non-volatile storage medium such as a flash memory, a hard disk, or an SSD (Solid State Disk).

[0058] The input unit 11 of the terminal 10 may be configured using a microphone, a camera, and the like. The microphone acquires the voice of the user U1. That is, the microphone is a type of audio sensor that acquires the voice. The camera acquires an image of the user U1. That is, the camera is a type of image sensor that acquires the image. Furthermore, the input unit 11 may also be configured using an operation device such as a keyboard, a mouse, or a touch panel. The control unit 12 of the terminal 10 may be configured similarly to the control unit 42 of the server 40. The control unit 12 operates based on the terminal program 13P to operate a general-purpose computer as the terminal 10. The storage unit 13 of the terminal 10 may be configured similarly to the storage unit 43 of the server 40. The communication unit 14 of the terminal 10 may be configured similarly to the communication unit 44 of the server 40. The output unit 15 of the terminal 10 may be configured using a display device such as a liquid crystal panel, an organic EL panel, or the like (which may be a touch panel). Furthermore, the output unit 15 may also be configured using a speaker.

[0059] <Configuration of the operation device> Fig. 3 is a perspective view of the operation device 50. Fig. 3 is a diagram showing a state in which the operation device 50 is operated by the fingers F of the users U1, U2, and U3. The operation device 50 is a device operated by the users U1, U2, and U3. An ear animation E is activated by the operation of the operation device 50 by the users U1, U2, and U3. The operation device 50 is a touch pad. The touch pad may be of a capacitive type, a pressure-sensitive touch type, an ultrasonic type, or the like.

[0060] The operation device 50 includes a main body 51 as a hardware configuration. The main body 51 has an operation surface 51A. The operation device 50 is operated by the fingers F of the hands H of the users U1, U2, and U3. The operation device 50 may be provided as a standalone device, or may be provided as part of a remote control for operating a television display or the like. In other words, the operation device 50 may be part of a device.

[0061] Fig. 4 is a diagram showing an example of functional blocks of the operation device 50. As shown in Fig. 4, the operation device 50 includes a communication unit 52. The communication unit 52 is a part that communicates with an external device by wireless communication or wired communication. Examples of wireless communication include LAN, Bluetooth (registered trademark), Wifi, etc. Examples of the external device are the terminal 10, the terminal 20, and the terminal 30. The communication unit 52 transmits operation data output from the control unit 54 to the terminal 10, the terminal 20, and the terminal 30.

[0062] The operation device 50 includes an input unit 53. The input unit 53 is a part that inputs operations of users U1, U2, and U3 on the operation surface 51A. The input unit 53 detects operations (gestures) on the operation surface 51A by the users U1, U2, and U3. Examples of operations include clicking, tapping, sliding, and scrolling. The input unit 53 outputs an operation signal to the control unit 54.

[0063] The operation device 50 includes a control unit 54. The control unit 54 generates operation data based on an operation signal from the input unit 53. The control unit 54 outputs the operation data to the communication unit 52.

[0064] The control unit 54 may be configured using a processor such as a CPU or a GPU, a clock, and an internal memory. The control unit 54 may be configured as a single piece of hardware (SoC) that integrates the processor, the clock, the internal memory, and the communication unit 52.

[0065] The operating device 50 includes a battery 55. The battery 55 supplies power to each part of the operating device 50.

[0066] Next, the operation of the operation device 50 by the users U1, U2, and U3 when the ear animation E of the character C is operated as, for example, "nodding", "agreement", "praise (clap)", "question", "raising hand" or "surprise" will be described. That is, the relationship between the movement of the ear animation E of the character C and the movement of the users U1, U2, and U3 detected by the operation device 50 will be described. The movement of the ear animation E and the movement of the fingers F of the users U1, U2, and U3 for operating the ear animation E are similar (related). For example, when operating the ear animation E of "nodding", the fingers F perform a movement similar to a nodding movement on the operation surface 51A of the operation device 50. That is, the users U1, U2, and U3 can operate the ear animation E by performing a movement similar to the ear animation E they want to operate on the operation device 50.

[0067] Fig. 5 is a diagram showing an example of the operation of the ear animation E. Pattern A in Fig. 5 shows the ear animation E in normal mode.

[0068] Pattern B in FIG. 5 shows a "nodding" motion in ear animation E. As shown in pattern B in FIG. 5, in the "nodding" motion, the upper half of the ear in ear animation E of character C is bent forward. When making ear animation E execute the "nodding" motion, users U1, U2, and U3 touch operation surface 51A of main body 51 of operation device 50 with two fingers F (e.g., index finger and middle finger). Alternatively, users U1, U2, and U3 slide two fingers F while touching operation surface 51A with two fingers F. The amount of sliding of fingers F is, for example, 1 cm to 2 cm.

[0069] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, 30. The control units 12 of the terminals 10, 20, 30 generate control data for causing the ear animation E to act as a "nod" based on the action data.

[0070] Pattern C in FIG. 5 shows the motion of "agree" in the ear animation E. In the motion of "agree", the ears of the character C's ear animation E are bent forward as a whole. When making the ear animation E perform the motion of "agree", the users U1, U2, and U3 slide the two fingers F (e.g., index finger and middle finger) while touching the operation surface 51A. The sliding distance of the fingers F is, for example, longer than 2 cm, for example, 3 cm to 4 cm. That is, when making the ear animation E perform the motion of "agree", the users U1, U2, and U3 slide the finger F longer than when making the ear animation E perform the motion of "nodding".

[0071] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, 30. The control units 12 of the terminals 10, 20, 30 generate control data for operating the ear animation E as “agreement” based on the action data.

[0072] Pattern D in Fig. 5 shows the "clap (praise)" motion in ear animation E. In the "clap (praise)" motion, the ears in the ear animation E of character C move closer together and apart. When making the ear animation E execute the "clap (praise)" motion, users U1, U2, and U3 touch two fingers F (e.g., thumb and index finger, or index finger and middle finger) to the operation surface 51A and then move the two fingers F close together (in contact) or apart.

[0073] When the above-mentioned motion is performed, the operation device 50 detects the motion with the input unit 53 and transmits the motion as motion data to the terminals 10, 20, and 30. The control units 12 of the terminals 10, 20, and 30 generate control data for causing the ear animation E to move as "claps (praise)" based on the motion data. The control units 12 of the terminals 10, 20, and 30 may generate control data for clapping multiple times in response to motion data relating to a single motion of bringing two fingers F close to each other and then separating them.

[0074] Pattern E in FIG. 5 shows a "question" action in ear animation E. In the "question" action, the ears of character C's ear animation E tangle. When making ear animation E perform the "question" action, users U1, U2, and U3 touch operation surface 51A with two fingers F (e.g., index finger and middle finger) and slide to draw a circle (semicircle). Note that when making ear animation E perform the "question" action, users U1, U2, and U3 may touch operation surface 51A with two fingers F (e.g., index finger and middle finger) and slide to draw ∞ (infinity).

[0075] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, 30. The control units 12 of the terminals 10, 20, 30 generate control data for operating the ear animation E as a "question" based on the action data.

[0076] Pattern F in Fig. 5 shows a "raise hand" motion in ear animation E. In the "raise hand" motion, the ears stretch in the ear animation E of character C. When making the ear animation E execute the "raise hand" motion, users U1, U2, and U3 touch operation surface 51A with two fingers F (e.g., index finger and middle finger) and slide only one finger F (middle finger) upward.

[0077] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, and 30. The control units 12 of the terminals 10, 20, and 30 generate control data for causing the ear animation E to act as "raise hand" based on the action data.

[0078] Pattern G in Fig. 5 shows a "surprise" motion in ear animation E. In the "surprise" motion, the tips of the ears in character C's ear animation E become larger. At this time, the body B of character C bends backward. When making the ear animation E perform the "surprise" motion, users U1, U2, and U3 touch the operation surface 51A with two fingers F (e.g., index finger and middle finger), and then slide the two fingers F upward and apart.

[0079] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, 30. The control units 12 of the terminals 10, 20, 30 generate control data for causing the ear animation E to act as "surprised" based on the action data.

[0080] <How to control ear animation> Next, a method of operating the movement of the ear animation E of the character C will be described. When operating the ear animation E, the users U1, U2, and U3 perform a movement corresponding to, for example, any one of pattern B of FIG. 5, pattern C of FIG. 5, pattern D of FIG. 5, pattern E of FIG. 5, pattern F of FIG. 5, and pattern G of FIG. 5. This causes the operation device 50 to generate movement data. The operation device 50 transmits the movement data to the terminals 10, 20, and 30.

[0081] When the terminals 10, 20, and 30 receive the motion data, they generate control data corresponding to the motion data. As a result, in the virtual space R of the terminals 10, 20, and 30, the ear animation E of the character C performs one of the motions of "nodding," "agreement," "praise (clap)," "question," "raise hand," or "surprise." In this way, the users U1, U2, and U3 operate the operation device 50 to control the motion of the ear animation E of the character C.

[0082] <Methods of communication> A method of conversation (interaction) between characters C in virtual space R will be described. First, virtual space R will be described. A plurality of characters C may exist in virtual space R. Virtual space R is, for example, a metaverse. In addition to characters C, various objects may exist in virtual space R. BGM may be played in virtual space R, and announcements (guidance) may be played. BGM and announcements may be changed depending on the position of character C. Various activities can be carried out in virtual space R. Examples of activities include lectures, speeches, training, meetings, reports, counseling, consultations, entertainment, and sightseeing.

[0083] In the virtual space R, a character C can be placed on a moving stage (object) S. That is, the character C can ride on the moving stage S. The moving stage S is displayed integrally with one or more characters C. The moving stage S forms an independent area in which the character C is located. An independent area is an area that can be distinguished from others. In other words, the area of ​​one moving stage S (hereinafter also referred to as the "first moving stage S") and the area of ​​another moving stage S (hereinafter also referred to as the "second moving stage S") different from it are independent from each other. This makes it possible to recognize at a glance that a certain character C is riding on the first moving stage S, and another character C is riding on the second moving stage S different from the first moving stage S. Note that the independent area may include multiple parts.

[0084] A character C is positioned on the moving stage S. The moving stage S is provided movably in the virtual space R. The moving direction and speed of the moving stage S can be operated by the user. That is, the user can move the moving stage S to any position. When multiple characters C are on the moving stage S, a cockpit may be provided on the moving stage S so that the user of the character C positioned in the cockpit can operate the moving stage S preferentially. The moving stage S may be moved by automatic control. In this case, the moving stage S may move automatically along a set route, for example, or may move automatically to a set destination. In addition, the user can arbitrarily change the display viewpoint. For example, the viewpoint can be rotated in 3D.

[0085] One or more characters C can be placed on the moving stage S. The size of the moving stage S may change depending on the number of characters C placed. Specifically, the size of the moving stage S may increase as the number of characters C placed increases, and decrease as the number of characters C placed decreases. The size of the moving stage S may change in stages, or the size may change continuously (gradually) as the number of people placed increases or decreases. The size, design, etc. of the moving stage S may be set appropriately by the user. Identification information, text information, etc. may be provided on the moving stage S.

[0086] Next, an example of a method for starting a conversation between a character C1 (any character among one or more characters C) and a character C2 (another character different from character C1) will be described with reference to FIG. 6. The user of character C1 may be user U1, U2, or U3. The user of character C2 may be user U1, U2, U3, or AI. In the following, an example will be described in which the user of character C1 is user U1 and the user of character C2 is user U2.

[0087] In the example shown in FIG. 6, character C1 is on a moving stage S, and character C2 is on a moving stage S. In the state shown in FIG. 6, character C1 and character C2 cannot communicate with each other. In other words, character C1 and character C2 cannot have a conversation. In the state shown in FIG. 6, each of character C1 and character C2 can hear background music, announcements, etc. playing in virtual space R.

[0088] When character C1 requests a conversation with character C2, an approval process is carried out to confirm whether character C2 is available for conversation. The approval process is carried out when an operation to associate character C1 with character C2 is accepted. In this embodiment, the operation to associate character C1 with character C2 is an operation of dragging and dropping character C1 onto the moving stage S of character C2.

[0089] In the consent process, when character C1 is dragged and dropped onto the moving stage S of character C2, a message for consenting to a conversation is automatically sent to user U2 of character C2, as shown in Fig. 7. The message may include text such as "May I speak to you now?" and user information (such as name and user name) that can identify character C1.

[0090] Character C1 may be dragged and dropped onto character C2, or the moving stage S on which character C1 is located may be dragged and dropped onto character C2 or the moving stage S on which character C2 is located. Also, as shown in Fig. 6, the operation to associate character C1 with character C2 may be performed in a state in which the moving stage S on which character C1 is located and the moving stage S on which character C2 is located are separated, or as shown in Fig. 8, the operation to associate character C1 with character C2 may be performed in a state in which the moving stage S on which character C1 is located and the moving stage S on which character C2 is located are close to each other (in contact).

[0091] When a message of conversation acceptance is sent to character C2, the message is displayed on the screen of user U2 of character C2. The above text and user information are displayed on the message display screen, along with an "OK button" for accepting the conversation and a "NG button" for rejecting the conversation. When the user of character C2 selects the "NG button", a message indicating that the conversation has not been accepted is displayed on the screen of user U1 of character C1. Examples of messages include "I'm sorry, but I have something to do," "I cannot respond until XX o'clock," and "I will be available from XX o'clock." The content of the message sent when the NG button is selected can be set by the user as appropriate. When the user of character C2 selects the "NG button," the operation of dragging and dropping character C1 onto the moving stage S of character C2 is invalidated (invalidated).

[0092] When the user U2 of the character C2 selects the "OK button", as shown in FIG. 9, an animation is executed showing the process of combining the moving stage S on which the character C1 is positioned and the moving stage S on which the character C2 is positioned, and one moving stage S is displayed as shown in FIG. 10. That is, the moving stage S of the character C1 and the moving stage S of the character C2 are integrated. The size of the moving stage S changes depending on the number of characters C positioned. Specifically, the moving stage S on which the two characters C1 and C2 are positioned is larger than the moving stages S on which the characters C1 and C2 are positioned. Note that the animation shown in FIG. 9 does not have to be executed, and a transition may be made from the state shown in FIG. 6 or FIG. 7 to the state shown in FIG. 10.

[0093] Characters C1 and C2 are located on the same moving stage S. Characters C1 and C2 are located on one moving stage S. On the moving stage S, characters C1 and C2 can converse with each other. That is, a conversation is possible (a conversation is established) between multiple characters C1 and C2 located on one (same) moving stage S. The conversation between characters C1 and C2 on the moving stage S cannot be heard by characters located outside the moving stage S. On the moving stage S, announcements and the like played in the virtual space R can be heard.

[0094] The volume of conversation within the moving stage S can be set as appropriate. The volume of conversation can also be adjusted so that some sound leaks outside the moving stage S. In this case, it is easier to create a natural atmosphere, as if someone is having a conversation, compared to a situation where there is no sound at all. Instead of sound, a changing sound linked to the sound may be emitted.

[0095] In addition, the moving stage S may have a function for contacting the characters C1, C2 from outside the moving stage S. For example, a user outside the moving stage S may be able to make voice calls or web communications (email, chat, etc.) to the characters C1, C2 riding on the moving stage S. For example, voice calls, etc. can be made on a character-by-character basis, or on a moving stage-by-moving stage basis. When making a voice call to a character, a number set for that character is contacted. When making a voice call to a station, a stage number set for each moving stage is contacted. The users of the characters C1, C2 can notify a specific user of the stage number.

[0096] In the above embodiment, when character C1 is dragged and dropped onto the moving stage S of character C2, a message for accepting the conversation is automatically sent to user U2 of character C2, but when user U1 of character C1 clicks on character C2, who is the conversation target, a message for accepting the conversation may be automatically sent to user U2 of character C2. Also, when character C2 is clicked, a function selection window is displayed, and a message may be sent to character C2 by selecting (clicking) the function of "send acceptance message" displayed in the function selection window. In this case, after the conversation is accepted by user U2 of character C2, when character C1 is dragged and dropped onto moving stage S of character C2, the moving stage S on which characters C1 and C2 ride is displayed.

[0097] In addition, if communication (such as a meeting schedule) has been set up in advance between the characters C1 and C2, the consent process may be omitted. In this case, when the character C1 is dragged and dropped onto the moving stage S of the character C2, the moving stage S on which the characters C1 and C2 are placed is displayed, and a conversation can be held between the characters C1 and C2.

[0098] When ending the conversation between the characters C1 and C2, the user U1 performs an operation to end the conversation. In this embodiment, the operation to end the conversation is an operation to drag and drop the character C to the outside of the moving stage S. Specifically, as shown in FIG. 11, the user U1 drops and drags, for example, the character C1 to the outside of the moving stage S. As a result, the moving stage S is newly displayed, and the character C1 is positioned on the moving stage S (see FIG. 6). In this state, the characters C1 and C2 cannot communicate with each other. In other words, the characters C1 and C2 cannot have a conversation with each other. As a result, the conversation between the characters C1 and C2 ends.

[0099] Also, the conversation between the character C1 and the character C2 may be automatically ended based on a preset time. The preset time may be an arbitrarily set time or may be a time set by a timer. When the preset time is reached, the state shown in FIG. 11 automatically transitions to the state shown in FIG. 6. That is, two moving stages S are displayed, and the characters C1 and C2 are located on each of the two moving stages S. Alternatively, the characters C1 and C2 automatically move to a gathering place (which may be a moving stage S where multiple characters C gather, or may be any place in the virtual space R). This ends the conversation between the character C1 and the character C2. Also, the conversation between the character C1 and the character C2 may be ended by the administrator (the administrator of the virtual space R). In this case, too, the state shown in FIG. 11 may automatically transition to the state shown in FIG. 6, or the characters C1 and C2 may automatically move to the gathering place.

[0100] Next, a case where multiple characters C are on a moving stage S and multiple characters C request a conversation together will be described. In the example shown in Fig. 12, characters C1, C2, C3, and C4 are on one moving stage S, and characters C5 and C6 are on another moving stage S different from the moving stage S. Characters C1, C2, C3, and C4 make up group 1. Characters C5 and C6 make up group 2. A case where group 2 requests a conversation with group 1 will be described below.

[0101] An administrator is set for each of group 1 and group 2. The administrator has authority to collectively control conversations in the group. The administrator has authority to operate the moving stage S. The administrator can be set to one or more users. Here, an example will be described in which the administrator for group 1 is character C1 and the administrator for group 2 is character C5.

[0102] When Group 2 requests a conversation with Group 1, an approval process is carried out to confirm whether or not conversation is possible in Group 1. In the approval process, when the administrator character C5 is dragged and dropped onto the moving stage S of Group 1, a message for conversation approval is automatically sent to the user of character C1, the administrator of Group 1.

[0103] When the user of character C1 selects the "NG button", a message is displayed on the screen of the user of character C5 indicating that the conversation has not been accepted. When the user of character C1 selects the "OK button", one moving stage S is displayed as shown in Fig. 13. That is, the moving stage S of group 1 and the moving stage S of group 2 are integrated.

[0104] Characters C1, C2, C3, C4, C5, and C6 are located on the same moving stage S. Characters C1, C2, C3, C4, C5, and C6 are located on one moving stage S. On the moving stage S, characters C1, C2, C3, C4, C5, and C6 can talk to each other. That is, a plurality of characters C1, C2, C3, C4, C5, and C6 located on one moving stage S can talk to each other (conversation is established). On the moving stage S, all of characters C1, C2, C3, C4, C5, and C6 can talk to each other, and each of characters C1, C2, C3, C4, C5, and C6 can talk to each other (for example, character C1 and character C2, character C3 and character C4, character C5 and character C6).

[0105] Next, a case where a conversation is ended in a group will be described. When a conversation is ended in group units, for example, the user of character C1, who is the administrator of group 1, surrounds characters C1, C2, C3, and C4, which are members of group 1, as shown in FIG. 13. As a result, characters C1, C2, C3, and C4 are surrounded by, for example, a broken line. The user of character C1 drops and drags the part surrounded by the broken line to the outside of the moving stage S. As a result, a new moving stage S is displayed, and characters C1, C2, C3, and C4 of group 1 are positioned on the moving stage S. That is, as shown in FIG. 12, characters C1, C2, C3, and C4 get on one moving stage S, and characters C5 and C6 get on another moving stage S different from the moving stage S. As a result, the conversation between group 1 and group 2 ends.

[0106] In addition, the designation of members of Group 1 is not limited to the above method, and may also be performed by clicking and selecting characters C1, C2, C3, and C4, or by inputting information that can identify characters C1, C2, C3, and C4 on a specified screen.

[0107] In the state shown in FIG. 13, when ending a conversation on a character-by-character basis, for example, the user of character C1 drops and drags character C1 outside the moving stage S. As a result, a new moving stage S is displayed, character C1 is positioned on one moving stage S, and characters C2, C3, C4, C5, and C6 are positioned on other moving stages S different from the one. In this state, character C1 cannot communicate with characters C2, C3, C4, C5, and C6. In other words, character C1 cannot converse with characters C2, C3, C4, C5, and C6. This ends the conversation.

[0108] As in the above case, the conversation between group 1 and group 2 may be ended automatically based on a preset time, or may be ended by an administrator.

[0109] As shown in FIG. 14, a cover CV can be set on the moving stage S. The cover CV is provided on the moving stage S so as to cover, for example, the characters C1 and C2. The size of the cover CV can change according to the size of the moving stage S. The cover CV has, for example, a hemispherical shape. The cover CV may be transparent, semi-transparent, or may not have transparency. In the example shown in FIG. 14, the cover CV is transparent.

[0110] The cover CV is information related to the conversation in the moving stage S. The cover CV has a function of informing the outside of the situation related to participation in the conversation. The information related to the conversation is information indicating whether or not participation in the conversation is possible, information indicating the degree of possibility of participation in the conversation, etc. In this embodiment, the information related to the conversation is information indicating whether or not participation in the conversation is possible. In this embodiment, when the cover CV is displayed, it basically indicates that participation in the conversation is impossible. Depending on the transparency and color of the cover CV, etc., it is possible to inform the users around the moving stage S of the degree of possibility of participation in the conversation in the moving stage S (the state within the moving stage S). For example, when the cover CV is transparent, it is possible to indicate that participation in the conversation is possible by applying for a conversation. When the cover CV does not have transparency (when it is opaque), it is possible to clearly indicate that participation in the conversation from the outside is rejected (rejected). In addition, depending on the degree of transparency of the cover CV (transparent, semi-transparent, opaque, etc.), it is possible to indicate the degree of possibility of participation in the conversation in stages.

[0111] The cover CV can be set to be visible or hidden by the user. When multiple characters C1 and C2 are on the moving stage S as shown in FIG. 14, a user set as an administrator may set the cover CV to be visible or hidden. The cover CV indicates that only authorized users can participate in the conversation. When the cover CV is displayed, as shown in FIG. 12, even if the user of character C drags and drops character C3 onto the moving stage S of characters C1 and C2, the drag and drop is rejected.

[0112] When a cover CV is set on the moving stage S, it may be possible to contact the characters C1 and C2 in the cover CV. For example, when a meeting is being held on the moving stage S, the user of the character C who joins the meeting late can make voice calls or web communications with the users on the moving stage S or the entire group. This allows the users of the characters C1 and C2 on the moving stage S to temporarily hide the cover CV, making it possible to drag and drop the character C3 onto the moving stage S, for example. Note that when the cover CV is opaque, it can also be set to reject contact from outside.

[0113] In addition, the necessity of the approval process may be set depending on whether the cover CV is displayed or not (whether or not the cover CV is present). Specifically, it is possible to set that the approval process is required when the cover CV is displayed, and that the approval process is not required when the cover CV is not displayed.

[0114] Also, as shown in FIG. 15, in the case where an object having a display function, for example, a projector board PB, is provided on the moving stage S, when the cover CV is not displayed, text, images, etc. displayed on the projector board PB may be made visible from outside the moving stage S, and when the cover CV is displayed, as shown in FIG. 16, the text, etc. displayed on the projector board PB may be made invisible from outside the moving stage S. In the example shown in FIG. 16, the cover CV is transparent, and the character C located on the moving stage S can be viewed from the outside, but the text, etc. displayed on the projector board PB cannot be viewed (hidden). In this case, when the cover CV is displayed, the character C located on the moving stage S can be viewed from the outside, but the display contents of the projector board PB cannot be viewed from the outside. As a result, for example, when a meeting is being held on the moving stage S, the participants and situation of the meeting can be viewed from the outside, but the contents of the meeting cannot be viewed from the outside.

[0115] In addition, when the cover CV is displayed, the projector board PB may be hidden or may be blurred, etc. In short, it is sufficient that the contents displayed on the projector board PB cannot be recognized (understood) from the outside of the moving stage S.

[0116] [effect] As described above, in the system 100 according to this embodiment, when a plurality of characters C are located on one moving stage S, the characters C can have a conversation with each other. That is, a conversation is established between the characters C located on one moving stage S. Therefore, it is possible to grasp at a glance which characters C are having a conversation with each other. In other words, it is possible to grasp at a glance which characters C are not having a conversation. In this way, the system 100 can visualize the state of communication. Therefore, in the system 100, for example, when communication is not taking place, it is possible to propose a chat to the character C. Therefore, in the system 100, chats and the like can be naturally generated. As a result, in the system 100, communication between the characters C can be easily generated in the virtual space R.

[0117] In the system 100 according to the present embodiment, when an operation of dragging and dropping an arbitrary character C of one or more characters C or a moving stage S on which the arbitrary character C is located onto another character C or a moving stage S on which the arbitrary character C is located is accepted, the moving stage S on which the arbitrary character C was located and the moving stage S on which the other character C was located are integrated, and the arbitrary character C and the other character C are positioned on the integrated single moving stage S. In this configuration, multiple characters C can be positioned on one moving stage S by a simple operation of dragging and dropping. In addition, by integrating the moving stages S, the sense of unity of the conversation is improved. Furthermore, by providing a joining transition process of dragging and dropping, the user who operates the character C can prepare mentally, so that it is possible to approach a more natural feeling.

[0118] In the system 100 according to the present embodiment, when the drag-and-drop operation is accepted, a screen is displayed for the user of the other character C to select whether or not to permit the conversation, and if the user of the other character C permits the conversation, the moving stage S on which the arbitrary character C was located and the moving stage S on which the other character C was located are integrated, and the arbitrary character C and the other character C are positioned on the integrated single moving stage S, and if the user of the other character C rejects the conversation, the drag-and-drop operation is invalidated. With this configuration, the user of the character C can select whether or not to permit the conversation.

[0119] Although the embodiments of the present disclosure have been described above, the present disclosure is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit and scope of the present disclosure.

[0120] In the above embodiment, the ear animation E is moved in one of "nodding", "agreement", "praise (clap)", "question", "raising hand" and "surprise" as an example. However, the movement of the ear animation E according to the user's movement is not limited to this. The movement of the ear animation E may be other movements or may further include other movements. The bending angle and speed of the ear animation E can be set (changed) as appropriate. This allows delicate emotions to be expressed and the intensity of the emotion expression can be adjusted (stronger, weaker). When the movement intensity of the ear animation E is set to be strong, for example, the non-verbal information expression of a person who is not good at expressing emotions can be supported by the content, intensity and speed of the movement of the ear animation E. In addition, for communication between cultures with different non-verbal information expressions, the movement form can be converted according to the culture of the users U1, U2, and U3.

[0121] In the above embodiment, an example has been described in which the motion of the hand H or the finger F of the user U1, U2, or U3 is detected by the operation device 50. However, the hand H or the finger F of the user U1, U2, or U3 may be detected by an image sensor (camera, laser, or the like).

[0122] The movement of the ear animation E is basically executed based on the operations of the users U1, U2, and U3, but may be combined with the facial expressions of the users U1, U2, and U3 (facial recognition by a camera), the voices of the users U1, U2, and U3, and expression control by AI (Artificial Intelligence), etc. The movement of the ear animation E may be controlled by selecting the movement of the character C in an operation window displayed on the screen.

[0123] In the above embodiment, the operation device may be any device capable of detecting acceleration and angular velocity. The operation device may be, for example, a mobile phone (smartphone, etc.) having a display (display unit) that displays the ear animation E. In the case of a mobile phone, the operation data is generated by operating the mobile phone in the same manner as the operation device 50.

[0124] In the above embodiment, an example has been described in which the control data is generated in the terminal 10, the terminal 20, and the terminal 30. However, the control data may be generated by the server 40. In this configuration, the terminal 10, the terminal 20, and the terminal 30 transmit operation data to the server 40, and the server 40 generates the control data based on the operation data.

[0125] In the above embodiment, a form in which the body B and the ear animation E are displayed on the terminals 10, 20, and 30 has been described as an example. However, when the terminal is a wearable terminal or a head mounted display (HMD) terminal, the body B and the ear animation E may be displayed on a lens or the like, or may be displayed as a hologram. That is, the character C is displayed in cyberspace. Cyberspace may include VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality).

[0126] In the above embodiment, a form in which a character C having an ear animation E is used in a virtual space R has been described as an example. However, the form of the character is not limited to this. The character may be an avatar. The avatar may or may not include a facial expression. The avatar may have an ear animation E.

[0127] Character C may be transformed. For example, a display unit or the like may be displayed on the body B of character C. The display unit may display conversation text obtained by converting the voice of character C into text, a translation of the conversation text, and a translation of the display content of the display unit. Furthermore, the display unit may display at least one of necessary figures, sentences, music, videos, still images, and websites by using AI based on at least one of the voice and text input of character C.

[0128] In the above embodiment, the character C is on the moving stage S. However, the object forming the area where the character C is located is not limited to the moving stage S. For example, the object forming the area where the character C is located may be a simple disk, a flying carpet, a spaceship, an airship, a large bus, a flying classroom, or the like.

[0129] Furthermore, the object may be a band-like object that surrounds the character C. In this case, when the character C is located in the object, it means that the character C is located within the band-like object.

[0130] In the above embodiment, the character C moves in response to the operation of the operation device 50. However, the movement of the character C may be controlled by other methods. For example, the state of the users U1, U2, and U3 may be acquired and the movement of the character C may be controlled based on the acquired state.

[0131] In the above embodiment, the operation of associating the character C1 with the character C2 is described as an example of an operation of dragging and dropping the character C1 onto the moving stage S of the character C2. However, the operation of associating the characters C with each other is not limited to dragging and dropping. For example, when associating the character C1 with the character C2, the associating operation may be an operation in which the user of the character C1 selects (clicks, etc.) the character C2, an operation in which the user of the character C1 selects the character C2 on a predetermined selection screen, or an operation in which the user of the character C1 inputs information or the like that identifies the character C2. In short, the associating operation may be an operation in which the user of the character C1 can identify the character C2, who is the conversation partner (target).

[0132] In the above embodiment, when a conversation acceptance message is sent to character C, who is a conversation partner, in the acceptance process, the message is displayed on the screen of the user of character C, and the user can select whether or not to accept the conversation on the message screen. However, the selection of whether or not to accept the conversation may be made by the user's voice (such as "accept", "OK", "do not accept", "NG" etc.).

[0133] In the above embodiment, an example has been described in which the operation for ending a conversation between characters C is an operation for dragging and dropping character C outside of the moving stage S. However, the operation for ending a conversation is not limited to dragging and dropping. For example, the operation for ending a conversation may be an operation for selecting an item for ending the conversation on a specific screen (such as a menu screen), or an operation for clicking an "x" mark displayed on part of character C. In short, the operation for ending a conversation may be an operation in which the user of the character who ends the conversation indicates an intention to end the conversation.

[0134] In the above embodiment, the cover CV is described as being hemispherical in shape as an example. However, the shape of the cover CV is not limited to this. In addition, the cover CV is described as being information related to participation in a conversation on the moving stage S as an example. However, the information related to participation in a conversation on the moving stage S is not limited to this.

[0135] For example, the information related to participation in a conversation may be a cloud CL displayed above the character C as shown in pattern A of FIG. 17, or may be a wall-like member WL arranged on the periphery of the moving stage S as shown in pattern B of FIG. 17. The member WL may be transparent or opaque. Furthermore, the information related to participation in a conversation may be something like a planting composed of one or more trees TR as shown in pattern A of FIG. 18. Furthermore, the information related to participation in a conversation may be an animal (such as a bird) displayed above the character C, or may be something like a curtain or something like mist.

[0136] As shown in pattern B of FIG. 18, the information may be information indicating a specific group such as "XYZ" displayed on the moving stage S. In this case, by displaying "XYZ", it is possible to externally suggest that participation of users other than users related to "XYZ" is rejected (participation is asked to be refrained from). In addition, if the user is related to "XYZ", that is, if information related to "XYZ" is set in the user's attribute information, participation in the conversation may be permitted. In other words, if information related to "XYZ" is not set in the user's attribute information, participation in the conversation may be rejected. Even if information related to "XYZ" is not set in the user's attribute information, participation in the conversation may be permitted if permitted by the character C located on the moving stage S.

[0137] In the above embodiment, when an operation of dragging and dropping an arbitrary character C or a moving stage S on which the arbitrary character C is located onto another character C or a moving stage S on which the arbitrary character C is located has been accepted, the moving stage S on which the arbitrary character C was located and the moving stage S on which the other character C was located are integrated, and the arbitrary character C and the other character C are positioned on the integrated single moving stage S. However, the process of positioning the arbitrary character C and the other character C on the single moving stage S may be performed automatically.

[0138] With reference to Fig. 19, an example of a method in which character C automatically starts a conversation will be described. In the example shown in Fig. 19, in a virtual space R, each of character C1, character C2, character C3, character C4, character C5, and character C6 is on a moving stage S. The users of character C1, character C2, character C3, character C4, character C5, and character C6 may be users U1, U2, and U3. At least one of character C1, character C2, character C3, character C4, character C5, and character C6 may be an AI user.

[0139] The virtual space R is used for gatherings where multiple characters converse (meetings, training sessions, social gatherings, social gatherings, etc.). Conversations between characters in the virtual space R are managed by an administrator. The administrator can set the time the meeting is held, the duration of each conversation, the number of people conversing, etc. The duration of each conversation can be set to, for example, 3 to 4 minutes. The number of people conversing is not limited, but can be set to, for example, two people.

[0140] In the virtual space R, for example, characters C1, C2, C3, C4, C5, and C6 may exist, whose participation has been permitted by the administrator. The users of the characters C1 to C6 may apply to the administrator in advance to participate, and when the administrator permits their participation, they may receive a URL or the like related to their participation, transmitted by the administrator. The users of the characters C1 to C6 may enter the virtual space R by participating from the URL at a specified date and time.

[0141] In the situation shown in FIG. 19 (before the conversation starts), the administrator may give a lecture on how to proceed with the conversation, etc. For example, an overview of the conversation time, etc. may be explained. The administrator may also give a lecture on how to conduct the conversation. For example, an explanation may be given on how to conduct the conversation, such as first introducing yourself with your name, etc., and then talking about the theme, etc. In addition, an explanation may be given on how to listen when having a conversation (for example, listening while repeating words, nodding, expressing surprise, or praising, etc.). During the lecture by the administrator, questions, etc. may be accepted from the users of the characters C1 to C6.

[0142] When an operation to start a conversation is performed by the administrator, as shown in FIG. 20, two characters are automatically positioned on one moving stage S. That is, two characters are automatically matched. In the example shown in FIG. 20, character C1 and character C3 are positioned on the same moving stage S, character C2 and character C5 are positioned on the same moving stage S, and character C4 and character C6 are positioned on the same moving stage S. The characters positioned on the moving stage S may be selected by the administrator or may be selected randomly. It is preferable that the characters positioned on the moving stage S are set so as not to be matched with characters that have been matched in the past. That is, it is preferable that the matching partner is set to be a different character each time.

[0143] A character C7 may exist in the virtual space R. The character C7 is operated by, for example, a facilitator user. The user of the character C7 may be an administrator.

[0144] The conversations between characters C1 and C3 on the moving stage S, the conversations between characters C2 and C5, and the conversations between characters C4 and C6 cannot be heard by characters located outside the moving stage S. Announcements by character C7 can be heard on each moving stage S. The user of character C7 may be able to hear the conversations on all moving stages S. In other words, the user of character C7 may be able to monitor all moving stages S. In addition, the user of character C7 may be able to record (audio) the conversations on all moving stages S.

[0145] As shown in FIG. 20, the moving stage S may be provided with a projector board (display unit) PB. The projector board PB may display the theme of the conversation (information related to the content of the conversation). For example, favorite foods, favorite places, things to do, favorite activities (hobbies), and desired items may be displayed on the projector board PB. The projector board PB may also display a translation of the conversation between the characters. If the projector board PB is not provided, the theme of the conversation may be announced. The moving stage S may also display a clock, timer, and the like. In the case of a timer, the remaining time of the conversation is displayed. An alarm may also be set to sound when the remaining time reaches a predetermined time (for example, 10 seconds).

[0146] As shown in Fig. 21, for example, the display screen of the user of character C1 and the user of character C3 may display only the moving stage S where character C1 and character C3 are located. The user can switch the display of the screen. By performing a predetermined operation, the user can display a screen (whole screen) on which all characters C1 to C7 (moving stages S) are displayed as shown in Fig. 20, or a screen (individual screen) on which only the user's own character and the opponent's character are displayed as shown in Fig. 21.

[0147] The characters C1 to C6 can give their conversation partner a card CD as shown in Fig. 22. The users of the characters C1 to C6 can give the card CD to their conversation partner by performing a predetermined operation (for example, selecting a card sending item on the operation screen). The card CD is created in advance by the users of the characters C1 to C6.

[0148] The card CD includes at least information that can identify the user of the character. For example, a character is displayed on the card CD. For example, text can be displayed on the character. As text, a name, nickname, identification information, etc., can be displayed. A face photo, etc. may be displayed on the character. Furthermore, an image P can be displayed in the display area DA of the card CD. In the display area DA, a message can be displayed, music can be embedded, and video can be displayed. A received card CD is automatically saved. A saved card CD can be viewed. It may be possible to contact the user of the card CD by clicking on the card CD (a message creation screen is displayed).

[0149] When a predetermined time has elapsed since the conversation started in the state shown in Fig. 20, the state automatically returns to the state shown in Fig. 19, and then the conversation partner is automatically changed as shown in Fig. 23. That is, when a predetermined time has elapsed in a state in which multiple characters are positioned on one moving stage S (the state shown in Fig. 20), a new moving stage S is displayed and the character that has finished the conversation is positioned on the newly displayed moving stage S (the state shown in Fig. 19). Then, as shown in Fig. 23, two new characters are automatically positioned on one moving stage S.

[0150] 23, characters C3 and C5 are located on the same moving stage S, characters C1 and C6 are located on the same moving stage S, and characters C2 and C4 are located on the same moving stage S. In other words, the characters are matched with a different character from the previous time.

[0151] In the state shown in Fig. 23, when a predetermined time has elapsed since the start of the conversation, the state automatically returns to the state shown in Fig. 19 once, as described above, and then the conversation partner is automatically changed. This process is performed multiple times within a time period (e.g., 30 minutes) set by the administrator. When the time period set by the administrator has elapsed, the state returns to the state shown in Fig. 19.

[0152] In the above configuration, the moving stage S on which an arbitrary character was located and the moving stage S on which other characters were located are automatically integrated, and the arbitrary character and the other characters are positioned on one integrated moving stage S. In this configuration, multiple characters can be automatically positioned on one moving stage S. That is, the characters can be automatically matched. This allows communication not only with a specific character (user) but also with new characters. Therefore, by coming into contact with a new personality of a user with whom there has been no communication until now, a sense of intimacy can be improved. Therefore, subsequent communication becomes easier, psychological safety is improved, and improvements in creativity, productivity, etc. can be expected.

[0153] Furthermore, when a predetermined time has elapsed while multiple characters are positioned on one moving stage S, a new moving stage S is displayed and one or more characters who have finished their conversation are positioned on the newly displayed moving stage S. In this configuration, the conversation is forcibly ended when the predetermined time has elapsed. In this way, the communication time is limited, so communication with the same character does not drag on. This makes it possible to communicate with many characters.

[0154] In addition, a projector board PB that displays information related to the content of the conversation is displayed on one moving stage S where multiple characters are located. In this configuration, the topic of the conversation is displayed (provided) on the projector board PB, so communication can be carried out smoothly and efficiently within a limited time. Therefore, conversations that touch on the personality of the character's user can be carried out efficiently.

[0155] Also, information related to a predetermined time may be displayed in one moving stage S where multiple characters are located. In this configuration, for example, the remaining time of the predetermined time can be confirmed, so that communication can be carried out taking into consideration time allocation (end time).

[0156] Also, a character operated by a facilitator user may be displayed in the virtual space R. In this configuration, when applied to a meeting in which many users participate (for example, a training session, an information exchange meeting, a social gathering, etc.), the meeting can be smoothly conducted because the meeting is conducted by the facilitator.

[0157] In addition, when multiple characters are located on one moving stage S, it is possible to exchange cards CD that contain at least information that can identify the user of the character. In this configuration, after the conversation ends, it is possible to confirm the character (user) with whom the conversation took place.

[0158] In the above embodiment, an example in which two characters are positioned on one moving stage S has been described, but the number of characters positioned on one moving stage S may be three or more. The number of characters may be appropriately set by the administrator. [Explanation of symbols]

[0159] 10...terminal, 11...input section, 12...control section, 13...storage section, 13P...terminal program, 14...communication section, 15...output section, 20...terminal, 30...terminal, 40...server, 42...control section, 43...storage section, 43P...server program, 43S...received data, 44...communication section, 50...operation device, 51A...operation surface, 52...communication section, 53...input section, 54...control section, 55...battery, 100...system, AP...access point, B...body, BS...base station, C...character, C1...character (any character character), C2...character (other character), C3...character, C4...character, C5...character, C6...character, C7...character, CD...card, CL...cloud, CV...cover, DA...display area, E...ear animation, F...finger, H...hand, N...network, P...image, PB...projector board, R...virtual space, S...moving stage, first moving stage, second moving stage, TR...tree, U1...user, U2...user, U3...user, WL...component.

Claims

1. A communication support program for supporting communication between a plurality of characters in a virtual space, comprising: Displaying an object on which one or more of the characters are located integrally with one or more of the characters; integrating the object on which any one of the one or more characters was located with the object on which the other one of the characters was located, and positioning the any one of the characters and the other one of the characters on the integrated object; A communication support program that enables a plurality of characters located on one of the objects to converse with each other.

2. A communication support program as described in claim 1, which, when an operation to associate any one of the characters with another one of the characters different from the one character is received, integrates the object in which the any one of the characters was located with the object in which the other one of the characters was located.

3. When the associating operation is accepted, the user of the other character is prompted to select whether or not to allow the user to have a conversation with the other character; If the user of the other character allows conversation, integrating the object in which the arbitrary character was located with the object in which the other character was located, and positioning the arbitrary character and the other character on the integrated single object; The communication support program according to claim 2 , wherein the associating operation is invalidated when the user of the other character refuses to have a conversation.

4. 4. The communication support program according to claim 2, wherein the associating operation is an operation of dragging and dropping any one of the characters or the object on which the character is located onto another one of the characters or the object on which the character is located.

5. 4. The communication support program according to claim 2 or claim 3, wherein when a plurality of said characters are positioned on one of said objects and an operation to end a conversation of one or more of said characters is received, said object is newly displayed and the one or more of said characters that have ended the conversation are positioned on said newly displayed object.

6. 6. The communication support program according to claim 5, wherein the operation for ending the conversation is an operation of dragging and dropping one or more of the characters from the object to the outside.

7. The communication support program according to claim 1 , wherein the object is movable in the virtual space.

8. 3. The communication support program according to claim 1, further comprising: a communication support program for enabling communication between the character located on the object and the character not located on the object.

9. 3. The communication support program according to claim 1, further comprising displaying information relating to participation in a conversation on said object.

10. 3. The communication support program according to claim 1 or claim 2, wherein when one or more of the characters are located on one of the objects, the characters are automatically moved to a predetermined location when a predetermined time arrives or a predetermined operation is received.

11. 3. The communication support program according to claim 1, wherein at least one of the plurality of characters located on one of the objects is operated by an AI.

12. A communication support program as described in claim 1, which automatically integrates the object in which any one of the characters was located with the object in which another character different from the character in question was located.

13. 13. The communication support program according to claim 12, wherein when a predetermined time has elapsed while a plurality of said characters are positioned at one said object, said object is newly displayed and one or more of said characters that have finished their conversation are positioned at the newly displayed object.

14. 14. The communication support program according to claim 12, further comprising: displaying a display unit that displays information related to a conversation content in one of the objects in which a plurality of the characters are located.

15. The communication support program according to claim 13 , further comprising displaying information relating to the predetermined time in one of the objects on which a plurality of the characters are positioned.

16. The communication support program according to claim 12 or 13, further comprising displaying the character operated by a facilitator user in the virtual space.

17. 14. The communication support program according to claim 12, wherein in a state where a plurality of said characters are located on one of said objects, exchange of cards including at least information capable of identifying a user of said character is possible.

18. The communication support program according to claim 12 or 13, wherein at least one of the plurality of characters located on one of the objects is operated by an AI.

19. A communication support method for supporting communication between a plurality of characters in a virtual space, comprising: Displaying an object on which one or more of the characters are located integrally with one or more of the characters; integrating the object on which any one of the one or more characters was located with the object on which the other one of the characters was located, and positioning the any one of the characters and the other one of the characters on the integrated object; A communication support method that enables a plurality of characters located on one of the objects to converse with each other.

20. A communication support system that supports communication between multiple characters in a virtual space, Displaying an object on which one or more of the characters are located integrally with one or more of the characters; integrating the object on which any one of the one or more characters was located with the object on which the other one of the characters was located, and positioning the any one of the characters and the other one of the characters on the integrated object; A communication support system that enables a plurality of characters located on one object to converse with each other.

Images