Communication assistance program, communication assistance method, and communication assistance system
The communication support system addresses the challenge of generating spontaneous interactions in virtual spaces by integrating characters on shared objects, enabling visualization of conversation status and automatic matching, thus enhancing natural communication in virtual environments.
Patent Information
- Application Number
- PCT/JP2023/039937
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-17
- Filing Date
- 2023-11-06
- Publication Date
- 2024-11-21
AI Technical Summary
In virtual communication spaces, such as the metaverse, it is difficult to generate spontaneous and natural small talk between characters due to challenges in expressing nonverbal information like line of sight and conversation reactions, making it hard to understand conversation status and group dynamics.
A communication support system that allows characters to be integrated onto a shared object within a virtual space, enabling easy conversation initiation, visualization of communication status, and automatic matching for conversation participation, with features like drag-and-drop operations and AI-controlled characters.
Facilitates easy and natural generation of communication between characters in virtual spaces by visualizing conversation status and allowing spontaneous interactions, improving the sense of unity and familiarity through automatic matching and limited conversation time.
Smart Images

Figure JP2023039937_21112024_PF_FP_ABST
Abstract
Description
Communication support program, communication support method, and communication support system
[0001]
[0001] One aspect of the present disclosure relates to a communication support program, a communication support method, and a communication support system. This application claims priority to Japanese Application No. 2023-081344, filed on May 17, 2023, and incorporates by reference all of the contents of said Japanese application.
[0002] In recent years, Teams (registered trademark), Zoom (registered trademark), and the like have become mainstream in remote communication. In these remote communication services, participants are set and invited in advance, and when a predetermined time arrives, the participants access the web or the like to conduct communication such as a conference. Non-Patent Document 1 discloses Teams as an example of remote communication.
[0003] “Microsoft Teams”, [online], Microsoft, [searched May 10, 2023], Internet,<URL:https: / / www.microsoft.com / ja-jp / microsoft-teams / group-chat-software>
[0004] A communication support program according to one embodiment of the present disclosure is a communication support program that supports communication between multiple characters in a virtual space, and displays an object in which one or more characters are located integrally with the one or more characters, allowing multiple characters located in a single object to converse with each other.
[0005] FIG. 1 is a diagram illustrating an example of a schematic configuration of a display system according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of a terminal and a server. FIG. 3 is a perspective view of an operation device. FIG. 4 is a diagram illustrating an example of functional blocks of the operation device illustrated in FIG. 3. FIG. 5 is a diagram illustrating an example of the operation of an ear animation. FIG. 6 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 7 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 8 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 9 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 10 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 11 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 12 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 13 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 14 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 15 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 16 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 17 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 18 is a diagram illustrating an example of a screen displayed in a virtual space. FIG. 19 is a diagram illustrating an example of a screen displayed in a virtual space. Fig. 20 is a diagram showing an example of a screen displayed in a virtual space. Fig. 21 is a diagram showing an example of a screen displayed in a virtual space. Fig. 22 is a diagram showing an example of a screen displayed in a virtual space. Fig. 23 is a diagram showing an example of a screen displayed in a virtual space.
[0006] [Problem to be Solved by the Present Disclosure] In the above-described remote communication, although communication such as conversation can be conducted with members designated as participants, conversations without a predetermined communication partner, so-called casual conversations, are unlikely to occur spontaneously or spontaneously. For example, in places where people actually gather (party venues, smoking rooms, free spaces, tea rooms (water rooms), break rooms, dining rooms, hallways, etc.), casual conversations naturally occur among people present. In such places, conversation participants look at each other's faces, nod, and respond. Therefore, by observing these subtle changes in facial expressions, it is possible to grasp the conversational status of the conversation participants and recognize the conversation group. Therefore, by observing the behavior of the conversation participants and choosing the right timing, it is possible to call out to the conversation participants, and casual conversations naturally occur.
[0007] In virtual spaces such as the metaverse, which is a form of remote communication, characters (avatars) can move freely within the virtual space and communicate with other characters. This allows for casual conversations between characters. However, it is not easy to engage in casual conversation within the metaverse. This is because the metaverse does not accurately represent non-verbal information such as the characters' gazes and conversational responses, making it difficult to grasp the situation between characters and to recognize conversation groups, such as which characters are conversing.
[0008] The present disclosure aims to provide a communication support program, a communication support method, and a communication support system that can easily generate communication between characters in a virtual space.
[0009] Effect of the Present Disclosure According to the present disclosure, communication between characters can be easily generated in a virtual space.
[0010] [Description of Embodiments of the Present Disclosure] First, the contents of the embodiments of the present disclosure will be listed and described. At least some of the embodiments described below may be combined in any combination.
[0011] (1) A communication support program according to one aspect of the present disclosure is a communication support program that supports communication between multiple characters in a virtual space, in which an object on which one or more characters are located is displayed integrally with the one or more characters, and in which multiple characters located on a single object can converse with each other.
[0012] In a communication assistance program according to one aspect of the present disclosure, when multiple characters are located on a single object, the characters can converse with each other. That is, a conversation can be established between characters located on a single object. Therefore, it is possible to grasp at a glance which characters are conversing with each other. In other words, it is possible to grasp at a glance which characters are not conversing. In this way, the communication assistance program can visualize the state of communication. Therefore, for example, when communication is not taking place, the communication assistance program can prompt the characters to engage in casual conversation. Therefore, the communication assistance program can naturally generate casual conversations, etc. As a result, the communication assistance program can easily generate communication between characters in a virtual space.
[0013] (2) In the communication assistance program described in (1) above, when an operation to associate any one of one or more characters with another character different from the one character is received, the object where the one character was located may be integrated with the object where the other character was located, and the one character and the other character may be positioned on a single integrated object. In this configuration, the operation to associate the one character with the other character can position multiple characters on a single object. Furthermore, by integrating the objects, a sense of unity in the conversation is improved.
[0014] (3) In the communication assistance program of (2) above, when the associating operation is accepted, the user of the other character may be allowed to choose whether or not to allow the conversation, and if the user of the other character allows the conversation, the object where the given character was located may be integrated with the object where the other character was located, and the given character and the other character may be positioned in a single integrated object, and if the user of the other character refuses the conversation, the associating operation may be invalidated. In this configuration, the user of the character may choose whether or not to allow the conversation.
[0015] (4) In the communication assistance program of (2) or (3), the associating operation may be a drag-and-drop operation of any character or an object on which the character is located onto another character or an object on which the character is located. In this configuration, multiple characters can be placed on a single object by a simple drag-and-drop operation. Furthermore, by providing a drag-and-drop transition process, the user operating the characters can prepare mentally, resulting in a more natural experience.
[0016] (5) In the communication assistance program according to any one of (2) to (4), when a plurality of characters are positioned on a single object and an operation to end a conversation between one or more characters is received, the object may be newly displayed and the one or more characters whose conversation has ended may be positioned on the newly displayed object. In this configuration, when the conversation ends, the character whose conversation has ended may be positioned on the object again.
[0017] (6) In the communication assistance program of (5), the operation for ending the conversation may be an operation of dragging and dropping one or more characters from the object to the outside. In this configuration, the conversation can be ended by a simple operation of dragging and dropping.
[0018] (7) In the communication assistance program according to any one of (1) to (6), the object may be movable in the virtual space. In this configuration, a character located on the object can be moved in the virtual space.
[0019] (8) In the communication assistance program according to any one of (1) to (7), a character located on an object may be able to communicate with a character not located on the object. In this configuration, for example, the character located on the object may be able to confirm in advance whether or not the character can participate in the conversation.
[0020] (9) In the communication assistance program according to any one of (1) to (8), information regarding participation in a conversation may be displayed on the object. With this configuration, it is possible to grasp at a glance whether the object can participate in a conversation, the degree of participation in the conversation, etc.
[0021] (10) In the communication assistance program according to any one of (1) to (9), when one or more characters are located in one object, the characters may be automatically moved to a predetermined location when a predetermined time arrives or a predetermined operation is received. In this configuration, a conversation in one object may be ended and the characters may be automatically gathered at a predetermined location (such as a meeting place).
[0022] (11) In the communication assistance program of any one of (1) to (10) above, at least one of the characters located on one object may be operated by AI.
[0023] (12) In the communication assistance program described in (1) above, an object where one or more characters are located may be automatically integrated with an object where another character different from the character is located, and the character and the other character may be positioned in a single integrated object. In this configuration, multiple characters can be automatically positioned in a single object. In other words, characters can be automatically matched. This allows communication not only with a specific character (user) but also with new characters.
[0024] (13) In the communication assistance program of (12) above, when a predetermined time has elapsed while multiple characters are positioned on a single object, the object may be newly displayed and one or more characters who have finished their conversation may be positioned on the newly displayed object. In this configuration, the conversation ends after the predetermined time has elapsed. In this way, the communication time is limited, so communication with the same character does not drag on for too long. This makes it possible to communicate with many characters.
[0025] (14) In the communication assistance program of (12) or (13) above, a display unit that displays information related to the content of a conversation may be displayed on an object where multiple characters are located. In this configuration, the information related to the content of the conversation, such as the topic of the conversation, is displayed (provided) on the display unit, thereby enabling smooth communication.
[0026] (15) In the communication assistance program of (13) above, information relating to a predetermined time may be displayed on a single object where multiple characters are located. In this configuration, for example, the remaining time of the predetermined time can be checked, and communication can be carried out taking into consideration time allocation (end time).
[0027] (16) In any one of the communication support programs (12) to (15) above, a character operated by a user who is a facilitator may be displayed in a virtual space. In this configuration, when applied to a meeting attended by many users (for example, a training session, an information exchange meeting, a social gathering, etc.), the meeting can be smoothly conducted because the meeting is conducted by the facilitator.
[0028] (17) In the communication assistance program according to any one of (12) to (16), when multiple characters are located on a single object, it may be possible to exchange cards containing at least information that identifies the user of the character. In this configuration, after the conversation ends, it is possible to confirm the character (user) that has had the conversation.
[0029] (18) In the communication assistance program of any one of (12) to (17) above, at least one of the characters located on one object may be operated by AI.
[0030] (19) A communication support method according to one aspect of the present disclosure is a communication support method that supports communication between multiple characters in a virtual space, in which an object in which one or more characters are located is displayed integrally with the one or more characters, and multiple characters located on a single object are able to converse with each other.
[0031] A communication assistance method according to one aspect of the present disclosure includes the same features as the above-described communication assistance program, and therefore can achieve the same effects and advantages as the above-described communication assistance program.
[0032] (20) A communication support system according to one aspect of the present disclosure is a communication support method that supports communication between multiple characters in a virtual space, in which an object on which one or more characters are located is displayed integrally with the one or more characters, and multiple characters located on a single object are able to converse with each other.
[0033] A communication assistance system according to an aspect of the present disclosure includes the same features as the above-described communication assistance program, and can therefore achieve the same functions and effects as the above-described communication assistance program.
[0034] [Details of the embodiments of the present disclosure] Specific examples of the embodiments of the present disclosure will be described below with reference to the drawings. The present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.
[0035] 1 is a diagram showing an example of the schematic configuration of a communication support system (hereinafter, sometimes simply referred to as the "system"). The system 100 is a computer system that supports communication in a virtual space R (see FIG. 6) using a character C (see pattern A in FIG. 5) that can be controlled by a user and whose ear animation E (see pattern A in FIG. 5) moves in response to the user's operation.
[0036] The character C may be operated by a human user U1, U2, or U3, or may be operated by a non-human. The character C may be operated by, for example, AI (Chat GPT, Siri (registered trademark), Google Assistant (registered trademark), Alexa (registered trademark), or Cortana (registered trademark). In other words, the user operating the character C may be AI. The user means an operator or the like operating the character C.
[0037] Character C represents the user. Character C is a dynamic character. Character C is generated by image material that is independent of the actual image of the user (image data, etc.). Image material is a material for generating an animated character. Character C may be rendered by two-dimensional or three-dimensional computer graphics (CG).
[0038] As an example, character C includes a body B (see pattern A in FIG. 5 ) and an ear animation E. In character C, the body B and the ear animation E are displayed as an integrated unit. Displaying them as an integrated unit means that when looking at character C, the body B and the ear animation E are visually recognized as a single entity. In other words, displaying them as an integrated unit only requires that it be recognizable at a glance that the body B and the ear animation E are connected (corresponding). Therefore, displaying them as an integrated unit may mean that the body B and the ear animation E are in contact with each other, or may mean that the body B and the ear animation E are separated from each other.
[0039] Body B may include at least one of an image representing the user and text representing the user. A shadow may be displayed on character C. Body B may be spherical and may include text (such as "A," "B," or "C") that identifies the user. Character C may include a talking animation (not shown) or animation that resembles arms, legs, or the like.
[0040] The ear animation E is an animation that expresses non-verbal information. The ear animation E is an animation that shows the user's state (movement) and emotions. The ear animation E is an animation that expresses the user's state in a form different from the user's actual moving parts. The ear animation E may not include dynamically changing facial expressions, or may include dynamically changing facial expressions. The ear animation E is displayed outside the display area of the body B so as to protrude from the display area. The ear animation E is an animation that imitates the ears of an animal (e.g., a rabbit), and moves in response to changes in the user's state. The ear animation E moves in response to the user's movements. For example, the ear animation E moves by expanding, contracting, bending, swinging, etc. The ear animation E is freely deformable, and visually changes in response to the user's state to evoke the user's state.
[0041] In the system 100, the ear animation E of the character C allows the user to check the user's state, emotions, reactions, etc.
[0042] <System Configuration> The system 100 includes a terminal 10, a terminal 20, and a terminal 30. In the example shown in FIG. 1 , the terminal 10 is a laptop personal computer (PC) used by a user U1. The terminal 20 is a mobile phone (smartphone, etc.) used by a user U2. The terminal 30 is a desktop PC used by a user U3. The terminals 10, 20, and 30 are connectable to a network N. In this example, the terminal 10 is connected to the network N via an access point AP, the terminal 20 is connected to the network N via a base station BS, and the terminal 30 is connected to the network N by wire (not shown). In addition to the PC and mobile phone described above, the terminals 10, 20, and 30 may be tablet terminals, wearable terminals, head-mounted display (HMD) terminals, and various other terminals. Note that the number of terminals included in the system 100 is not limited to the example shown in FIG. 1 .
[0043] The system 100 includes a server 40. The server 40 is connected to a network N. Therefore, the terminals 10, 20, 30, and the server 40 can communicate with each other via the network N. Note that the number of servers included in the system 100 is not limited to the example in FIG. 1 .
[0044] There are no limitations on the configuration of the network N. For example, the network N may be configured to include the Internet (public network), a communication carrier network, a business network, and the like.
[0045] 2 is a diagram showing examples of functional blocks of the terminal 10, the terminal 20, the terminal 30, and the server 40. The functional blocks of the terminals 20 and 30 are similar to the functional blocks of the terminal 10, so the following will describe the terminal 10 and the server 40. First, the terminal 10 will be described, and then the server 40 will be described.
[0046] <Configuration of Terminal> The terminal 10 includes an input unit 11. The input unit 11 is a part to which voice data and image data of the user U1 are input. The input unit 11 may also receive video data of the user U1. The video includes voice data and image data of the user U1.
[0047] The terminal 10 includes a control unit 12. The control unit 12 is a part that generates control data for the user U1. The control data for the user U1 is generated based on motion data output from the operation device 50. In this embodiment, for example, the control data may be generated to cause the ear animation E of the character C to perform six motions: "nodding," "agreement," "praise (clap)," "question," "raise hand," and "surprise." The control unit 12 may also generate control data for other motions (for example, motions indicating "YES" and "NO").
[0048] The control data is associated with the action data. In this embodiment, for example, the action data is associated with each of the control data of "nodding," "agreement," "praise (applause)," "question," "raise hand," and "surprise." The action data is data transmitted from the operating device 50. When the action data corresponds to the control data, the control unit 12 generates the control data.
[0049] The control data may be generated based on the voice data and image data of the user U1 input to the input unit 11. The image data is composed of a series of unit frames. A unit frame typically contains image information of the user U1. For example, image data at 60 frames per second contains 60 unit frames per second. The generation of the control data by the control unit 12 will be described in detail later.
[0050] The control unit 12 is also a part that controls the behavior of the character C based on the control data. Based on the generated control data and the received control data, the control unit 12 displays the character C in the virtual space R and controls the behavior of the ear animation E of the character C.
[0051] The control unit 12 generates video data of the virtual space R. The video data of the virtual space R includes video data of the character C controlled based on the control data. In addition, various objects that appear in the virtual space R may be included in the video data of the virtual space R. The method by which the control unit 12 generates the virtual space R is not particularly limited, and various known methods may be used. For example, the control unit 12 may generate the video data of the virtual space R based on an image captured by a camera. Alternatively, the control unit 12 may generate the video data of the virtual space R based on three-dimensional CAD data.
[0052] The terminal 10 includes a storage unit 13. The storage unit 13 is a unit that stores information necessary for controlling (processing) the terminal 10. In Fig. 2, a terminal program 13P is shown as an example of information stored in the storage unit 13. The terminal program 13P is a program for operating the terminal 10, and causes the computer to operate so that control by the control unit 12 and the like are executed. Since the system 100 is a communication support system, the terminal program 13P can also be said to be a communication support program.
[0053] The terminal 10 includes a communication unit 14. The communication unit 14 is a part that communicates with devices external to the terminal 10 via a network N (see FIG. 1 ). Examples of devices external to the terminal 10 are the terminal 20, the terminal 30, the server 40, and the operation device 50. The communication unit 14 transmits voice data and control data generated by the control unit 12 to the server 40, and receives voice data and control data from the server 40. The communication unit 14 is a part that communicates with the operation device 50. The communication unit 14 receives operation data from the operation device 50.
[0054] The terminal 10 includes an output unit 15. The output unit 15 outputs an image of the virtual space R. The image of the virtual space R is an image based on image data of the virtual space R generated by the control unit 12.
[0055] <Configuration of Server> The server 40 includes a control unit 42. The control unit 42 controls the operation of the server 40. The server 40 includes a storage unit 43. The storage unit 43 is a unit that stores information necessary for controlling the server 40. FIG. 3 illustrates a server program 43P and received data 43S as examples of information stored in the storage unit 43. The server program 43P is a program for operating the server 40, and causes the computer to operate so that control by the control unit 42 and the like are executed. The received data 43S is data sent from the terminals 10, 20, and 30 to the server 40, and may include control data generated by the terminals 10, 20, and 30.
[0056] The server 40 includes a communication unit 44. The communication unit 44 is a part that communicates with devices external to the server 40 via the network N (see FIG. 1 ). Examples of devices external to the server 40 are the terminals 10, 20, and 30. The communication unit 44 receives control data of user U1, control data of user U2, and control data of user U3 from the terminals 10, 20, and 30, respectively, and transmits control data generated by the terminals 10, 20, and 30 to the terminals 10, 20, and 30, respectively.
[0057] <Hardware Configuration> An example of the hardware configuration of each functional part of the terminal 10 and the server 40 will be described. The communication unit 44 of the server 40 may be configured using a network card or a wireless communication device to access the network N. The control unit 42 of the server 40 may be configured using a processor such as a central processing unit (CPU) or a graphics processing unit (GPU), a clock, and an internal memory. The control unit 42 may be configured as a single piece of hardware (SoC: System on a Chip) that integrates the processor, clock, internal memory, storage unit 43, and communication unit 44. The control unit 42 operates based on a server program 43P to operate the server computer as the server 40. The storage unit 43 of the server 40 may be configured using a non-volatile storage medium such as a flash memory, a hard disk, or an SSD (Solid State Disk).
[0058] The input unit 11 of the terminal 10 may be configured using a microphone, a camera, etc. The microphone acquires the voice of the user U1. That is, the microphone is a type of audio sensor that acquires voice. The camera acquires an image of the user U1. That is, the camera is a type of image sensor that acquires images. Furthermore, the input unit 11 may also be configured using operation devices such as a keyboard, a mouse, and a touch panel. The control unit 12 of the terminal 10 may be configured similarly to the control unit 42 of the server 40. The control unit 12 operates based on a terminal program 13P to operate a general-purpose computer as the terminal 10. The memory unit 13 of the terminal 10 may be configured similarly to the memory unit 43 of the server 40. The communication unit 14 of the terminal 10 may be configured similarly to the communication unit 44 of the server 40. The output unit 15 of the terminal 10 may be configured using a display device such as a liquid crystal panel, an organic EL panel, etc. (a touch panel may also be used). Furthermore, the output unit 15 may also be configured using a speaker.
[0059] <Configuration of Operation Device> Fig. 3 is a perspective view of the operation device 50. Fig. 3 is a diagram showing a state in which the operation device 50 is being operated by the fingers F of users U1, U2, and U3. The operation device 50 is a device operated by users U1, U2, and U3. An ear animation E is activated by the operation of the operation device 50 by users U1, U2, and U3. The operation device 50 is a touchpad. The touchpad may be of a capacitive type, a pressure-sensitive touch type, an ultrasonic type, or the like.
[0060] The operation device 50 includes a main body 51 as a hardware configuration. The main body 51 has an operation surface 51A. The operation device 50 is operated by the fingers F of the hands H of the users U1, U2, and U3. The operation device 50 may be provided as a standalone device, or may be provided as part of a remote control for operating a television display or the like. In other words, the operation device 50 may be part of a device.
[0061] Fig. 4 is a diagram showing an example of functional blocks of the operating device 50. As shown in Fig. 4, the operating device 50 includes a communication unit 52. The communication unit 52 is a part that communicates with an external device by wireless communication or wired communication. Examples of wireless communication include LAN, Bluetooth (registered trademark), Wi-Fi, etc. Examples of external devices are the terminal 10, the terminal 20, and the terminal 30. The communication unit 52 transmits operation data output from the control unit 54 to the terminal 10, the terminal 20, and the terminal 30.
[0062] The operation device 50 includes an input unit 53. The input unit 53 is a part that inputs operations of the users U1, U2, and U3 on the operation surface 51A. The input unit 53 detects operations (gestures) on the operation surface 51A by the users U1, U2, and U3. Examples of operations include clicking, tapping, sliding, and scrolling. The input unit 53 outputs an operation signal to the control unit 54.
[0063] The operation device 50 includes a control unit 54. The control unit 54 generates operation data based on an operation signal from the input unit 53. The control unit 54 outputs the operation data to the communication unit 52.
[0064] The control unit 54 may be configured using a processor such as a CPU or GPU, a clock, and an internal memory. The control unit 54 may also be configured as a single piece of hardware (SoC) that integrates the processor, the clock, the internal memory, and the communication unit 52.
[0065] The operating device 50 includes a battery 55. The battery 55 supplies power to each part of the operating device 50.
[0066] Next, the operations of the operation device 50 by the users U1, U2, and U3 when moving the ear animation E of the character C as, for example, "nodding," "agreement," "praise (clap)," "question," "raise hand," or "surprise" will be described. That is, the relationship between the movement of the ear animation E of the character C and the movements of the users U1, U2, and U3 detected by the operation device 50 will be described. The movements of the ear animation E and the movements of the fingers F of the users U1, U2, and U3 for moving the ear animation E are similar (related). For example, when moving the ear animation E of "nodding," the users U1, U2, and U3 perform a movement similar to a nodding movement with their fingers F on the operation surface 51A of the operation device 50. That is, the users U1, U2, and U3 can move the ear animation E by performing a movement on the operation device 50 similar to the ear animation E they want to move.
[0067] 5 is a diagram showing an example of the operation of the ear animation E. Pattern A in FIG. 5 shows the ear animation E in normal mode.
[0068] Pattern B in FIG. 5 shows a "nodding" motion in ear animation E. As shown in pattern B in FIG. 5, in the "nodding" motion, the upper half of the ear in ear animation E of character C is bent forward. To cause ear animation E to perform the "nodding" motion, users U1, U2, and U3 touch the operation surface 51A of the main body 51 of the operation device 50 with two fingers F (e.g., index finger and middle finger). Alternatively, users U1, U2, and U3 slide the two fingers F while touching the operation surface 51A. The amount of sliding of the fingers F is, for example, 1 cm to 2 cm.
[0069] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, and 30. The control unit 12 of the terminals 10, 20, and 30 generates control data for causing the ear animation E to act as a "nod" based on the action data.
[0070] Pattern C in FIG. 5 shows an "agree" motion in the ear animation E. In the "agree" motion, the ears of the character C's ear animation E are bent forward as a whole. To make the ear animation E perform the "agree" motion, users U1, U2, and U3 slide two fingers F (e.g., index finger and middle finger) while touching the operation surface 51A. The sliding distance of the fingers F is, for example, longer than 2 cm, e.g., 3 cm to 4 cm. In other words, when making the ear animation E perform the "agree" motion, users U1, U2, and U3 slide the fingers F a longer distance than when making the ear animation E perform the "nodding" motion.
[0071] When the above-mentioned operation is performed, the operation device 50 detects the operation with the input unit 53 and transmits the operation as operation data to the terminals 10, 20, and 30. The control units 12 of the terminals 10, 20, and 30 generate control data for operating the ear animation E as "agreement" based on the operation data.
[0072] Pattern D in Fig. 5 shows a "clap (praise)" motion in the ear animation E. In the "clap (praise)" motion, the ears of the ear animation E of the character C move closer together and apart. When causing the ear animation E to perform the "clap (praise)" motion, the users U1, U2, and U3 touch two fingers F (for example, the thumb and index finger, or the index finger and middle finger) to the operation surface 51A, and then move the two fingers F close together (in contact) and apart.
[0073] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminal 10, 20, 30. The control unit 12 of the terminal 10, 20, 30 generates control data for causing the ear animation E to act as "claps (praise)" based on the action data. The control unit 12 of the terminal 10, 20, 30 may generate control data for causing multiple claps in response to action data relating to a single action of bringing two fingers F close together and then separating them.
[0074] Pattern E in FIG. 5 shows a "question" motion in ear animation E. In the "question" motion, the ears of character C's ear animation E are tangled. To cause ear animation E to perform the "question" motion, users U1, U2, and U3 touch the operation surface 51A with two fingers F (e.g., index finger and middle finger) and slide them to draw a circle (semicircle). Note that to cause ear animation E to perform the "question" motion, users U1, U2, and U3 may touch the operation surface 51A with two fingers F (e.g., index finger and middle finger) and slide them to draw an infinity (∞) sign.
[0075] When the above-mentioned operation is performed, the operation device 50 detects the operation with the input unit 53 and transmits the operation data to the terminals 10, 20, and 30. The control unit 12 of the terminals 10, 20, and 30 generates control data for operating the ear animation E as a "question" based on the operation data.
[0076] Pattern F in Fig. 5 shows a "raise hand" action in ear animation E. In the "raise hand" action, the ears of character C extend in ear animation E. To cause ear animation E to perform the "raise hand" action, users U1, U2, and U3 touch operation surface 51A with two fingers F (e.g., index finger and middle finger) and then slide only one finger F (middle finger) upward.
[0077] When the above-mentioned action is performed, the operation device 50 detects the action using the input unit 53 and transmits the action as action data to the terminals 10, 20, and 30. The control unit 12 of the terminals 10, 20, and 30 generates control data for causing the ear animation E to act as a "raise hand" based on the action data.
[0078] Pattern G in Figure 5 shows a "surprise" motion in ear animation E. In the "surprise" motion, the tips of the ears in character C's ear animation E become larger. At this time, the body B of character C bends backward. To make the ear animation E perform the "surprise" motion, users U1, U2, and U3 touch the operation surface 51A with two fingers F (e.g., index finger and middle finger), and then slide the two fingers F upward and apart.
[0079] When the above-mentioned action is performed, the operation device 50 detects the action with the input unit 53 and transmits the action as action data to the terminals 10, 20, 30. The control unit 12 of the terminals 10, 20, 30 generates control data for causing the ear animation E to act as "surprised" based on the action data.
[0080] <Method of Operating the Movement of Ear Animation> Next, a method of operating the movement of the ear animation E of character C will be described. When operating the ear animation E, users U1, U2, and U3 perform a movement corresponding to, for example, pattern B of Fig. 5, pattern C of Fig. 5, pattern D of Fig. 5, pattern E of Fig. 5, pattern F of Fig. 5, or pattern G of Fig. 5. As a result, movement data is generated in the operation device 50. The operation device 50 transmits the movement data to the terminals 10, 20, and 30.
[0081] Upon receiving the action data, the terminals 10, 20, and 30 generate control data corresponding to the action data. As a result, in the virtual space R of the terminals 10, 20, and 30, the ear animation E of the character C performs one of the actions of "nodding," "agreement," "praise (clap)," "question," "raise hand," or "surprise." In this way, the users U1, U2, and U3 operate the operation device 50 to control the action of the ear animation E of the character C.
[0082] <Method of Communication> A method of conversation (interaction) between characters C in virtual space R will be described. First, virtual space R will be described. Multiple characters C can exist in virtual space R. Virtual space R is, for example, a metaverse. In addition to characters C, various objects may exist in virtual space R. BGM may be played in virtual space R, and announcements (guidance) may be played. The BGM and announcements may be changed depending on the position of character C. Various activities can be carried out in virtual space R. Examples of activities include lectures, speeches, training, meetings, reports, counseling, consultations, entertainment, sightseeing, etc.
[0083] In the virtual space R, a character C can be placed on a moving stage (object) S. That is, the character C can ride on the moving stage S. The moving stage S is displayed integrally with one or more characters C. The moving stage S forms an independent area in which the character C is located. An independent area is an area that can be distinguished from others. In other words, the area of one moving stage S (hereinafter also referred to as a "first moving stage S") and the area of another moving stage S (hereinafter also referred to as a "second moving stage S") are independent from each other. This makes it possible to recognize at a glance that a certain character C is riding on the first moving stage S, and another character C is riding on a second moving stage S that is different from the first moving stage S. Note that an independent area may include multiple parts.
[0084] A character C is positioned on the moving stage S. The moving stage S is provided movable in the virtual space R. The moving direction and speed of the moving stage S can be controlled by the user. That is, the user can move the moving stage S to any position. When multiple characters C are riding on the moving stage S, a cockpit may be provided on the moving stage S, so that the user of the character C sitting in the cockpit can operate the moving stage S preferentially. The moving stage S may also be moved by automatic control. In this case, the moving stage S may move automatically, for example, along a set route, or may move automatically to a set destination. The user can also arbitrarily change the display viewpoint. For example, the viewpoint can be rotated in 3D.
[0085] One or more characters C can be placed on the moving stage S. The size of the moving stage S may change depending on the number of characters C to be placed. Specifically, the size of the moving stage S may increase as the number of characters C placed increases, and decrease as the number of characters C placed decreases. The size of the moving stage S may change in stages, or the size may change continuously (gradually) as the number of people placed increases or decreases. The size, design, etc. of the moving stage S may also be set appropriately by the user. Identification information, text information, etc. may be provided on the moving stage S.
[0086] Next, an example of a method for starting a conversation between character C1 (any character among one or more characters C) and character C2 (another character different from character C1) will be described with reference to FIG. 6 . The user of character C1 may be user U1, U2, or U3. The user of character C2 may be user U1, U2, U3, or AI. In the following, an example will be described in which the user of character C1 is user U1 and the user of character C2 is user U2.
[0087] In the example shown in Fig. 6, character C1 is on a moving stage S, and character C2 is on a moving stage S. In the state shown in Fig. 6, character C1 and character C2 cannot communicate with each other. In other words, character C1 and character C2 cannot have a conversation. In the state shown in Fig. 6, character C1 and character C2 can each hear background music, announcements, etc. playing in virtual space R.
[0088] When character C1 proposes a conversation to character C2, an acceptance process is carried out to confirm whether character C2 is available for conversation. The acceptance process is carried out when an operation to associate character C1 with character C2 is accepted. In this embodiment, the operation to associate character C1 with character C2 is an operation of dragging and dropping character C1 onto the moving stage S of character C2.
[0089] In the consent process, as shown in Fig. 7, when character C1 is dragged and dropped onto the moving stage S of character C2, a message for consenting to the conversation is automatically sent to user U2 of character C2. The message may include text such as "May I speak to you now?" and user information (such as name or user name) that can identify character C1.
[0090] Note that character C1 may be dragged and dropped onto character C2, or the moving stage S on which character C1 is located may be dragged and dropped onto character C2 or the moving stage S on which character C2 is located. Also, as shown in Fig. 6, the operation to associate character C1 with character C2 may be performed in a state where the moving stage S on which character C1 is located and the moving stage S on which character C2 is located are separated, or as shown in Fig. 8, the operation to associate character C1 with character C2 may be performed in a state where the moving stage S on which character C1 is located and the moving stage S on which character C2 is located are close to each other (in contact).
[0091] When a message of acceptance of the conversation is sent to character C2, a message is displayed on the screen of user U2 of character C2. The message display screen displays the above text and user information, as well as an "OK button" to accept the conversation and an "NG button" to reject the conversation. When the user of character C2 selects the "NG button," a message indicating that the conversation has not been accepted is displayed on the screen of user U1 of character C1. Examples of messages include "Sorry, but I have something to do," "I am not available until XX o'clock," and "I will be available from XX o'clock." The content of the message sent when the NG button is selected can be set by the user as appropriate. When the user of character C2 selects the "NG button," the operation of dragging and dropping character C1 onto character C2's moving stage S is invalid (invalidated).
[0092] When user U2 of character C2 selects the "OK button," an animation is executed showing the process of combining the moving stage S on which character C1 is located and the moving stage S on which character C2 is located, as shown in FIG. 9, and one moving stage S is displayed, as shown in FIG. 10. That is, the moving stage S of character C1 and the moving stage S of character C2 are integrated. The size of the moving stage S changes depending on the number of characters C located there. Specifically, the moving stage S on which two characters C1 and C2 are located is larger than the moving stages S on which characters C1 and C2 are located, respectively. Note that the animation shown in FIG. 9 does not have to be executed, and a transition may be made from the state shown in FIG. 6 or FIG. 7 to the state shown in FIG. 10.
[0093] Character C1 and character C2 are located on the same moving stage S. Character C1 and character C2 are located on one moving stage S. On the moving stage S, character C1 and character C2 can converse. In other words, multiple characters C1 and C2 located on one (same) moving stage S can converse (conversation can be established). The conversation between character C1 and character C2 on the moving stage S cannot be heard by characters located outside the moving stage S. On the moving stage S, announcements and the like played in the virtual space R can be heard.
[0094] The volume of conversations within the moving stage S can be set as appropriate. The volume of conversations can also be adjusted so that some sound leaks outside the moving stage S. In this case, it is easier to create a natural atmosphere, as if someone is having a conversation, compared to when there is no sound at all. Instead of sound, a changing sound linked to the sound may be emitted.
[0095] The moving stage S may also have a function for contacting characters C1 and C2 from outside the moving stage S. For example, a user outside the moving stage S may be able to make voice calls or web communications (email, chat, etc.) with characters C1 and C2 riding on the moving stage S. For example, voice calls, etc. can be made on a character-by-character basis or on a moving stage-by-stage basis. When making a voice call to a character, the number set for that character is contacted. When making a voice call to a station, the stage number set for each moving stage is contacted. The users of characters C1 and C2 can notify a specific user of the stage number.
[0096] In the above embodiment, when character C1 is dragged and dropped onto the moving stage S of character C2, a message for accepting the conversation is automatically sent to user U2 of character C2. However, when user U1 of character C1 clicks on character C2, the conversation target, a message for accepting the conversation may also be automatically sent to user U2 of character C2. Alternatively, when character C2 is clicked, a function selection window is displayed, and a message may be sent to character C2 by selecting (clicking) the function "Send Acceptance Message" displayed in the function selection window. In this case, after user U2 of character C2 accepts the conversation, when character C1 is dragged and dropped onto moving stage S of character C2, a moving stage S with characters C1 and C2 on it is displayed.
[0097] Furthermore, if communication (such as a meeting schedule) has been set up in advance between the characters C1 and C2, the consent process may be omitted. In this case, dragging and dropping the character C1 onto the moving stage S of the character C2 displays the moving stage S on which the characters C1 and C2 are riding, allowing the characters C1 and C2 to converse with each other.
[0098] To end the conversation between characters C1 and C2, user U1 performs an operation to end the conversation. In this embodiment, the operation to end the conversation is an operation of dragging and dropping character C outside of the moving stage S. Specifically, as shown in FIG. 11, user U1 drops and drags, for example, character C1 outside of the moving stage S. As a result, a new moving stage S is displayed, and character C1 is positioned on the moving stage S (see FIG. 6). In this state, characters C1 and C2 cannot communicate with each other. In other words, characters C1 and C2 cannot converse with each other. This ends the conversation between characters C1 and C2.
[0099] The conversation between character C1 and character C2 may also be automatically ended based on a preset time. The preset time may be an arbitrarily set time or a time set by a timer. When the preset time is reached, the state shown in FIG. 11 automatically transitions to the state shown in FIG. 6. That is, two moving stages S are displayed, and characters C1 and C2 are positioned on each of the two moving stages S. Alternatively, characters C1 and C2 automatically move to a meeting place (which may be a moving stage S where multiple characters C are gathered, or may be any location in virtual space R). This ends the conversation between character C1 and character C2. The conversation between character C1 and character C2 may also be ended by an administrator (the administrator of virtual space R). In this case, too, the state shown in FIG. 11 may automatically transition to the state shown in FIG. 6, or characters C1 and C2 may automatically move to the meeting place.
[0100] Next, a case will be described in which multiple characters C are on a moving stage S and multiple characters C request a conversation together. In the example shown in Fig. 12, characters C1, C2, C3, and C4 are on one moving stage S, and characters C5 and C6 are on another moving stage S different from that moving stage S. Characters C1, C2, C3, and C4 make up group 1. Characters C5 and C6 make up group 2. Below, a case will be described in which group 2 requests a conversation with group 1.
[0101] An administrator is set for each of group 1 and group 2. The administrator has the authority to collectively control conversations within the group. The administrator has the authority to operate the moving stage S. The administrator can be set for one or more users. Here, an example will be described in which the administrator for group 1 is character C1 and the administrator for group 2 is character C5.
[0102] When Group 2 requests a conversation with Group 1, an approval process is carried out to confirm whether the conversation is possible in Group 1. In the approval process, when the administrator character C5 is dragged and dropped onto the moving stage S of Group 1, a message for accepting the conversation is automatically sent to the user of character C1, the administrator of Group 1.
[0103] When the user of character C1 selects the "NG button," a message indicating that the conversation has not been accepted is displayed on the screen of the user of character C5. When the user of character C1 selects the "OK button," one moving stage S is displayed, as shown in Figure 13. In other words, the moving stage S of group 1 and the moving stage S of group 2 are integrated.
[0104] Characters C1, C2, C3, C4, C5, and C6 are located on the same moving stage S. Characters C1, C2, C3, C4, C5, and C6 are located on one moving stage S. On the moving stage S, characters C1, C2, C3, C4, C5, and C6 can converse with each other. That is, multiple characters C1, C2, C3, C4, C5, and C6 located on one moving stage S can converse (conversation can be established). On the moving stage S, all of characters C1, C2, C3, C4, C5, and C6 can converse with each other, or individual characters C1, C2, C3, C4, C5, and C6 can converse with each other (for example, character C1 and character C2, character C3 and character C4, or character C5 and character C6).
[0105] Next, a case where a conversation is ended in a group will be described. When ending a conversation on a group-by-group basis, for example, the user of character C1, who is the administrator of group 1, surrounds characters C1, C2, C3, and C4, which are members of group 1, as shown in FIG. 13 . As a result, characters C1, C2, C3, and C4 are surrounded by, for example, a dashed line. The user of character C1 then drops and drags the portion surrounded by the dashed line outside the moving stage S. As a result, a new moving stage S is displayed, and characters C1, C2, C3, and C4 of group 1 are positioned on the moving stage S. That is, as shown in FIG. 12 , characters C1, C2, C3, and C4 are on one moving stage S, and characters C5 and C6 are on another moving stage S different from the moving stage S. This ends the conversation between group 1 and group 2.
[0106] In addition, the designation of members of Group 1 is not limited to the above method, and may also be done by clicking and selecting characters C1, C2, C3, and C4, or by entering information that can identify characters C1, C2, C3, and C4 on a specified screen.
[0107] In the state shown in FIG. 13 , to end a conversation on a character-by-character basis, for example, the user of character C1 drops and drags character C1 outside of the moving stage S. As a result, a new moving stage S is displayed, with character C1 positioned on one moving stage S and characters C2, C3, C4, C5, and C6 positioned on other moving stages S different from that moving stage S. In this state, character C1 cannot communicate with characters C2, C3, C4, C5, and C6. In other words, character C1 cannot converse with characters C2, C3, C4, C5, and C6. This ends the conversation.
[0108] As in the above case, the conversation between group 1 and group 2 may be ended automatically based on a preset time, or may be ended by an administrator.
[0109] As shown in FIG. 14 , a cover CV can be set on the moving stage S. The cover CV is provided on the moving stage S so as to cover, for example, characters C1 and C2. The size of the cover CV can change depending on the size of the moving stage S. The cover CV has, for example, a hemispherical shape. The cover CV may be transparent, semi-transparent, or not translucent. In the example shown in FIG. 14 , the cover CV is transparent.
[0110] The cover CV is information related to a conversation on the moving stage S. The cover CV has the function of externally informing the user of the status of participation in the conversation. The information related to the conversation includes information indicating whether or not the user can participate in the conversation, information indicating the degree of possibility of participation in the conversation, etc. In this embodiment, the information related to the conversation is information indicating whether or not the user can participate in the conversation. In this embodiment, when the cover CV is displayed, it basically indicates that the user cannot participate in the conversation. The transparency, color, etc. of the cover CV can inform users around the moving stage S of the degree of possibility of participation in the conversation on the moving stage S (the state within the moving stage S). For example, if the cover CV is transparent, it can indicate that the user can participate in the conversation by requesting a conversation. If the cover CV is not transparent (opaque), it can clearly indicate that participation in the conversation from the outside is being rejected (rejected). Furthermore, the degree of transparency of the cover CV (transparent, translucent, opaque, etc.) can indicate the degree of possibility of participation in the conversation in stages.
[0111] The cover CV can be set to be displayed or hidden by the user. As shown in FIG. 14, when multiple characters C1 and C2 are on the moving stage S, a user set as an administrator may set the cover CV to be displayed or hidden. The cover CV indicates that only authorized users can participate in the conversation. When the cover CV is displayed, as shown in FIG. 12, even if the user of character C drags and drops character C3 onto the moving stage S of characters C1 and C2, the drag and drop is rejected.
[0112] If a cover CV is set on the moving stage S, it may be possible to contact the characters C1 and C2 within the cover CV. For example, if a meeting is being held on the moving stage S, the user of character C who joins the meeting late can make voice calls or web communications with users on the moving stage S or the entire group. This allows the users of characters C1 and C2 on the moving stage S to temporarily hide the cover CV, making it possible, for example, to drag and drop character C3 onto the moving stage S. Note that if the cover CV is opaque, it can also be set to reject contact from outside.
[0113] Furthermore, whether or not the consent process is required may be set depending on whether or not the cover CV is displayed (whether or not the cover CV is present). Specifically, it is possible to set the consent process to be required when the cover CV is displayed, and not to be required when the cover CV is not displayed.
[0114] 15 , when an object having a display function, such as a projector board PB, is provided on the moving stage S, when the cover CV is not displayed, text, images, etc. displayed on the projector board PB can be viewed from outside the moving stage S. As shown in FIG. 16 , when the cover CV is displayed, text, etc. displayed on the projector board PB can be made invisible from outside the moving stage S. In the example shown in FIG. 16 , the cover CV is transparent, and the character C positioned on the moving stage S can be viewed from outside, but the text, etc. displayed on the projector board PB cannot be viewed (is not displayed). In this case, when the cover CV is displayed, the character C positioned on the moving stage S can be viewed from outside, but the display content of the projector board PB cannot be viewed from outside. This allows, for example, when a meeting is being held on the moving stage S, the participants and status of the meeting can be confirmed from outside, but the content of the meeting cannot be confirmed from outside.
[0115] When the cover CV is displayed, the projector board PB may be hidden or may be blurred, etc. The point is that the content displayed on the projector board PB cannot be recognized (understood) from the outside of the moving stage S.
[0116] [Effects] As described above, in the system 100 according to this embodiment, when multiple characters C are located on one moving stage S, the characters C can converse with each other. That is, a conversation can be established between the characters C located on one moving stage S. Therefore, it is possible to grasp at a glance which characters C are conversing with each other. In other words, it is possible to grasp at a glance which characters C are not conversing. In this way, the system 100 can visualize the state of communication. Therefore, in the system 100, for example, when no communication is taking place, it is possible to initiate casual conversation with the characters C. Therefore, the system 100 can naturally generate casual conversations, etc. As a result, the system 100 can easily generate communication between the characters C in the virtual space R.
[0117] In the system 100 according to the present embodiment, when an operation of dragging and dropping any one of one or more characters C or the moving stage S on which that character C is located onto another character C or the moving stage S on which that character C is located is received, the moving stage S on which the any one of the characters C was located and the moving stage S on which the other character C was located are integrated, and the any one of the characters C and the other character C are positioned on a single integrated moving stage S. In this configuration, multiple characters C can be positioned on a single moving stage S with the simple operation of dragging and dropping. Furthermore, by integrating the moving stages S, the sense of unity in the conversation is improved. Furthermore, by providing a participation transition process such as dragging and dropping, the user operating the character C can be mentally prepared, resulting in a more natural feeling.
[0118] In the system 100 according to this embodiment, when the drag-and-drop operation is accepted, a screen is displayed for the user of the other character C to select whether or not to allow the conversation, and if the user of the other character C allows the conversation, the moving stage S on which the given character C was located and the moving stage S on which the other character C was located are integrated, and the given character C and the other character C are positioned on the single integrated moving stage S, and if the user of the other character C refuses the conversation, the drag-and-drop operation is invalidated. With this configuration, the user of character C can choose whether or not to allow the conversation.
[0119] Although the embodiments of the present disclosure have been described above, the present disclosure is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present disclosure.
[0120] In the above embodiment, the ear animation E is described as moving in one of the following ways: "nodding," "agreement," "praise (applause)," "question," "raising a hand," or "surprise." However, the movement of the ear animation E in response to the user's movements is not limited to these. The movement of the ear animation E may be other movements or may include other movements. The bending angle and speed of the ear animation E can be set (changed) as appropriate. This allows for the expression of subtle emotions and the intensity of the emotional expression to be adjusted (stronger or weaker). For example, if the movement intensity of the ear animation E is set to be strong, the content, intensity, and speed of the movement of the ear animation E can support the non-verbal expression of a person who is not good at expressing emotions. Furthermore, for communication between cultures with different non-verbal expression techniques, the movement form can be configured to be converted to correspond to the cultures of users U1, U2, and U3.
[0121] In the above embodiment, an example has been described in which the operation device 50 detects the movements of the hands H or fingers F of the users U1, U2, and U3. However, the hands H or fingers F of the users U1, U2, and U3 may be detected by an image sensor (camera, laser, or the like).
[0122] Furthermore, the movement of the ear animation E is basically executed based on the operations of the users U1, U2, and U3, but may also be combined with the facial expressions of the users U1, U2, and U3 (face recognition by a camera), the voices of the users U1, U2, and U3, expression control by AI (Artificial Intelligence), etc. The movement of the ear animation E may also be controlled by selecting the movement of the character C in an operation window displayed on the screen.
[0123] In the above embodiment, the operation device may be any device capable of detecting acceleration and angular velocity. The operation device may be, for example, a mobile phone (smartphone, etc.) having a display (display unit) that displays the ear animation E. In the case of a mobile phone, the operation data is generated by operating the mobile phone in the same manner as the operation device 50.
[0124] In the above embodiment, an example has been described in which the control data is generated in the terminal 10, the terminal 20, and the terminal 30. However, the control data may be generated by the server 40. In this configuration, the terminal 10, the terminal 20, and the terminal 30 transmit operation data to the server 40, and the server 40 generates the control data based on the operation data.
[0125] In the above embodiment, an example has been described in which a body B, an ear animation E, and the like are displayed on the terminals 10, 20, and 30. However, if the terminal is a wearable terminal or a head-mounted display (HMD) terminal, the body B, the ear animation E, and the like may be displayed on lenses or as a hologram. In other words, a character C is displayed in cyberspace. Cyberspace may include VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality).
[0126] In the above embodiment, an example has been described in which a character C having ear animation E is used in a virtual space R. However, the character's form is not limited to this. The character may be an avatar. The avatar may or may not include a facial expression. The avatar may have ear animation E.
[0127] Character C may be transformed. For example, a display unit or the like may be displayed on the body B of character C. The display unit may display conversational text obtained by converting the voice of character C into text, a translation of the conversational text, a translation of the content displayed on the display unit, etc. Furthermore, the display unit may display at least one of necessary diagrams, sentences, music, videos, still images, and websites by using AI based on at least one of the voice and text input of character C.
[0128] In the above embodiment, an example has been described in which the character C is riding on a moving stage S. However, the object that forms the area where the character C is located is not limited to the moving stage S. For example, the object that forms the area where the character C is located may be a simple disk, a flying carpet, a spaceship, an airship, a large bus, or a flying classroom.
[0129] The object may also be a band-like object that surrounds the character C. In this case, character C being located in the object means that character C is located within the band-like object.
[0130] In the above embodiment, an example has been described in which the character C moves in response to the operation of the operation device 50. However, the movement of the character C may be controlled by other methods. For example, the states of the users U1, U2, and U3 may be acquired and the movement of the character C may be controlled based on the acquired states.
[0131] In the above embodiment, an example has been described in which the operation of associating character C1 with character C2 is an operation of dragging and dropping character C1 onto the moving stage S of character C2. However, the operation of associating characters C with each other is not limited to dragging and dropping. For example, when associating character C1 with character C2, the associating operation may be an operation in which the user of character C1 selects (clicks, etc.) character C2, an operation in which the user of character C1 selects character C2 on a predetermined selection screen, or an operation in which the user of character C1 inputs information identifying character C2. In short, the associating operation may be an operation in which the user of character C1 can identify character C2, who is the conversation partner (target).
[0132] In the above embodiment, when a conversation acceptance message is sent to character C, who is the conversation partner, in the acceptance process, a message is displayed on the screen of the user of character C, and the user can select whether to accept the conversation on the message screen. However, the selection of whether to accept the conversation may be made by the user's voice (such as "accept," "OK," "don't accept," or "NG").
[0133] In the above embodiment, an example has been described in which the operation for ending a conversation between characters C is an operation of dragging and dropping character C outside of the moving stage S. However, the operation for ending a conversation is not limited to dragging and dropping. For example, the operation for ending a conversation may be an operation of selecting an item for ending the conversation on a predetermined screen (such as a menu screen), or an operation of clicking an "x" mark displayed on part of character C. In short, the operation for ending a conversation may be an operation by which the user of the character whose conversation is to be ended indicates their intention to end the conversation.
[0134] In the above embodiment, the cover CV has been described as being hemispherical in shape as an example. However, the shape of the cover CV is not limited to this. Furthermore, the cover CV has been described as being an example of information related to participation in a conversation on the moving stage S. However, the information related to participation in a conversation on the moving stage S is not limited to this.
[0135] For example, the information related to participation in a conversation may be a cloud CL displayed above the character C, as shown in pattern A of Fig. 17 , or a wall-like member WL arranged on the periphery of the moving stage S, as shown in pattern B of Fig. 17 . The member WL may be transparent or opaque. Furthermore, the information related to participation in a conversation may be something like a planting composed of one or more trees TR, as shown in pattern A of Fig. 18 . Furthermore, the information related to participation in a conversation may be an animal (such as a bird) displayed above the character C, or something like a curtain, or something like mist.
[0136] As shown in pattern B of FIG. 18 , the information may indicate a specific group, such as "XYZ," displayed on the moving stage S. In this case, displaying "XYZ" can externally indicate that participation by users other than users related to "XYZ" is denied (requesting that they refrain from participating). Furthermore, if a user is related to "XYZ," i.e., if information related to "XYZ" is set in the user's attribute information, the user may be permitted to participate in the conversation. In other words, if information related to "XYZ" is not set in the user's attribute information, the user's participation in the conversation may be rejected. Even if information related to "XYZ" is not set in the user's attribute information, the user may be permitted to participate in the conversation if the character C located on the moving stage S allows it.
[0137] In the above embodiment, an example has been described in which, when an operation to drag and drop an arbitrary character C or a moving stage S on which the arbitrary character C is located onto another character C or a moving stage S on which the arbitrary character C is located is received, the moving stage S on which the arbitrary character C was located and the moving stage S on which the other character C was located are integrated, and the arbitrary character C and the other character C are positioned on the integrated single moving stage S. However, the process of positioning the arbitrary character C and the other character C on the single moving stage S may be performed automatically.
[0138] An example of a method for character C to automatically start a conversation will be described with reference to Figure 19. In the example shown in Figure 19, characters C1, C2, C3, C4, C5, and C6 are each on a moving stage S in a virtual space R. The users of characters C1, C2, C3, C4, C5, and C6 may be users U1, U2, and U3. At least one of characters C1, C2, C3, C4, C5, and C6 may be an AI user.
[0139] Virtual space R is used for gatherings where multiple characters gather and converse (meetings, training sessions, social gatherings, social gatherings, etc.). Conversations between characters in virtual space R are managed by an administrator. The administrator can set the duration of the gathering, the length of each conversation, the number of people conversing, etc. The length of each conversation can be set to, for example, 3 to 4 minutes. The number of people conversing is not limited, but can be set to, for example, two people.
[0140] In virtual space R, for example, characters C1, C2, C3, C4, C5, and C6 may exist, each of which has been permitted to participate by the administrator. For example, users of characters C1 to C6 may apply to the administrator in advance to participate, and if permitted by the administrator, receive a URL or the like related to participation sent by the administrator. Users of characters C1 to C6 can enter virtual space R by participating via the URL at a specified date and time.
[0141] In the situation shown in FIG. 19 (before the conversation starts), the administrator may give a lecture on how to proceed with the conversation, etc. For example, an overview of the conversation time, etc. may be explained. The administrator may also give a lecture on how to conduct the conversation. For example, an explanation may be given on how to conduct the conversation, such as first introducing yourself by name, etc., and then talking about the topic. The administrator may also give an explanation on how to listen when conducting a conversation (for example, repeating words, listening while nodding, expressing surprise, or praising, etc.). During the lecture by the administrator, questions, etc. may be accepted from the users of characters C1 to C6.
[0142] When the administrator performs an operation to start a conversation, the two characters are automatically positioned on one moving stage S, as shown in FIG. 20 . That is, the two characters are automatically matched. In the example shown in FIG. 20 , character C1 and character C3 are positioned on the same moving stage S, character C2 and character C5 are positioned on the same moving stage S, and character C4 and character C6 are positioned on the same moving stage S. The characters positioned on the moving stage S may be selected by the administrator or may be selected randomly. It is preferable that the characters positioned on the moving stage S are set so as not to be matched with characters that have been matched in the past. That is, it is preferable that the matching partner is set so as to be a different character each time.
[0143] A character C7 may exist in the virtual space R. The character C7 is operated by, for example, a facilitator user. The user of the character C7 may also be an administrator.
[0144] Conversations between characters C1 and C3, between characters C2 and C5, and between characters C4 and C6 on the moving stage S cannot be heard by characters located outside the moving stage S. Announcements by character C7 can be heard on each moving stage S. The user of character C7 may be able to hear conversations on all moving stages S. In other words, the user of character C7 may be able to monitor all moving stages S. The user of character C7 may also be able to record (audio) conversations on all moving stages S.
[0145] As shown in FIG. 20 , the moving stage S may be provided with a projector board (display unit) PB. The projector board PB may display the topic of the conversation (information related to the content of the conversation), etc. The projector board PB may display, for example, favorite foods, favorite places, things to do, favorite activities (hobbies), and things that people want. The projector board PB may also display a translation of the conversation between the characters. If a projector board PB is not provided, the topic of the conversation, etc. may be announced. The moving stage S may also display a clock, timer, etc. In the case of a timer, the remaining time of the conversation is displayed. An alarm may also sound when the remaining time reaches a predetermined time (for example, 10 seconds).
[0146] As shown in Fig. 21 , for example, the display screens of the user of character C1 and the user of character C3 may display only the moving stage S on which character C1 and character C3 are located. The user can switch the screen display. By performing a predetermined operation, the user can display a screen (full screen) on which all characters C1 to C7 (moving stages S) are displayed, as shown in Fig. 20 , or a screen (individual screen) on which only the user's own character and the opponent's character are displayed, as shown in Fig. 21 .
[0147] Characters C1 to C6 can give their conversation partners cards CDs as shown in Fig. 22. The users of characters C1 to C6 can give their conversation partners cards CDs by performing a predetermined operation (for example, selecting a card sending item on the operation screen). The cards CDs are created in advance by the users of characters C1 to C6.
[0148] The card CD contains at least information that can identify the user of the character. For example, a character is displayed on the card CD. For example, text can be displayed on the character. As text, a name, nickname, identification information, etc. can be displayed. A face photo, etc. can be displayed on the character. Furthermore, an image P can be displayed in the display area DA of the card CD. A message can be displayed, music can be embedded, or a video can be displayed in the display area DA. A received card CD is automatically saved. Saved card CDs can be viewed. It may be possible to contact the user of the card CD by clicking on the card CD (a message creation screen will be displayed).
[0149] When a predetermined time has elapsed since the start of a conversation in the state shown in Fig. 20, the state automatically returns to the state shown in Fig. 19, and then the conversation partner is automatically changed, as shown in Fig. 23. That is, when a predetermined time has elapsed in a state in which multiple characters are positioned on one moving stage S (the state shown in Fig. 20), a new moving stage S is displayed, and the character who has finished the conversation is positioned on the newly displayed moving stage S (the state shown in Fig. 19). Then, as shown in Fig. 23, two new characters are automatically positioned on one moving stage S.
[0150] 23, characters C3 and C5 are located on the same moving stage S, characters C1 and C6 are located on the same moving stage S, and characters C2 and C4 are located on the same moving stage S. In other words, the characters are matched with characters different from the previous time.
[0151] In the state shown in Fig. 23, when a predetermined time has elapsed since the start of the conversation, the state automatically returns to the state shown in Fig. 19, as described above, and then the conversation partner is automatically changed. This process is performed multiple times within a time period (e.g., 30 minutes) set by the administrator. When the time set by the administrator has elapsed, the state returns to the state shown in Fig. 19.
[0152] In the above configuration, the moving stage S on which a given character was located and the moving stage S on which another character was located are automatically integrated, and the given character and the other characters are positioned on a single integrated moving stage S. With this configuration, multiple characters can be automatically positioned on a single moving stage S. In other words, characters can be automatically matched. This allows communication not only with a specific character (user), but also with new characters. Therefore, by coming into contact with a new personality of a user with whom the user has not previously communicated, a sense of intimacy can be improved. This makes subsequent communication easier, improves psychological safety, and is expected to lead to improvements in creativity, productivity, and the like.
[0153] Furthermore, when a predetermined time has elapsed while multiple characters are positioned on one moving stage S, a new moving stage S is displayed and one or more characters who have finished their conversation are positioned on the newly displayed moving stage S. In this configuration, the conversation is forcibly ended after the predetermined time has elapsed. In this way, the communication time is limited, so communication with the same character does not drag on for too long. This makes it possible to communicate with many characters.
[0154] Furthermore, a projector board PB that displays information related to the content of the conversation is displayed on one moving stage S where multiple characters are positioned. In this configuration, the topic of the conversation is displayed (provided) on the projector board PB, allowing for smooth and efficient communication within a limited time. Therefore, conversations that touch on the personality of the character's user can be carried out efficiently.
[0155] Furthermore, information relating to a predetermined time may be displayed on one moving stage S where multiple characters are located. In this configuration, for example, the remaining time of the predetermined time can be confirmed, and communication can be carried out taking into consideration time allocation (end time).
[0156] Furthermore, a character operated by a facilitator user may be displayed in the virtual space R. In this configuration, when applied to a meeting attended by many users (for example, a training session, an information exchange meeting, a social gathering, etc.), the meeting can be smoothly conducted because the meeting is led by a facilitator.
[0157] Furthermore, when multiple characters are positioned on one moving stage S, it is possible to exchange cards CD containing at least information that can identify the user of the character. In this configuration, after the conversation ends, it is possible to confirm the character (user) that has had the conversation.
[0158] In the above embodiment, an example in which two characters are positioned on one moving stage S has been described, but three or more characters may be positioned on one moving stage S. The number of characters may be set appropriately by the administrator.
[0159] 10...terminal, 11...input unit, 12...control unit, 13...storage unit, 13P...terminal program, 14...communication unit, 15...output unit, 20...terminal, 30...terminal, 40...server, 42...control unit, 43...storage unit, 43P...server program, 43S...received data, 44...communication unit, 50...operation device, 51A...operation surface, 52...communication unit, 53...input unit, 54...control unit, 55...battery, 100...system, AP...access point, B...body, BS...base station, C...character, C1...character (any character C1...character (other character), C2...character, C3...character, C4...character, C5...character, C6...character, C7...character, CD...card, CL...cloud, CV...cover, DA...display area, E...ear animation, F...finger, H...hand, N...network, P...image, PB...projector board, R...virtual space, S...moving stage, first moving stage, second moving stage, TR...tree, U1...user, U2...user, U3...user, WL...component.
Claims
1. A communication support program that supports communication between multiple characters in a virtual space, comprising: displaying an object in which one or more of the characters are located integrally with the one or more of the characters; and enabling multiple characters located on one of the objects to converse with each other.
2. A communication support program as described in claim 1, wherein, when an operation is received to associate any one of one or more of the characters with another one of the characters different from the character in question, the object in which the any one of the characters was located is integrated with the object in which the other one of the characters was located, and the any one of the characters and the other one of the characters are positioned on the single integrated object.
3. A communication support program as described in claim 2, wherein, when the associating operation is accepted, the user of the other character is prompted to choose whether or not to allow the conversation; if the user of the other character allows the conversation, the object in which any of the characters was located is integrated with the object in which the other character was located, and the any of the character and the other character are positioned on the single integrated object; and if the user of the other character refuses the conversation, the associating operation is invalidated.
4. A communication support program as described in claim 2 or 3, wherein the associating operation is an operation of dragging and dropping any of the characters or the object on which the character is located onto another of the characters or the object on which the character is located.
5. A communication support program as described in any one of claims 2 to 4, wherein when a plurality of said characters are positioned on one of said objects and an operation to end a conversation of one or more of said characters is received, said object is newly displayed and the one or more of said characters that have ended the conversation are positioned on the newly displayed object.
6. The communication support program according to claim 5, wherein the operation for terminating the conversation is an operation of dragging and dropping one or more of the characters from the object to the outside.
7. The communication support program according to any one of claims 1 to 6, wherein the object is movable in the virtual space.
8. A communication support program according to any one of claims 1 to 7, which enables communication between the character located on the object and the character not located on the object.
9. The communication support program according to any one of claims 1 to 8, wherein information relating to participation in a conversation is displayed on the object.
10. A communication support program as described in any one of claims 1 to 9, which automatically moves one or more of the characters to a predetermined location when a predetermined time arrives or a predetermined operation is received when one or more of the characters are located on one of the objects.
11. A communication support program as described in any one of claims 1 to 10, wherein at least one of the multiple characters located on one of the objects is operated by AI.
12. A communication support program as described in claim 1, which automatically integrates the object in which any one of one or more of the characters was located with the object in which another character different from the character was located, and positions the any one of the characters and the other character in the single integrated object.
13. A communication support program as described in claim 12, wherein when a predetermined time has elapsed while a plurality of said characters are positioned on one of said objects, said object is newly displayed and one or more of said characters that have finished their conversation are positioned on said newly displayed object.
14. A communication support program according to claim 12 or 13, further comprising a display unit for displaying information relating to the contents of a conversation in one of said objects in which a plurality of said characters are located.
15. The communication support program according to claim 13, further comprising displaying information relating to the predetermined time in one of the objects in which a plurality of the characters are positioned.
16. A communication support program according to any one of claims 12 to 14, wherein the character operated by a facilitator user is displayed in the virtual space.
17. A communication support program as described in any one of claims 12 to 16, wherein when multiple characters are located on one object, it is possible to exchange cards containing at least information capable of identifying the users of the characters.
18. A communication support program as described in any one of claims 12 to 17, wherein at least one of the multiple characters located on one of the objects is operated by AI.
19. A communication support method for supporting communication between multiple characters in a virtual space, comprising: displaying an object in which one or more of the characters are located integrally with the one or more of the characters; and enabling multiple characters located on one of the objects to converse with each other.
20. A communication support system that supports communication between multiple characters in a virtual space, comprising: an object in which one or more of the characters are located, which is displayed integrally with the one or more of the characters; and multiple characters located on one of the objects are able to converse with each other.
Citation Information
Patent Citations
Communication method and system using multiple avatars simultaneously
JP2020101950A
Systems and methods for triggering livestream communications between users based on proximity-based criteria for avatars within virtual environments that correspond to the users
US20220124285A1
