Information processing system, information processing method, and storage medium

Abstract

To effectively facilitate interaction between avatars.SOLUTION: An information processing system includes: an avatar processing unit configured to process motion of each avatar in a virtual space; an association processing unit configured to associate, with each avatar, text information specified by a user corresponding to the avatar or text information associated with the avatar; and a first determination unit configured to determine a relationship between text information associated with a first avatar and text information associated with a second avatar. In a case where the first avatar and the second avatar have a first positional relationship in the virtual space, the avatar processing unit controls the second avatar or both the first and second avatars to automatically perform a predetermined motion based on a result determined by the first determination unit.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a storage medium. [Background technology]

[0002] A known technique generates display control data that causes the arrangement of multiple virtual characters to conform to an arrangement acquired by a virtual character arrangement acquisition unit until a first trigger such as a conversation start trigger is detected, and generates display control data in which the arrangement in the virtual space of a first virtual character associated with the first trigger is changed in response to the detection of the first trigger. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-010077 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the above-described conventional techniques, it is difficult to effectively promote interaction between avatars.

[0005] Therefore, in one aspect, an object of the present disclosure is to effectively promote interaction between avatars. [Means for solving the problem]

[0006] In one aspect, an information processing system is provided, comprising: an avatar processing unit that processes the movements of each avatar in a virtual space, including specific actions and automatic reactions; a correspondence processing unit that corresponds each avatar with text information specified by a corresponding user or text information associated with the avatar itself; and an activation condition determination unit that determines whether a predetermined activation condition for activating the automatic reaction between avatars is met when a predetermined relationship exists between the attribute of the specific action or the text information associated with a first avatar and the attribute of the specific action or the text information associated with a second avatar; and when the first avatar and the second avatar are in a first predetermined positional relationship in the virtual space, the avatar processing unit causes the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction based on the determination result by the activation condition determination unit. [Effects of the Invention]

[0007] In one aspect, the present disclosure makes it possible to effectively promote interaction between avatars. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a virtual reality generation system according to an embodiment of the present invention. [Figure 2] FIG. 10 is an explanatory diagram of a terminal image viewable through a head-mounted display. [Figure 3] FIG. 10 is an explanatory diagram of a terminal image viewable via a smartphone. [Figure 4] FIG. 1 is an explanatory diagram showing an example of a scene (part 1) in which an automatic reaction is activated in a virtual space. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a scene (part 2) in which an automatic reaction is activated in a virtual space. [Figure 6] FIG. 10 is an explanatory diagram showing another example of a scene (part 1) in which an automatic reaction is activated in a virtual space. [Figure 7]FIG. 10 is an explanatory diagram showing another example of a scene (part 2) in which an automatic reaction is activated in a virtual space. [Figure 8] FIG. 10 is an explanatory diagram of inter-character distances based on tag information. [Figure 9] FIG. 10 is an explanatory diagram of a restriction flag. [Figure 10] FIG. 2 is a schematic block diagram showing the functions of a server device related to automatic reactions. [Figure 11] FIG. 10 is an explanatory diagram showing an example of data in an automatic reaction related data storage unit. [Figure 12] FIG. 4 is an explanatory diagram illustrating an example of data in a user information storage unit. [Figure 13] FIG. 10 is an explanatory diagram of profile information. [Figure 14] FIG. 2 is an explanatory diagram illustrating an example of data in an avatar information storage unit. [Figure 15] 10 is a schematic flowchart illustrating an example of a process executed by a server device in relation to an automatic reaction of an avatar. [Figure 16] 10 is a schematic flowchart showing another example of processing executed by the server device in relation to an automatic reaction of an avatar. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described in detail below with reference to the accompanying drawings. Note that in the accompanying drawings, for ease of viewing, only some of the reference symbols may be used for multiple components with the same attribute.

[0010] An overview of a virtual reality generation system 1 according to one embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of the virtual reality generation system 1 according to this embodiment. Fig. 2 is an explanatory diagram of a terminal image viewable through a head-mounted display.

[0011] The virtual reality generation system 1 includes a server device 10 and one or more terminal devices 20. For simplicity, three terminal devices 20, 20A, and 20B are shown in Fig. 1, but the number of terminal devices 20 may be two or more.

[0012] The server device 10 is an information processing system such as a server managed by an operator that provides one or more virtual realities. The terminal device 20 is a device used by a user, such as a mobile phone, a smartphone, a tablet terminal, a personal computer (PC), a head-mounted display, or a game device. A plurality of terminal devices 20 can be connected to the server device 10 via the network 3, typically in a different manner for each user.

[0013] The terminal device 20 is capable of executing a virtual reality application according to this embodiment. The virtual reality application may be received by the terminal device 20 from the server device 10 or a predetermined application distribution server via the network 3, or may be stored in advance in a storage device provided in the terminal device 20 or in a storage medium such as a memory card readable by the terminal device 20. The server device 10 and the terminal device 20 are communicatively connected via the network 3. For example, the server device 10 and the terminal device 20 cooperate to execute various processes related to virtual reality.

[0014] The terminal devices 20 are connected to each other so as to be able to communicate with each other via the server device 10. In the following, "one terminal device 20 transmits information to another terminal device 20" means "one terminal device 20 transmits information to another terminal device 20 via the server device 10". Similarly, "one terminal device 20 receives information from another terminal device 20" means "one terminal device 20 receives information from another terminal device 20 via the server device 10". However, in a modified example, the terminal devices 20 may be connected to each other so as to be able to communicate with each other without going through the server device 10.

[0015] The network 3 may include a wireless communication network, the Internet, a Virtual Private Network (VPN), a Wide Area Network (WAN), a wired network, or any combination of these.

[0016] In the example shown in Fig. 1, the virtual reality generation system 1 includes studio units 30A and 30B. The studio units 30A and 30B are host-side devices, similar to the host-side terminal device 20A. The studio units 30A and 30B may be located in a studio, room, hall, or the like for content production. Various types of equipment for motion capture may be installed in the studio units 30A and 30B.

[0017] Each studio unit 30 may have the same functions as the terminal device 20A on the host side and / or the server device 10. Hereinafter, when distinguishing between the host side and the participant side, for simplicity of explanation, a mode in which the terminal device 20A on the host side distributes various contents to the terminal devices 20B on each participant side via the server device 10 will be mainly described. However, instead of or in addition to this, the studio units 30A and 30B facing the host side user may have the same functions as the terminal device 20A on the host side, and thereby distribute various contents to the terminal devices 20B on each participant side via the server device 10. In a modified example, the virtual reality generation system 1 may not include the studio units 30A and 30B.

[0018] In the following, the virtual reality generation system 1 realizes an example of an information processing system, but each element of a specific terminal device 20 (see the terminal communication unit 21 to the terminal control unit 25 in FIG. 1) may realize an example of an information processing system, or multiple terminal devices 20 may cooperate to realize an example of an information processing system. Also, the server device 10 may independently realize an example of an information processing system, or the server device 10 and one or more terminal devices 20 may cooperate to realize an example of an information processing system.

[0019] Here, an overview of virtual reality according to this embodiment will be described. The virtual reality according to this embodiment is a virtual reality for any reality, such as education, travel, role-playing, simulation, and entertainment such as games and concerts, and a virtual reality medium such as an avatar is used in executing the virtual reality. For example, the virtual reality according to this embodiment may be realized by a three-dimensional virtual space, various virtual reality media appearing in the virtual space, and various contents provided in the virtual space.

[0020] Virtual reality media are electronic data used in virtual reality, and include any media such as cards, items, points, in-service currency (or virtual reality currency), tokens (e.g., non-fungible tokens (NFTs)), tickets, characters, avatars, parameters, etc. Virtual reality media may also be virtual reality-related information such as level information, status information, parameter information (such as stamina and attack power), or ability information (skills, abilities, spells, jobs, etc.). Virtual reality media are electronic data that can be acquired, owned, used, managed, exchanged, synthesized, enhanced, sold, discarded, or donated by users within virtual reality, but the manner of use of virtual reality media is not limited to those explicitly described in this specification.

[0021] An avatar is typically in the form of a character facing forward, and may have the form of a person, animal, or the like. Avatars can have a variety of appearances (appearances when drawn) by being associated with various avatar items. In the following, due to the nature of avatars, the user and the avatar may be considered to be the same in the description. Therefore, for example, "an avatar does ____" may be synonymous with "a user does ____."

[0022] The user may wear a wearable device on part of their head or face and view the virtual space through the wearable device. The wearable device may be a head-mounted display or a glasses-type device. The glasses-type device may be so-called AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. In either case, the wearable device may be separate from the terminal device 20, or may implement some or all of the functions of the terminal device 20. The terminal device 20 may be implemented by a head-mounted display.

[0023] (Server device configuration) The configuration of the server device 10 will be specifically described. The server device 10 is configured by a server computer. The server device 10 may be realized by a plurality of server computers working together. For example, the server device 10 may be realized by a server computer that provides various types of content, a server computer that functions as various authentication servers, and the like working together. The server device 10 may also include a web server. In this case, some of the functions of the terminal device 20, which will be described later, may be realized by a browser processing HTML documents received from the web server and various programs (Javascript) associated with the HTML documents.

[0024] As shown in FIG. 1, the server device 10 includes a server communication unit 11, a server storage unit 12, and a server control unit 13.

[0025] The server communication unit 11 includes an interface for communicating with an external device wirelessly or via a wired connection to send and receive information. The server communication unit 11 may include, for example, a wireless LAN (Local Area Network) communication module or a wired LAN communication module. The server communication unit 11 is capable of sending and receiving information to and from the terminal device 20 via the network 3.

[0026] The server storage unit 12 is, for example, a storage device, and stores various information and programs required for various processes related to virtual reality.

[0027] The server control unit 13 may include a dedicated microprocessor or a CPU (Central Processing Unit) that implements specific functions by loading specific programs, a GPU (Graphics Processing Unit), etc. For example, the server control unit 13 cooperates with the terminal device 20 to execute a virtual reality application in response to a user input.

[0028] (Terminal Device Configuration) The following describes the configuration of the terminal device 20. As shown in Fig. 1, the terminal device 20 includes a terminal communication unit 21, a terminal storage unit 22, a display unit 23, an input unit 24, and a terminal control unit 25.

[0029] The terminal communication unit 21 includes an interface for communicating with an external device wirelessly or via a wired connection to transmit and receive information. The terminal communication unit 21 may include a wireless communication module, a wireless LAN communication module, or a wired LAN communication module that supports mobile communication standards such as LTE (Long Term Evolution) (registered trademark), LTE-A (LTE-Advanced), a fifth-generation mobile communication system, or UMB (Ultra Mobile Broadband). The terminal communication unit 21 can transmit and receive information to and from the server device 10 via the network 3.

[0030] The terminal storage unit 22 includes, for example, a primary storage unit and a secondary storage unit. For example, the terminal storage unit 22 may include a semiconductor memory, a magnetic memory, an optical memory, or the like. The terminal storage unit 22 stores various information and programs used in virtual reality processing received from the server device 10. The information and programs used in virtual reality processing may be acquired from an external device via the terminal communication unit 21. For example, a virtual reality application program may be acquired from a predetermined application distribution server. Hereinafter, the application program may also be simply referred to as an application.

[0031] The terminal storage unit 22 may also store data for rendering virtual spaces, such as images of indoor spaces such as buildings, outdoor spaces, etc. Note that multiple types of data for rendering virtual spaces may be prepared for each virtual space and used separately.

[0032] Furthermore, the terminal storage unit 22 may store various images (texture images) to be projected (texture mapping) onto various objects arranged in the three-dimensional virtual space.

[0033] For example, the device storage unit 22 stores avatar drawing information relating to an avatar as a virtual reality medium associated with each user. The avatar in the virtual space is drawn based on the avatar drawing information relating to the avatar.

[0034] The device storage unit 22 also stores drawing information related to various objects (virtual reality media) different from avatars, such as various gift objects, buildings, walls, or NPCs (Non-Player Characters). Various objects in the virtual space are drawn based on the drawing information. A gift object is an object corresponding to a gift from one user to another user and is a part of an item. A gift object may be something worn by an avatar (clothes or accessories), decorations (fireworks, flowers, etc.), backgrounds (wallpaper), or the like, or a ticket or the like that can be used in a gacha (lottery). The term "gift" used in the present application has the same concept as the term "token." Therefore, the technology described in the present application can be understood by replacing the term "gift" with the term "token."

[0035] The display unit 23 includes a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 23 is capable of displaying a variety of images. The display unit 23 is configured, for example, with a touch panel, and functions as an interface that detects a variety of user operations. The display unit 23 may be built into a head-mounted display.

[0036] The input unit 24 may include physical keys or any other input interface, such as a pointing device like a mouse. The input unit 24 may also be capable of accepting non-contact user input, such as voice input, gesture input, or gaze input. For gesture input, sensors for detecting various user states (such as image sensors, acceleration sensors, and distance sensors), dedicated motion capture devices integrating sensor technology and cameras, or controllers such as joypads may be used. The gaze detection camera may be located within a head-mounted display. The various user states may be, for example, the user's orientation, position, movement, or the like. In this case, the user's orientation, position, and movement are concepts that include not only the orientation, position, and movement of the user's entire or partial body, such as the face or hands, but also the orientation, position, and movement of the user's gaze, or the like.

[0037] Note that user input using gestures may be used to change the viewpoint of the virtual camera. For example, as schematically shown in Fig. 3, when a user changes the orientation of the terminal device 20 while holding it in their hand, the viewpoint of the virtual camera may change in accordance with the orientation. In this case, even when using a terminal device 20 with a relatively small screen such as a smartphone, a wide viewing area can be ensured in a manner similar to being able to view the surroundings through a head-mounted display.

[0038] The terminal control unit 25 includes one or more processors and controls the overall operation of the terminal device 20.

[0039] The terminal control unit 25 transmits and receives information via the terminal communication unit 21. For example, the terminal control unit 25 receives various information and programs used for various processes related to virtual reality from at least one of the server device 10 and another external server. The terminal control unit 25 stores the received information and programs in the terminal storage unit 22. For example, the terminal storage unit 22 may store a browser (Internet browser) for connecting to a web server.

[0040] The terminal control unit 25 launches a virtual reality application in response to a user operation. The terminal control unit 25 executes various processes related to virtual reality in cooperation with the server device 10. For example, the terminal control unit 25 causes the display unit 23 to display an image of a virtual space. For example, a GUI (Graphical User Interface) that detects a user operation may be displayed on the screen. The terminal control unit 25 can detect a user operation via the input unit 24. For example, the terminal control unit 25 can detect various operations performed by a user's gestures (operations corresponding to a tap operation, a long tap operation, a flick operation, a swipe operation, etc.). The terminal control unit 25 transmits operation information to the server device 10.

[0041] The terminal control unit 25 renders an avatar or the like together with the virtual space (image) and displays the terminal image on the display unit 23. In this case, for example, as shown in FIG. 2, a stereoscopic image for a head-mounted display may be generated by generating images G200 and G201 that are viewed by the left and right eyes, respectively. FIG. 2 schematically shows images G200 and G201 that are viewed by the left and right eyes, respectively. Note that, hereinafter, unless otherwise specified, an image of the virtual space refers to the entire image represented by images G200 and G201. Furthermore, the terminal control unit 25 realizes various movements of the avatar in the virtual space, for example, in response to various operations by the user.

[0042] The virtual space described below is a concept that includes not only an immersive space visible using a head-mounted display or the like, which is a continuous three-dimensional space in which a user can move freely (as in reality) via an avatar, but also a non-immersive space visible using a smartphone or the like as described above with reference to FIG. 3. The non-immersive space visible using a smartphone or the like may be a continuous three-dimensional space in which a user can move freely via an avatar, or a discontinuous two-dimensional space. Hereinafter, when making a distinction, a continuous three-dimensional space in which a user can move freely via an avatar (e.g., a 3D avatar) will be referred to as a "metaverse space," and other virtual spaces (e.g., discontinuous spaces) will be referred to as "non-metaverse spaces."

[0043] Here, in such diverse virtual spaces, a wide variety of users may exist. For example, a broadcasting user refers to a user who transmits information related to video and / or audio. For example, a broadcasting user may be a user who hosts or organizes a solo video broadcast, a collaborative broadcast in which multiple people can participate, a video chat or voice chat in which multiple people can participate and / or watch, or an event (such as a party) in a virtual space in which multiple people can participate and / or watch, in other words, a user who mainly carries out these activities. Therefore, the broadcasting user in this disclosure may also be referred to as a host user, organizer user, or organizer user.

[0044] On the other hand, a viewing user refers to a user who receives information about video and / or audio. However, a viewing user can be a user who not only receives the information but also reacts to it. For example, a viewing user may be a user who watches video distribution or collaborative distribution, or a user who participates in and / or watches video chat, voice chat, or an event. Therefore, viewing users in the present disclosure can also be referred to as guest users, participating users, listeners, spectator users, support users, etc.

[0045] In addition, the information processing system in an embodiment of the present disclosure can be used to provide a next Internet space (metaverse), a digital world in which many people can participate simultaneously and engage in free activities at a level similar to that of real life, such as interacting, working, and playing through character objects (avatars), and in which social activities can take place beyond the gap between reality and virtuality.

[0046] In such a metaverse space, users' avatars can move freely around the world and communicate with each other.

[0047] One of the avatars (character object) in the metaverse space may be configured to be able to distribute video as the character object of the distribution user. In other words, one-to-many video distribution may be possible in the many-to-many metaverse space.

[0048] In such a metaverse space, there may be no particular distinction between broadcasting users and viewing users.

[0049] Next, a characteristic configuration regarding the movement of an avatar in a virtual space will be described with reference to FIGS.

[0050] Figures 4 and 5 are explanatory diagrams showing an example of a scene in which an automatic reaction is activated in a virtual space. Figures 6 and 7 are explanatory diagrams showing another example of a scene in which an automatic reaction is activated in a virtual space. Figure 8 is an explanatory diagram of inter-character distance based on tag information. Figure 9 is an explanatory diagram of a control flag.

[0051] Note that, unless otherwise specified, various objects (e.g., soccer balls, pianos, etc.) appearing in the following description are objects in the virtual space and are different from real objects. Also, various events in the following description are various events in the virtual space (e.g., concerts, etc.) and are different from real-world events. Also, in the following description, when an avatar acquires (obtains) an object, it refers to a transition from a state in which the object is not associated with the avatar to a state in which the object is associated with the avatar. Also, as described above, an avatar and a user associated with the avatar are in a relationship that can be considered to be the same, and therefore, in some cases, the two will not be distinguished from each other. Note that it is also possible for one user to be associated with multiple avatars, but in this case, the user selects one avatar and acts within the virtual space. Therefore, in this case, too, at each point in time, an avatar and a user associated with the avatar are in a relationship that can be considered to be the same.

[0052] In this embodiment, the actions of the avatar in the virtual space include not only normal actions in response to user input from the user, but also predetermined actions based on text information (hereinafter also referred to as "automatic reactions"). Hereinafter, an avatar for which it is determined whether to perform an automatic reaction or an avatar that performs an automatic reaction (an example of a second avatar) is also referred to as a "reaction-based avatar." In this case, the automatic reaction is an automatic action based on text information associated with the reaction-based avatar. The text information may be information representing the preferences of the reaction-based avatar or the user associated with the reaction-based avatar (hereinafter also referred to as a "target user"). For example, the text information may represent favorite people, favorite things, people and things that the user dislikes, favorite attributes, and disliked attributes. The text information may also represent personality (such as "extrovert" or "introvert"). The text information may also include profile information associated with the reaction-based avatar. The text information may also be automatically generated according to various activities of the avatar in the virtual space (such as conversation / utterance information and behavioral history). Further specific examples of text information will be described later.

[0053] An automatic reaction is a reactive action that is automatically performed against other objects, other avatars, etc. Because an automatic reaction is performed automatically, no special user input is required from the user. However, each avatar may be configured to be able to set whether or not to perform an automatic reaction. In this case, an automatic reaction may be automatically performed only for each avatar that is configured to be able to perform an automatic reaction.

[0054] Automatic reactions may be rendered based on animation data prepared in advance. While the present disclosure addresses the challenge of effectively promoting interactions between avatars, tracking the automatic reactions of the advisor to be processed for every specific action taken by other avatars may result in excessive processing load. Furthermore, delays in reactions or the time required to send motion data depending on communication conditions may prevent timely reactions in important situations (e.g., first meetings). Therefore, the types of automatic reactions are limited. In this case, automatic reactions can be executed regardless of user input, eliminating the need to constantly acquire user input (e.g., tracking information via motion capture). This effectively reduces communication load. Furthermore, from the avatar (user) side, automatic reactions are automatically executed, eliminating the need for the user to react or select an emote each time, improving convenience.

[0055] Regarding the rendering of an automatic reaction, motion data indicating how an avatar should move may be transmitted from the server device 10 to the terminal device 20, and the terminal device 20 may move the avatar based on the motion data. For example, the motion data may be blended with animation pre-generated by the server device 10, and the terminal device 20 may track and operate the avatar. Specifically, while moving the avatar with a joystick (on-screen control) on the screen of the terminal device 20, the user may automatically turn their head or gaze toward an avatar of interest. In addition, the server device 10 may automatically ignite (appear) a heart-shaped particle as an emote or symbol called a smiling face mark to the terminal device 20. Furthermore, stamps or GIF animations may be displayed above the head or in a speech bubble. Note that, in the present disclosure, as described below, the determination of whether to execute an automatic reaction is described as being performed by the server device 10. However, the determination may be performed not only by the server device 10 but also by the terminal device 20 or the terminal device 20 itself. In the actual CG space where automatic reactions are rendered, the display resolution and the status of other displayed objects may make it difficult to display the subtleties and movements of the avatar. To address this situation, processing based on the Level of Detail (LoD) is used. This distance, which is a distance that is sufficiently far away that it cannot be distinguished from the display pixels, is used in relation to the three-dimensional distance at which detailed bone information and staged images, such as fingers, gaze, and facial expressions, need to be exchanged. While LoD is automatically or explicitly managed by developers within the game engine, the present disclosure may utilize the Level of Interest (LoI), which is the distance of interest between avatars that serves as a threshold for automatic reaction operations, in addition to the relationship between the three-dimensional positions at which the avatars should actually be displayed, to efficiently cull information, compress communication traffic, reduce rendering costs, and improve cache hit rates.

[0056] Drawing based on animation data may be applied only to a predetermined part of each part of the avatar. The predetermined part may be a part related to an automatic reaction. For example, if the automatic reaction is a gesture of raising one's hand to greet, the predetermined part may be the avatar's hand. In this case, other parts may be drawn using inverse kinematics. With this configuration, parts other than the predetermined part that are unrelated to the movement of the predetermined part may also be drawn based on user operation, thereby enabling the individuality of each avatar to be expressed. Note that a portion of the predetermined part may be a single part, or may include multiple parts rather than a single part, and animation may be performed in conjunction with multiple parts. For example, a gesture of greeting may include not only the hand but also the head, such as tilting one's head while waving.

[0057] In addition, if the specifications allow for the rendering of the avatar's facial expressions (for example, the movement of the eyes, mouth, etc.) and associated parts (for example, hair, hair accessories, etc.), the avatar's facial expressions and associated part movements during an automatic reaction may be similarly rendered by animation or may be rendered based on user operation. Furthermore, in the case of specifications that allow for the rendering of the avatar's facial expressions (for example, the movement of the eyes, mouth, etc.), the avatar's facial expressions during an automatic reaction may be rendered based on the user's facial image information or tracking information acquired by motion capture (for example, face capture).

[0058] An automatic reaction may be executed (i.e., automatically) regardless of subsequent user input from the user when a predetermined activation condition is met. The predetermined activation condition may be a condition based on text information associated with the reaction-subject avatar or the target user (hereinafter, represented by "text information associated with the reaction-subject avatar"). In other words, whether the predetermined activation condition is met may be determined based on the text information associated with the reaction-subject avatar.

[0059] An automatic reaction may be any action intended to promote interaction between avatars in a virtual space. For example, an automatic reaction may include a praising action such as clapping, a smiling action, a greeting action, a friendly pose (such as a thumbs-up sign) or action, a call action, giving a gift or a letter, or other friendly emote action, or any combination of these actions.

[0060] For example, if a reaction-dominant avatar automatically reacts by smiling at another avatar, it may encourage interaction between the two avatars, even if it is their first meeting. For example, the other avatar may notice the reaction-dominant avatar smiling and speak to it, potentially starting a conversation.

[0061] In this way, the automatic reaction may be an action directed at a specific partner's avatar (hereinafter also referred to as "partner's avatar") (an example of a first avatar). Therefore, the predetermined activation condition may be determined when the avatar that is the subject of the reaction and another avatar that could be the partner's avatar are in a predetermined positional relationship (hereinafter also referred to as "first predetermined positional relationship") in the virtual space. This allows the automatic reaction to be executed at an appropriate timing while reducing the processing load.

[0062] In this case, the first predetermined positional relationship is arbitrary, but may include being within a predetermined distance from each other (for example, a three-dimensional spatial distance or a two-dimensional distance in virtual space), or a positional relationship in which the reaction-subject avatar is visible to the other avatar that could be the counterpart avatar (i.e., a positional relationship in which the reaction-subject avatar is visible from the other avatar). Alternatively, the first predetermined positional relationship may include a positional relationship in which the reaction-subject avatar is not visible from the current viewpoint of the other avatar that could be the counterpart avatar, but the reaction-subject avatar becomes visible from the other avatar when the viewpoint of the other avatar is changed on the spot (for example, when the direction of the face or the direction of the body is changed).

[0063] 4 and 5 show a scene in which an automatic reaction can be performed. In this case, avatar A1, the reaction-subject avatar, is standing in a school hallway, while another avatar A2 is walking toward him. In this situation, avatar A1 and avatar A2 are meeting for the first time or are not yet close. In this situation, avatar A1 and avatar A2 are assumed to be interested in the same things (e.g., music, bands, etc.) based on the text information associated with each avatar. In FIG. 5, when a predetermined activation condition is met, avatar A1, the reaction-subject avatar, raises its hand to greet the person as an automatic reaction.

[0064] In the scenes shown in Figures 4 and 5, the predetermined activation condition may be satisfied when the relationship between the character information associated with avatar A1 and the character information associated with avatar A2 is a predetermined relationship. In this case, the predetermined relationship may include a relationship in which the character information associated with both avatars has something in common. Hereinafter, the predetermined activation condition whose success or failure is determined based on the relationship between the character information in this manner will also be referred to as a "predetermined first activation condition."

[0065] When avatar A1 performs an automatic reaction in this manner, the field of view of the virtual camera associated with avatar A2 may be changed so that avatar A1, the reacting subject, is included in the field of view. That is, a change in the viewpoint of avatar A2 may be realized so that avatar A1, the reacting subject, can be seen clearly. Alternatively, when an automatic reaction is initiated, guidance information informing avatar A2 of the position of avatar A1, the reacting subject, may be generated. Such changes in the field of view of the virtual camera and guidance information are effective in the metaverse space. That is, by increasing the degree of freedom in changing the viewpoint, changes in the field of view and guidance information become effective in the metaverse space, which displays a large amount of information.

[0066] In this way, avatar A2 notices avatar A1, which is primarily a reaction-oriented avatar, and this promotes interaction between avatar A1 and avatar A2. Because avatar A1 and avatar A2 share the same interests (e.g., music, bands, etc.) based on the text information associated with each, they are likely to become friends if they talk, which effectively promotes interaction. Furthermore, by making it easier to find avatars (users) with common text information (and therefore common interests), it is possible to effectively increase each user's frequency of use of the virtual space and their rate of continued use.

[0067] 4 and 5, avatar A2 may automatically respond to avatar A1, which is primarily responsible for the reaction. In this case, the automatic response is not one-way, and interaction can be promoted efficiently.

[0068] In this embodiment, when avatar A1 performs an automatic reaction, tag information (#band, #music, #drum in FIG. 5) may be displayed above avatar A1's head or the like. The tag information may be displayed in a manner recognizable only to avatar A2, or in a manner recognizable to other nearby avatars. The tag information may be a part or all of the text information, or may be generated through conversion, processing, or the like based on a part or all of the text information. When non-shared tag information is displayed in addition to shared tag information, the shared tag information may be displayed in a prominent manner. Alternatively, the tag information displayed may be only the tag information shared with avatar A2. In either case, even if avatar A2 meets avatar A1 for the first time, avatar A2 can immediately recognize the commonalities between avatar A1 and itself, thereby promoting subsequent conversation, etc. Note that the display of such tag information may be performed as part of the automatic reaction itself or as part of the automatic reaction. Alternatively, such tag information may be displayed before avatar A1 performs an automatic reaction (see FIG. 3).

[0069] Furthermore, when an automatic reaction is performed by avatar A1, message information ("Be a friend" in FIG. 5) may be displayed above the head of avatar A1. In this case, the content of such message information may be determined automatically or may be set in advance by avatar A1. In either case, this allows avatar A2 to easily understand the intention of avatar A1 (i.e., the intention behind the automatic reaction performed by avatar A1), thereby promoting subsequent conversation, etc. Note that the display of such message information may be performed as the automatic reaction itself or as a part of the automatic reaction.

[0070] In the examples shown in FIGS. 4 and 5, the automatic reaction is an action of raising one's hand to greet the other avatar, but the type of automatic reaction to be executed is not limited to this. The automatic reaction may be selected (determined) based on a predetermined rule from among a plurality of types of actions, including a favorable reaction toward the other avatar. For example, the type of automatic reaction may be determined according to the character information associated with avatar A1 and avatar A2. For example, the type of automatic reaction may be determined according to character information having the above-mentioned predetermined relationship among all the character information associated with avatar A1 or avatar A2. In this case, an automatic reaction according to the personality of the avatar becomes possible, and the variety of automatic reactions can be increased, and the personality of the avatar can be reflected in the automatic reaction.

[0071] Furthermore, the automatic reaction may be changed depending on the strength of the commonality between the text information. For example, in the examples shown in FIGS. 4 and 5, the automatic reaction may be different when the relationship between the text information associated with avatar A1 and the text information associated with avatar A2 is a first predetermined relationship and when it is a second predetermined relationship. For example, in the second predetermined relationship, the automatic reaction may be emphasized in a more prominent manner than in the first predetermined relationship. For example, in the first predetermined relationship, the automatic reaction may be a gesture of raising one's hand in greeting, whereas in the second predetermined relationship, the automatic reaction may be a gesture of raising one's hand and waving in greeting. Or, even if the gesture is the same, in the second predetermined relationship, the gesture may be a more extensive gesture than in the first predetermined relationship. In either case, the second predetermined relationship may be a relationship in which the text information has more in common than in the first predetermined relationship (for example, a relationship in which the distance between characters is closer, as described below).

[0072] 6 and 7 show another scene in which an automatic reaction can be performed. In this case, avatar B5 is performing a specific action (in this case, juggling a soccer ball) on a stage in a place such as a school gymnasium, with four other avatars B1 to B4 standing nearby. In this situation, avatar B5 and avatars B1 to B4 are meeting for the first time or are not yet familiar with each other. Also, in this situation, among avatars B1 to B4, avatars B2 and B3 are interested in avatar B5's specific action (e.g., soccer or sports in general) based on the text information associated with each of them. In FIG. 7, the predetermined activation condition is satisfied only for avatars B2 and B3, causing avatars B2 and B3, which are the reaction-subject avatars, to clap their hands as an automatic reaction to avatar B5, the other avatar.

[0073] In the scenes shown in FIGS. 6 and 7 , the predetermined activation condition may be satisfied when the relationship between the attribute of the specific action of avatar B5 and the text information associated with avatar B2 or avatar B3 is a predetermined relationship. In this case, the predetermined relationship may include a relationship in which the attribute of the specific action and the text information have something in common. Hereinafter, the predetermined activation condition, the success or failure of which is determined based on the relationship between the attribute of the specific action and the text information, will also be referred to as a "predetermined second activation condition" to distinguish it from the above-mentioned predetermined first activation condition. Furthermore, although not shown, the following activation condition may be referred to as a "predetermined third activation condition." That is, while the first activation condition is satisfied when the relationship between text information is a predetermined relationship, the second activation condition is satisfied when the relationship between the specific action and the text information is a predetermined relationship, the third activation condition is satisfied when the relationship between specific actions is a predetermined relationship. 6 and 7, an automatic reaction due to the third activation condition is an action in which avatar B3 and avatar B5 perform a specific action (in this case, juggling a soccer ball) and then, after the specific action is completed, they look at each other and greet each other, nod at the moment when the conversation ends and there is silence (for example, at a pause in the conversation), or applaud each other.In addition to this, it is also possible to change the state machine used to control the animation, detect when a specific state is reached and output the same state at the right time, or have the surrounding avatar B2 applaud after the target avatars B3 and B5 have completed their specific action.

[0074] When avatar B2 and avatar B3 perform an automatic reaction in this manner, the field of view of the virtual camera associated with avatar B5 may be changed so that avatar B2 and avatar B3 are in the field of view. That is, a change in the viewpoint of avatar B5 may be realized so that avatar B2 and avatar B3, who are the main reacters, can be clearly seen. Alternatively, when an automatic reaction is initiated, guidance information may be generated for avatar B5 informing it of the positions of avatar B2 and avatar B3, who are the main reacters (or the presence of avatars B2 and B3, who are clapping). Such changes in the field of view of the virtual camera and guidance information are effective in the metaverse space. That is, as described above, by increasing the degree of freedom in changing the viewpoint, changes in the field of view and guidance information become effective in the metaverse space, which displays a large amount of information.

[0075] In this way, avatar B5 notices avatars B2 and B3, which are mainly reactive, and this promotes interaction between avatars B2, B3, and B5. In this case, avatars B2, B3, and B5 are likely to become friends if they talk, as they are interested in the same things (for example, soccer or sports in general) based on the text information associated with each of them, and this promotes interaction efficiently.

[0076] Furthermore, in the scenes shown in FIGS. 6 and 7, when avatar B2 performs an automatic reaction, the field of view of the virtual camera associated with avatar B3 may be changed so that avatar B2 is within the field of view. Alternatively, guidance information informing avatar B3 of the presence of another avatar B2 clapping may be generated for avatar B3. The reverse is also true. That is, when avatar B3 performs an automatic reaction, the field of view of the virtual camera associated with avatar B2 may be changed so that avatar B3 is within the field of view. Alternatively, guidance information informing avatar B2 of the presence of another avatar B3 clapping may be generated for avatar B2. This can promote interaction between avatar B2 and avatar B3. That is, because avatar B2 and avatar B3 are interested in the same thing (e.g., soccer or sports in general) based on the text information associated with each of them, they are likely to become friends if they talk, which can efficiently promote interaction. Furthermore, by making it easier to find avatars (users) with common interests, the frequency and continuity of each user's use of the virtual space can be effectively increased.

[0077] In the examples shown in FIGS. 6 and 7, the automatic reaction is a clapping action, but the type of automatic reaction is not limited to this. The automatic reaction to a specific action may be selected (determined) based on a predetermined rule from among multiple types of actions, including favorable reactions toward the other avatar. For example, the type of automatic reaction may be determined according to the attributes of the specific action. For example, in this case, since the specific action is juggling a soccer ball, the automatic reaction may be a soccer-related or sports-related automatic reaction. Alternatively, the type of automatic reaction may be determined according to character information that has a predetermined relationship with the attribute of the specific action of avatar B5, among all the character information associated with avatar B2 or avatar B3. In this case, an automatic reaction according to the personality of the avatar is possible, and the variety of automatic reactions can be increased, and the personality of the avatar can be reflected in the automatic reaction.

[0078] Here, the specific behavior of an avatar may be executed (rendered) based on tracking information (for example, tracking information of each body part by motion capture) related to the user corresponding to that avatar, but is preferably rendered using animation data. In this case, since there is no need to track all of the user's movements, the processing load can be reduced and the specific behavior of each avatar can be efficiently expressed (rendered).

[0079] In this embodiment, the automatic reaction described above is executed when a predetermined activation condition is met, as described above. While the automatic reaction is effective for promoting interaction between avatars, as described above, it may have the trade-off of increasing the processing load. Therefore, it is desirable to efficiently determine whether the predetermined activation condition is met.

[0080] Therefore, whether or not a predetermined activation condition is satisfied is preferably determined using inter-character distances based on character information. For example, in a configuration in which a predetermined activation condition for a reaction subject avatar is satisfied when the relationship between character information associated with the reaction subject avatar and character information associated with another avatar that can be the partner avatar is a predetermined relationship, the presence or absence of the predetermined relationship may be determined using inter-character distances based on the character information. In this case, for example, word embedding in natural language processing such as Word2vec may be used. In Word2vec, word vectors of characters (words) included in the character information that are located close to each other in a vector space may be determined to have a predetermined relationship. By using such natural language processing, the presence or absence of a predetermined relationship (and therefore the success or failure of the predetermined activation condition) can be efficiently determined. Furthermore, the success or failure of the predetermined activation condition can be efficiently determined for a variety of languages.

[0081] 4 and 5, the predetermined relationship may include a relationship in which the inter-character distance (relationship) between the character information associated with avatar A1 and the character information associated with avatar A2 is within a predetermined distance, as described above. In the case of Word2vec, a predetermined first activation condition may be met when there are one or more pairs of word vectors that are located close to each other in the vector space among the characters (words) included in each piece of character information.

[0082] 6 and 7, the predetermined second activation condition may be, as described above, whether the inter-character distance (relationship) between the characters related to the specific behavior of avatar A5 and the character information associated with each of avatars B1 to B4 is within a predetermined distance as a predetermined relationship. In the case of Word2vec, for each of avatars B1 to B4, it may be determined whether there are one or more pairs of word vectors that are located nearby in the vector space among the characters (words) included in the character information for each of avatars B1 to B4 for the characters related to the specific behavior of avatar A5.

[0083] The character information used to calculate the inter-character distance may include tag information. FIG. 8 shows an example of tag information associated with multiple avatars. In FIG. 8, tag[avatar C1] represents tag information associated with avatar C1. Also, in FIG. 8, information such as [***, 100] indicates that "***" is a character, and may be, for example, an emoji as shown in FIG. 8. Emojis can be universal and improve convenience. Furthermore, a numerical value such as "100" represents a weight associated with the character, and a higher weight indicates a stronger representation of the avatar's preferences. Therefore, weights may be used in such a manner that the closer the "distance" between characters with higher weights, the greater the commonality. In this case, avatars C1 and C2 have a close inter-character distance (relationship) based on the tag information, and a predetermined activation condition is more likely to be met.

[0084] In this way, a predetermined activation condition based on tag information may be satisfied when tag information has a predetermined relationship. In this case, the predetermined relationship may be variable in multiple stages. For example, the predetermined relationship may be expressed as a perfect match of the wording related to the tag information as the strictest relationship, or may be expressed as a looser relationship that takes into account the meaning of the tag. For example, #music and #singer may be determined to have a predetermined relationship. Furthermore, the predetermined activation condition based on tag information may be cross-checked across multiple languages. In this case, for example, #baseball and #baseball may be determined to have a predetermined relationship.

[0085] Here, we will discuss the issue of "identification and target number of users." While it may be better to separate words with the same meaning, such as #baseball, #baseball, #professionalbaseball, #baseball, and #◎ (ball mark), by different languages ​​or cultures, it may also be better to identify and design them to facilitate smooth interactions between users with similar interests. While this condition can be met by building a database based on language or cultural classification, it is also reasonable to adjust the database based on the number of target users you want to connect. For example, if you want to efficiently group (identify) users with baseball-related hobbies in a world with 10 logged-in users (5 native Japanese speakers and 5 non-native speakers), you can imagine a line dividing the group in half in semantic space, 5 vs. 5, and then divide them by the language / words separated by that line. In the example of 10 users, the 5 Japanese speakers would have tags like {baseball, professional baseball, baseball}, and the other 5 would have tags like {#baseball, #◎ (ball mark), #MLB}, so it would be reasonable to actively group (identify) the former. In machine learning, this separation can be performed using a general algorithm such as SVM (Support Vector Machine).

[0086] In addition to being used to calculate the distance between characters, such tag information can also be used to promote interaction between avatars by being displayed in association with the avatars above their heads, as described above with reference to Fig. 5. For example, tag information associated with one avatar may be displayed toward another avatar only when any one or any combination of two or more of the following condition elements C1 to C4 is satisfied: Condition element C1: One avatar and one other avatar are associated with common tag information. Condition element C2: One avatar and another avatar are in a predetermined positional relationship. Condition element C3: One avatar and one other avatar have a specific intimate relationship. Condition element C4: Another avatar has specified one avatar and requested the display of tag information. Here, in condition element C2, the predetermined positional relationship may include, similar to the first predetermined positional relationship described above, being within a predetermined distance, a positional relationship in which one avatar is visible to another avatar that could potentially be the other avatar (i.e., a positional relationship in which one avatar is visible to another avatar), etc. Also, in condition element C3, the specific intimate relationship may include a relationship in which one person is registered in user information (described below) as friend information of the other person, a relationship in which both people are registered in user information (described below) as friend information of each other, a relationship in which they are registered in user information (described below) as follower and followedee, or any combination thereof.

[0087] If a beginner mode is provided, tag information may be displayed above the head of an avatar for which the beginner mode is set. In this case, the tag information may include a beginner mark. Alternatively, the beginner mark may be attached to the avatar's clothing, etc. An experienced user may also select the beginner mode, in which case the user's avatar and name may also be displayed. If multiple modes are set, such as an intermediate mode and an advanced mode in addition to the beginner mode, each mode may be manually selected by the user, or may be automatically set depending on the number of days since registration with the service or the number of logins.

[0088] When a condition including the condition element C4 is satisfied and tag information associated with an avatar is displayed to another avatar, information indicating this may be fed back to the avatar. In this case, the feedback may be realized via a display and / or vibration (for example, vibration of the input unit 24) indicating that the tag information is being viewed. This may be true not only for tag information but also for profile information, etc.

[0089] Incidentally, although the automatic reactions described above can promote interaction between avatars in a virtual space with a reduced processing load as described above, as the density of avatars in the virtual space increases, the frequency of automatic reactions is likely to increase accordingly, which may result in a corresponding increase in processing load.

[0090] Thus, in this embodiment, interactions between avatars may be restricted automatically and / or based on user settings.

[0091] A method for restricting interactions between avatars may be implemented using restriction flags such as block flags and mute flags. Restriction flags such as block flags and mute flags may be managed for each avatar in association with other avatars. FIG. 9 is an explanatory diagram of restriction flags, and shows an example of the states of restriction flags associated with other avatars D2, D3, and D4 for avatar D1. The restriction flags shown in FIG. 9 are in a state set by avatar D1, as schematically shown in the lower part of FIG. 9. In this case, avatar D1 can arbitrarily set restriction flags for interactions with other avatars D2, D3, D4, etc. In the state shown in FIG. 9, avatar D1 has the mute flag turned on for avatar D2, has neither the block flag nor the mute flag turned on for avatar D3, and has the block flag turned on for avatar D4.

[0092] In this case, avatar D2 with its mute flag set to on is essentially a non-existent avatar to avatar D1, and avatar D1 cannot see, talk to, or hear it. In other words, avatar D2 is not included in the list (browsable user ledger) within the virtual space in which avatar D1 is active. Therefore, there is no need to promote interaction between avatar D1 and avatar D2, and automatic reactions are not performed between them. This eliminates the need for various processes (e.g., determining whether a first predetermined positional relationship exists or whether a predetermined activation condition is met) for performing automatic reactions between avatar D1 and avatar D2, thereby efficiently reducing the processing load. Furthermore, avatar D1 can set avatars with which it does not want to interact as "non-existent avatars" based on its settings, improving convenience.

[0093] Here, we will explain more about muting and blocking. Avatar D1 is the avatar that uses muting and blocking, and avatar D2 is the avatar that is muted or blocked, but avatar D1's automatic reactions do not activate for avatar D2. First, regarding muting, avatar D1 cannot see avatar D2, but avatar D2 can see avatar D1. In other words, avatar D2 is not included in the decision on whether avatar D1 will perform an automatic reaction. Regarding voice conversations, avatar D2 can hear what avatar D1 says, but avatar D1 cannot hear what avatar D2 says. Regarding text chat, avatar D1 does not display the text of avatar D2's comments, and mentions and notifications are not processed. Avatar D2 only sees avatar D1 and is not permitted to interact with it at all, but avatar B2 does not need to be aware that he is muted. Avatar D1 may unmute avatar D2 from the mute list at any time. Regarding blocking, avatar D1 explicitly rejects avatar D2. Until avatar D1 removes avatar D2 from the block list, avatar D2 cannot search for avatar D1. To avatar D2, avatar D1 is considered a nonexistent avatar. Even if both avatars are in the same world, avatar D1 will not be displayed on avatar D2's terminal device 20, and neither avatar D1 nor any of its speech or text will be displayed. Muting and blocking may be set based on the preferences and experience of each avatar (user), or may be set broadly as default settings. Furthermore, muting and blocking may be set by the operator of the virtual reality service for purposes such as communication control, events such as festivals, or tutorials (individualized instruction) for novice users. Furthermore, muting and blocking may be dynamically controlled based on the aforementioned LoI (Loc of Interest) between avatars, taking into account the CG space in which automatic reactions are depicted.

[0094] On the other hand, unlike avatar D2, avatar D4 with its block flag set to on is an avatar that exists for avatar D1, but avatar D4 cannot interact with avatar D1, such as by talking to or approaching avatar D1. Furthermore, avatar D4 cannot view the profile information or tag information of avatar D1. For example, the above-mentioned condition element C4 is not satisfied for avatar D4 with respect to the tag information of avatar D1. Furthermore, promoting interaction between avatar D1 and avatar D4 is substantially unnecessary, and automatic reactions are not performed between avatar D1 and avatar D4. This eliminates the need for various processes for performing automatic reactions between avatar D1 and avatar D4, thereby efficiently reducing the processing load. Note that avatar D4 may be able to recognize that avatar D1 has set its block flag to on.

[0095] When the restriction flag is used in this way, automatic reactions are more likely to be executed only between avatars that have a high effect of promoting interaction. This makes it possible to efficiently reduce the processing load without unnecessarily restricting the opportunities for automatic reactions to be executed.

[0096] In this embodiment, the on / off state of the restriction flag may be changeable in response to a user input by the corresponding user and / or may be automatically changed based on text information. For example, in this case, if the text information associated with an avatar includes "bragging" as a dislike, avatars that brag a lot may be restricted from interacting with the avatar. Note that whether an avatar brags a lot may be determined based on conversation / utterance information related to the avatar.

[0097] In this embodiment, the automatic reaction may be limited depending on the processing load, such as the density of avatars in the virtual space. For example, in an area of ​​the virtual space where the density of avatars is high (e.g., a concert event venue, etc.), the automatic reaction described above may be limited. In this case, the automatic reaction function may be turned off, or the predetermined activation condition may be automatically changed so that the predetermined activation condition is less likely to be met. For example, in an area of ​​the virtual space where the density of avatars is high, if the relationship between text information associated with one avatar and text information associated with more than a predetermined number of avatars is a first predetermined relationship, the processing load may be significantly high. In this case, the automatic reaction may be executed only for avatars associated with text information that satisfies a second predetermined relationship, among the avatars exceeding the predetermined number. Note that in this case, the second predetermined relationship may be a relationship in which the text information has more in common than the first predetermined relationship. Alternatively, in an area of ​​a virtual space where the avatar density is high, if the relationship between the text information associated with one avatar and the text information associated with more than a predetermined number of avatars is a predetermined relationship, an automatic reaction may be executed only for avatars selected randomly or according to a predetermined rule from among the more than a predetermined number of avatars. In addition to the above, the automatic reaction may be limited depending on the characteristics of the area in the virtual space. For example, in a venue such as a virtual cinema, the automatic reaction function may be turned on in a lobby where audience conversation is permitted, and turned off in a theater room where audience members are concentrated in the cinema, even if there are many empty seats.

[0098] Next, with reference to FIG. 10 and subsequent figures, an example of the functional configuration of each of the server device 10 and the terminal device 20 related to the above-mentioned automatic reaction of the avatar will be described.

[0099] Fig. 10 is a schematic block diagram showing the functions of the server device 10 related to the above-mentioned automatic reactions. Fig. 11 is an explanatory diagram showing an example of data in the automatic reaction related data storage unit 142. Fig. 12 is an explanatory diagram showing an example of data in the user information storage unit 144. Fig. 13 is an explanatory diagram of profile information. Fig. 14 is an explanatory diagram showing an example of data in the avatar information storage unit 146. In Fig. 11 (and similarly in Fig. 12 below), "***" indicates that some information is stored, and "..." indicates that similar information is repeatedly stored.

[0100] As shown in FIG. 10, the server device 10 includes an automatic reaction related data storage unit 142, a user information storage unit 144, an avatar information storage unit 146, a user input acquisition unit 150, an avatar processing unit 152, a matching processing unit 153, an interaction restriction processing unit 154, an activation condition determination unit 155, a reaction type determination unit 156, a parameter calculation unit 158, a field of view processing unit 160, and a user information disclosure control unit 162.

[0101] 10, the automatic reaction related data storage unit 142, the user information storage unit 144, and the avatar information storage unit 146 can be realized by the server storage unit 12 of the server device 10 shown in Fig. 1. Furthermore, the functions of each unit from the user input acquisition unit 150 to the user information disclosure control unit 162 can be realized by the server control unit 13 and the server communication unit 11 of the server device 10 shown in Fig. 1.

[0102] Automatic reaction related data related to each of the above-mentioned automatic reactions is stored in the automatic reaction related data storage unit 142. The automatic reaction related data may include, for example, automatic reaction attributes, activation condition information, animation data, and playback information for each automatic reaction ID, as shown in Fig. 11 .

[0103] The automatic reaction ID is an ID that is automatically generated when each automatic reaction is generated. The automatic reaction attribute represents the attribute of the automatic reaction, and the attribute of the automatic reaction may include information about the intention of the automatic reaction, such as a greeting or praise.

[0104] The activation condition information may include information that represents the above-mentioned predetermined activation condition. Note that the activation condition information may be different for each automatic reaction ID, but is preferably the same (common) for multiple automatic reaction IDs. In this case, when a certain predetermined activation condition is met, an automatic reaction related to one or more automatic reaction IDs among the multiple automatic reaction IDs can be executed, thereby diversifying the automatic reactions.

[0105] The animation data includes animation data for drawing the automatic reactions described above. As described above, the animation data may be data related only to the movement of a predetermined part of the avatar that is the subject of the reaction. In this case, the animation data can be generated so that it can be shared among multiple avatars, thereby improving data efficiency. The animation data may be generated based on tracking information obtained when an actual person (user) performs an actual movement related to the automatic reaction. The animation data may also be selected according to the avatar's type (F type <female type>, M type <male type>, etc.) or attributes (personality, age (generation), etc.).

[0106] The playback information includes basic information indicating the start and end timings of animation data playback. The start timing of animation data playback may be determined based on the distance between the avatar performing the reaction and the other avatar. Alternatively, the start timing of animation data playback may be synchronized with the timing at which it is determined that a predetermined activation condition has been met, and may be substantially the same as the timing of this determination, for example.

[0107] Information about each user is stored in the user information storage unit 144. The information about each user may be generated, for example, when the user is registered, and may thereafter be updated as appropriate. For example, in the example shown in FIG. 12, the user information storage unit 144 associates, for each user ID, a username, an avatar ID, profile information, conversation / utterance information, activity information, character information (tag information), friend information, preference information, restriction flag information, item information, and automatic reaction availability information.

[0108] The user ID is automatically generated when the user registers.

[0109] The user name is a name registered by each user and is arbitrary.

[0110] The avatar ID is an ID that represents an avatar used by a user. The avatar ID may be associated with avatar drawing information (see FIG. 14) for drawing the corresponding avatar. The avatar drawing information associated with one avatar ID may be added or edited based on input from the corresponding user.

[0111] As described above, multiple avatar IDs may be associated with one user. In this case, each avatar ID may be associated with profile information, conversation / utterance information, activity information, text information, friend information, preference information, restriction flag information, item information, and automatic reaction availability information.

[0112] The profile information is information representing a user profile (or an avatar profile) and is generated based on information input by a user. For example, the profile information may be generated based on information input by a user as shown in Fig. 13. Alternatively, the profile information may be selected via a user interface generated on the terminal device 20 and provided to the server device 10 by a JSON (JavaScript Object Notation) request or the like.

[0113] In this embodiment, the profile information may include information indicating its update mode (such as update frequency and update amount). The update mode of the profile information may differ for each avatar, and some avatars may update their profile information relatively frequently, while others may only update their profile information once and never at all.

[0114] Conversation / utterance information represents information about the content of utterances made by a corresponding avatar in a virtual space. The conversation / utterance information may include conversation information related to conversations between avatars and utterance information related to utterances other than conversations (for example, monologues or utterances during broadcasts). In this case, the conversation information and utterance information may be managed separately or together. The conversation / utterance information may be text data. The conversation / utterance information may also include information indicating the language spoken (for example, a locale ID). The conversation / utterance information may also include information on the first-person pronoun, language, dialect, etc. The conversation / utterance information may be raw data before processing (i.e., a raw utterance log), or may also include text data (converted into characters). In the latter case, storage space can be made more efficient.

[0115] The conversation / utterance information may also include keyword information. In this case, the keyword information may be, for example, information representing keywords in a conversation that took place in a talk room to which the corresponding user has joined. The keyword information may be generated using Text-to-Speech (TTS) or the like. The keyword information may accurately represent the preferences of the user, and may be used in processing by the association processing unit 153, which will be described later.

[0116] The activity information includes information representing various activity histories and behavioral histories in the virtual space. The various activities may include not only special activities such as participating in and hosting events, but also information representing ordinary activities (behaviors) such as accessing the virtual space and the time spent in the virtual space. The various activities may also include activities such as creating objects in the virtual space.

[0117] The character information is as described above, and may represent the preferences of the avatar, etc. The character information may be updated in response to input from the user, etc. The character information may also be automatically extracted based on conversation / utterance information, behavior history, etc. Some or all of the character information associated with one avatar may be associated with the one avatar and displayed in the form of tag information, as described above with reference to Figures 4 and 5, etc. Conversely, tag information may function as character information. The character information may also be in the form of metadata.

[0118] The friend information may be information (e.g., user ID) that can identify users who are friends. The friend information may include information such as followers, as described above. The friend information may include information about people with whom users have had past conversations, information about avatars that have performed past automatic reactions, and information indicating the presence or absence and degree of interactions or friendships between users.

[0119] The preference information represents the preferences of the corresponding user and may represent preferences in general or preferences related to conversations (e.g., topics of conversation). The preference information is optional and may include the user's preferred language settings or preferred keywords. The user may also be able to set their favorite people and things, disliked people and things, etc. in advance, in which case the preference information may include such settings. Note that the preference information may include information similar to the user profile information, and may therefore be integrated with the user profile information.

[0120] The restriction flag information indicates the setting state of the restriction flag described above with reference to Fig. 9. As described above, the setting state of the restriction flag may be changeable by the user.

[0121] The item information represents a predetermined item associated with the corresponding user. The predetermined item may be one element of the virtual reality medium described above and may be a virtual reality medium that represents the user's preference. For example, the predetermined item may include an avatar item, any object such as various musical instruments or a ball.

[0122] The automatic reaction availability information is information set by the user regarding whether or not the above-mentioned automatic reaction is to be executed. The automatic reaction availability information can be used to distinguish between users who do not want the above-mentioned automatic reaction to be executed and users who do. Note that the user may be able to set whether or not to execute the automatic reaction for each scene, situation, location, date and time.

[0123] The avatar information storage unit 146 stores avatar drawing information for drawing each user's avatar. In the example shown in FIG. 14 , the avatar drawing information associates each avatar ID with a facial part ID, a hairstyle part ID, a clothing part ID, and the like. Appearance-related part information, such as the facial part ID, hairstyle part ID, and clothing part ID, is a parameter that characterizes an avatar and may be selected by each user. For example, multiple types of appearance-related information, such as the facial part ID, hairstyle part ID, and clothing part ID, associated with an avatar may be prepared. Furthermore, with regard to the facial part ID, a part ID may be prepared for each type, such as face shape, eyes, mouth, and nose, and the information related to the facial part ID may be managed as a combination of IDs of each part constituting the face. In this case, each avatar can be drawn not only on the server device 10 but also on the terminal device 20 side based on the appearance-related IDs associated with each avatar ID.

[0124] The user input acquisition unit 150 acquires various user inputs by each user input via the input unit 24 of the terminal device 20. The various inputs are as described above, and may include tracking information acquired by motion capture.

[0125] When a predetermined condition is met, the avatar processing unit 152 causes the avatar to perform an automatic reaction. The automatic reaction is as described above.

[0126] Avatar processing unit 152 includes an avatar movement processing unit 1521 and a drawing processing unit 1522 .

[0127] Avatar movement processing unit 1521 determines the movement of each avatar (change in position, movement of each part, etc.) based on various inputs from each corresponding user for each avatar.

[0128] The rendering processor 1522 generates an image of a virtual space including an avatar, which is to be viewed on the terminal device 20 (an image for a terminal). The rendering processor 1522 generates an image for each avatar (an image for the terminal device 20) based on the values ​​of the imaging parameters (position, line of sight direction, angle of view, etc.) of the virtual camera associated with each avatar. Note that the values ​​of the imaging parameters of the virtual camera associated with one avatar may basically correspond to the values ​​of various parameters related to the field of view of the one avatar (for example, eye position, line of sight direction, visual acuity, etc.) (except when the field of view processor 160, described later, intervenes). In this embodiment, the rendering processor 1522 renders an avatar when performing an automatic reaction. In this case, the rendering processor 1522 can render an avatar performing an automatic reaction with a reduced processing load based on the automatic reaction-related data in the automatic reaction-related data storage unit 142 shown in FIG. 11.

[0129] The association processing unit 153 associates each avatar with text information related to the preferences and character (hereinafter, represented by "preferences") of the corresponding user or avatar itself. In this embodiment, the association processing unit 153 generates and updates user information in the user information storage unit 144, thereby associating each avatar with text information related to the preferences of the corresponding user or avatar itself.

[0130] For example, the association processing unit 153 may associate text information specified by the corresponding user with each avatar. Alternatively, the association processing unit 153 may associate text information corresponding to various information related to the preferences of the corresponding user or avatar with each avatar based on the various information. In the example shown in FIG. 12, the various information related to the preferences of the corresponding user or avatar includes profile information, friend information, and preference information. Furthermore, conversation / utterance information, activity information, and item information may represent the preferences of the corresponding avatar. Therefore, the various information related to the preferences of the corresponding user or avatar may include at least one of conversation / utterance information, activity information, and item information.

[0131] The character information associated by association processing unit 153 in this manner may be manually edited or otherwise corrected by the corresponding user.

[0132] The interaction restriction processing unit 154 determines, for each avatar, other avatars that can interact with the corresponding avatar (other avatars that can be partner avatars) based on text information etc. associated with the corresponding avatar. For example, the interaction restriction processing unit 154 may determine other avatars that can interact with the corresponding avatar based on at least one of the profile information associated with the avatar, the conversation / utterance information by the avatar, the item information associated with the avatar, and the behavior history of the avatar in the virtual space.

[0133] For example, if the behavioral history is used, an avatar that has a strong tendency to use specific negative emotes or give negative gifts, an avatar that tends to cause trouble to other avatars, or an avatar that has received a certain number of warnings, alerts, or reports from other avatars or third avatars because their reactions to specific actions violate rules or manners may be deemed to be ineligible for interaction with other avatars. Furthermore, if the behavioral history is used, the range of other avatars that can be interacted with may be expanded for users who pay a lot of in-service currency, based on their consumption (billing status), etc.

[0134] In this case, the interaction restriction processing unit 154 may determine, based on the character information associated with a corresponding avatar, an avatar that is associated with character information that satisfies the specified activation conditions for the automatic reaction described above as another avatar with which interaction is possible.

[0135] Furthermore, the interaction restriction processing unit 154 may determine, based on the friend information, an avatar whose follower has set the mute flag or block flag as an avatar that cannot be interacted with. In other words, in this way, the flag restriction information for an avatar may inherit the same flag restriction information associated with the follower of the avatar.

[0136] In this way, when the interaction restriction processing unit 154 determines other avatars that can interact with a given avatar, it generates or updates restriction flag information related to the given avatar from the user information stored in the user information storage unit 144. Note that the restriction flag information associated with the given avatar may be changeable as appropriate based on a user input by the user associated with the given avatar.

[0137] The activation condition determination unit 155 determines whether the above-mentioned predetermined activation condition is met for each pair of avatars.

[0138] In this embodiment, the activation condition determination unit 155 includes a determination target determination unit 1550, a first determination unit 1551, a second determination unit 1552, and a third determination unit 1553.

[0139] The determination target determining unit 1550 determines the avatar that will be the target of determination of the above-mentioned predetermined activation condition.

[0140] The judgment object determining unit 1550 includes a first judgment object determining unit 15501 and a second judgment object determining unit 15502 .

[0141] The first judgment target determination unit 15501 determines, for each avatar, other avatars (hereinafter also referred to as "first judgment target avatars") that can be the counterpart avatar when one avatar is determined as the reaction-subject avatar. Note that the reaction-subject avatars may be limited to only avatars that wish to perform an automatic reaction, based on the automatic reaction availability information of the user information in the user information storage unit 144 described above. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0142] The first judgment target determination unit 15501 may limit the first judgment target avatar based on the positional relationship of the avatar of one reaction subject with respect to the avatar of the one reaction subject. For example, the first judgment target determination unit 15501 may limit the first judgment target avatar to only avatars whose positional relationship with the avatar of the one reaction subject is the first predetermined positional relationship described above. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0143] Furthermore, the first judgment target determination unit 15501 may limit the first judgment target avatar for a single reaction subject avatar based on the restriction flag information for the single reaction subject avatar. Specifically, the first judgment target determination unit 15501 may limit the first judgment target avatar based on the restriction flag information for the single reaction subject avatar so that only avatars for which the mute flag or block flag is not set to on are included in the first judgment target avatar. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0144] When one avatar performs the above-mentioned specific behavior, the second judgment target determination unit 15502 determines an avatar that can be the avatar that will be the subject of the reaction (hereinafter also referred to as the "second judgment target avatar"). Note that in this case as well, the avatar that will be the subject of the reaction may be limited to only avatars that wish to execute an automatic reaction, based on the automatic reaction availability information of the user information in the user information storage unit 144. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0145] The second judgment target determination unit 15502 may limit the second judgment target avatars based on their positional relationship with respect to an avatar performing a specific behavior. For example, the second judgment target determination unit 15502 may limit the second judgment target avatars to only those avatars whose positional relationship with respect to the avatar performing the specific behavior is the first predetermined positional relationship described above. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0146] Furthermore, the second judgment target determination unit 15502 may limit the second judgment target avatars for an avatar performing a specific behavior based on the restriction flag information for the avatar performing the specific behavior. Specifically, the second judgment target determination unit 15502 may limit the second judgment target avatars based on the restriction flag information for the avatar performing the specific behavior so that only avatars for which the mute flag or block flag is not set to on are included in the second judgment target avatars. In this case, the processing load on the activation condition determination unit 155 can be efficiently reduced.

[0147] When one avatar is determined to be a reaction-subject avatar, the first determination unit 1551 determines the above-mentioned predetermined first activation condition for each first judgment target avatar determined by the first judgment target determination unit 15501. The predetermined first activation condition is as described above with reference to Figs. 4 and 5, etc.

[0148] When one avatar performs the above-mentioned specific behavior, the second determination unit 1552 determines the above-mentioned predetermined second activation condition for each second avatar to be judged determined by the second judgment target determination unit 15502. The predetermined second activation condition is as described above with reference to Figs. 6 and 7, etc.

[0149] The third determination unit 1553 determines whether the positional relationship between the avatars related to the reaction subject avatar and the other avatar is a first predetermined positional relationship. The first predetermined positional relationship is as described above. The third determination unit 1553 may make the determination based on at least one parameter, for example, the distance between the avatars related to the reaction subject avatar and the other avatar, the presence or absence of an obstacle (another object) between the avatars, the presence or absence of the reaction subject avatar within the field of view of the other avatar, and whether a transition from a state in which the reaction subject avatar is located outside the field of view of the other avatar to a state in which the other avatar is located within the field of view of the reaction subject avatar is possible. Note that, when an obstacle is taken into consideration, it may be determined whether the reaction subject avatar is within the field of view of the other avatar due to the obstacle. Furthermore, the transition from a state in which the reaction subject avatar is located outside the field of view of the other avatar to a state in which the other avatar is located within the field of view of the reaction subject avatar may include a transition enabled by processing by the field of view processing unit 160, which will be described later.

[0150] When a reaction subject avatar performs the above-mentioned automatic reaction, the reaction type determination unit 156 selects the type of the automatic reaction from among a plurality of selectable types. In this embodiment, it is assumed that a plurality of types of automatic reactions are prepared. That is, as described above with reference to FIG. 11, the same activation condition information is associated with a plurality of automatic reaction IDs.

[0151] The selection rule by the reaction type determination unit 156 is arbitrary and may be, for example, a random selection rule. However, the reaction type determination unit 156 may select a type of automatic reaction corresponding to the character information that has satisfied a predetermined first activation condition, based on the character information. For example, if the character information that has satisfied the predetermined first activation condition includes the word "band," an automatic reaction type that suggests music or a band may be selected. Similarly, the reaction type determination unit 156 may preferably select a type of automatic reaction corresponding to the character information and / or the specific behavior that has satisfied a predetermined second activation condition, based on the attributes of the character information and / or the specific behavior at that time.

[0152] The parameter calculation unit 158 ​​calculates the values ​​of various parameters that can be used to determine whether a predetermined activation condition is met. The various parameters may include parameters related to text information.

[0153] In this embodiment, the parameter calculation unit 158 ​​includes a first calculation unit 1581 and a second calculation unit 1582.

[0154] The first calculation unit 1581 calculates the inter-character distance between the character information associated with the reaction subject avatar and the character information associated with the first determination target avatar. The method for calculating the inter-character distance may be as described above. In this case, since character information tends to be composed of short words, the processing load for calculating the inter-character distance can be efficiently reduced.

[0155] The second calculation unit 1582 calculates the inter-character distance between the characters related to the specific behavior and the character information associated with the second determination target avatar. The method for calculating the inter-character distance may be as described above. In this case, since character information inherently tends to be composed of short words, the processing load for calculating the inter-character distance can be efficiently reduced.

[0156] Furthermore, characters related to specific actions may be converted into characters in advance according to the attributes of the specific actions. In this case, characters related to specific actions may be stored for each specific action ID. This configuration eliminates the need for a process to derive characters related to specific actions, thereby reducing the processing load.

[0157] The visual field processing unit 160 executes a process to temporarily change the values ​​of the imaging parameters (position, line of sight direction, angle of view, etc.) of the virtual camera associated with a specific avatar, regardless of direct user input from the user.

[0158] In this embodiment, the visual field processing unit 160 includes a first visual field change processing unit 1601 and a second visual field change processing unit 1602 .

[0159] When a reaction-subject avatar automatically reacts to a partner avatar, the first view change processing unit 1601 changes the values ​​of the imaging parameters of the virtual camera associated with the partner avatar so that the reaction-subject avatar is within the view of the partner avatar. As a result, as described above, the partner avatar can easily notice the reaction-subject avatar, and interaction between the two can be efficiently promoted.

[0160] When the positional relationship between an avatar and another avatar transitions to a second predetermined positional relationship, the second field of view change processing unit 1602 temporarily changes the field of view of one of the avatars so that the other avatar appears within the field of view of the other avatar. The second predetermined positional relationship may be the same as the first predetermined positional relationship described above, or may include a positional relationship that is farther away than the first predetermined positional relationship described above. For example, when a new avatar enters a talk room, the fields of view of all or some of the avatars in the talk room are temporarily changed, making it easier for the new avatar to appear within the field of view of the avatars in the talk room. This efficiently promotes interaction between the two avatars. Note that in this embodiment, the field of view of the avatar is temporarily changed to highlight the avatar that should be noticed, etc. However, instead of or in addition to this, the avatar that should be noticed, etc. may be emphasized, or a guide display may be output to guide the location of the avatar that should be noticed, etc. (the location of an avatar outside the field of view). In addition to or instead of the guidance display, this guidance may be performed by vibrating a controller such as a joypad held by the user, or even by sound.

[0161] The user information disclosure control unit 162 allows other avatars (users) to view a portion of the user information in the user information storage unit 144 (hereinafter also referred to as "viewing user information"). The viewing user information may include profile information, text information, etc. For example, the profile information may be made viewable in response to a predetermined user input (hereinafter also referred to as a "viewing request"). In this case, the user information disclosure control unit 162 may make the viewing user information related to an avatar whose mute flag is on inaccessible to users who have turned on the mute flag for that avatar. Furthermore, the user information disclosure control unit 162 may make the viewing user information related to users who have turned on the block flag for that avatar inaccessible to avatars whose block flag is on.

[0162] A viewing request may be generated in any manner. For example, when using a terminal device 20 such as a smartphone, a viewing request for viewing user information related to one avatar (a viewing request by another avatar) may be generated by tapping the one avatar on the screen. Alternatively, a viewing request may be generated by holding a magnifying glass over the one avatar. Furthermore, in the metaverse space, a viewing request for viewing user information related to one avatar (a viewing request by another avatar) may be generated by directing one's gaze toward the one avatar for a predetermined period of time or more. Alternatively, a viewing request may be generated by making eye contact (i.e., eye contact) with the one avatar.

[0163] The above-described division of functions between the server device 10 and the terminal device 20 is merely an example, and as described above, various changes are possible. That is, some or all of the functions of the server device 10 may be realized by the terminal device 20 as appropriate.

[0164] Next, with reference to FIG. 15 and subsequent figures, an example of the operation of the virtual reality generation system 1 related to the automatic reaction of the avatar described above will be described.

[0165] Fig. 15 is a schematic flowchart showing an example of processing executed by the server device 10 in relation to the above-mentioned automatic reaction of an avatar. The processing shown in Fig. 15 is processing related to one specific avatar (hereinafter also referred to as "processing target avatar") and may be repeatedly executed at predetermined intervals. The processing shown in Fig. 15 may be executed in parallel for each avatar in the virtual space.

[0166] In step S2700, server device 10 acquires a user input for the current cycle from the user corresponding to the avatar to be processed.

[0167] In step S2701, the server device 10 updates the position and field of view (virtual camera) of the avatar to be processed based on the user input acquired in step S2700.

[0168] In step S2702, it is determined whether or not the setting is in a state where an automatic reaction is possible based on the automatic reaction possibility information. The automatic reaction possibility information is as described above in relation to the user information in the user information storage unit 144. If the determination result is "YES", the process proceeds to step S2703; otherwise, the process proceeds to step S2714.

[0169] In step S2703, the server device 10 extracts one or more other avatars located within the field of view of the target avatar. Although not shown, if there are no other avatars, the processing for the current cycle may be ended.

[0170] In step S2704, the server device 10 determines whether or not one or more of the other avatars extracted in step S2703 has a mute flag set to on, based on the restriction flag information (see FIG. 12) associated with the avatar to be processed. In a modified example, avatars with a mute flag set to on may be initially excluded from processing. If the determination result is "YES," the process proceeds to step S2705; otherwise, the process proceeds to step S2706.

[0171] In step S2705, the server device 10 excludes the avatars whose mute flags are set to ON from the processing targets of step S2706 and thereafter, and executes the avatar mute process, which, as described above, is a process for essentially eliminating the existence of the avatar to be processed.

[0172] In step S2706, the server device 10 determines whether or not one or more of the other avatars extracted in step S2703 has a block flag set to ON, based on the restriction flag information (see FIG. 12) associated with the avatar to be processed. If the determination result is "YES," the process proceeds to step S2707; otherwise, the process proceeds to step S2708.

[0173] In step S2707, the server device 10 executes an avatar blocking process to exclude the avatar whose block flag is set to ON from the processing targets of the following step S2708 and thereafter, and to exclude the avatar from the interaction targets. As described above, the avatar blocking process is a process to prevent the processing target avatar from being approached or spoken to.

[0174] In step S2708, the server device 10 determines whether or not another avatar exists in the first predetermined positional relationship with the avatar to be processed. If the determination result is "YES", the process proceeds to step S2709; otherwise, the process proceeds to step S2714.

[0175] In step S2709, the server device 10 extracts character information associated with another avatar that is located in a first predetermined positional relationship with the processing target avatar. Note that if there are multiple other avatars that are located in the first predetermined positional relationship with the processing target avatar, the processes from step S2709 to step S2713 may be executed sequentially for each of the other avatars.

[0176] In step S2710, the server device 10 calculates the inter-character distance based on the character information associated with the processing target avatar and the character information extracted in step S2709. The method for calculating the inter-character distance is as described above.

[0177] In step S2711, the server device 10 determines whether the inter-character distance is equal to or less than a predetermined distance L0. The predetermined distance L0 may be a fixed value or a variable value. For example, the predetermined distance L0 may be set by the user. If the determination result is "YES," it is determined that a predetermined first activation condition is satisfied, and the process proceeds to step S2712; otherwise, the process proceeds to step S2714.

[0178] In step S2712, the server device 10 determines the type of automatic reaction. In this case, the server device 10 may determine the type of automatic reaction based on the character information for which the inter-character distance is determined to be equal to or less than the predetermined distance L0 in step S2711. For example, if the character information for which the inter-character distance is determined to be equal to or less than the predetermined distance L0 is the same as "alcohol," the type of automatic reaction may be an action related to a "drinking gesture." Furthermore, if the character information for which the inter-character distance is determined to be equal to or less than the predetermined distance L0 is the same as "basketball (sports)," the type of automatic reaction may be an action related to a "shooting gesture." Furthermore, in a modified example, the strength of the action related to the automatic reaction may be changed depending on the inter-character distance. For example, if the type of automatic reaction is a hand-waving action, the type of automatic reaction may be determined in such a manner that the smaller the inter-character distance, the greater the hand-waving action selected.

[0179] In step S2713, the server device 10 causes the processing target avatar to execute the type of automatic reaction determined in step S2712. As mentioned above, the object of the present disclosure is to effectively promote interaction between avatars, but the type of automatic reaction is limited because tracking the processing target advisor's automatic reactions to all specific actions by other avatars could result in an excessively high processing load, and if reactions are delayed or it takes time to send motion data depending on communication conditions, it could be impossible to react in a timely manner in important situations (for example, when meeting for the first time).

[0180] In step S2714, the server device 10 causes the processing target avatar to perform an action in accordance with the user input.

[0181] FIG. 16 is a schematic flowchart showing another example of processing executed by the server device 10 in connection with the above-described automatic reaction of an avatar. Similar to the processing shown in FIG. 15, the processing shown in FIG. 16 is processing related to one specific avatar (hereinafter also referred to as "processing target avatar") and may be repeatedly executed at predetermined intervals. The processing shown in FIG. 16 may be executed in parallel for each avatar in the virtual space. Furthermore, the processing shown in FIG. 16 may be executed in parallel with the processing shown in FIG. 15 described above.

[0182] The processing from step S2900 to step S2907 may be the same as step S2700 to step S2707 shown in FIG. 15, and the description thereof will be omitted.

[0183] In step S2908, the server device 10 determines whether another avatar within the field of view of the target avatar is (or is) performing a specific action. The specific action is as described above. If the determination result is "YES," the process proceeds to step S2909; otherwise, the process proceeds to step S2914. Note that, in making this determination, even if the specific action of another avatar is outside the field of view of the target avatar at a given time, the specific action may be subject to an automatic reaction if it can be brought into view by changing the viewpoint of the target avatar.

[0184] In step S2909, the server device 10 determines the attribute of the specific action and determines (or derives) characters according to the attribute. For example, if the specific action is "juggling a soccer ball," the characters according to the attribute of the specific action may be "soccer" and / or "juggling." Furthermore, if the soccer ball has a distinctive design, characters according to the design may be determined. Note that, if there are multiple other avatars performing the specific action within the field of view of the processing target avatar, the processes from step S2909 to step S2913 may be executed sequentially for each of the other avatars.

[0185] In step S2910, the server device 10 calculates the inter-character distance based on the character information associated with the processing target avatar and the character information extracted in step S2909. The method for calculating the inter-character distance is as described above.

[0186] In step S2911, the server device 10 determines whether the inter-character distance is equal to or less than a predetermined distance L0. The predetermined distance L0 may be a fixed value or a variable value. For example, the predetermined distance L0 may be set by the user. If the determination result is "YES," it is determined that a predetermined first activation condition is satisfied, and the process proceeds to step S2912; otherwise, the process proceeds to step S2914.

[0187] In step S2912, the server device 10 determines the type (attribute) of the automatic reaction. In this case, the server device 10 may determine the type of the automatic reaction based on the character information (characters corresponding to the attribute of the specific behavior) whose inter-character distance is determined to be equal to or less than the predetermined distance L0 in step S2911. This is the same as step S2712 described above with reference to FIG. 15.

[0188] In step S2913, the server device 10 causes the processing target avatar to perform the automatic reaction of the type determined in step S2912.

[0189] In step S2914, the server device 10 causes the processing target avatar to perform an action in accordance with the user input.

[0190] Although each embodiment has been described in detail above, it is not limited to a specific embodiment, and various modifications and changes are possible within the scope of the claims. It is also possible to combine all or some of the components of the above-described embodiments. [Explanation of symbols]

[0191] 1 Virtual reality generation system 3 Network 10 Server device 11 Server Communication Unit 12 Server storage unit 13 Server control unit 20, 20A, 20B terminal equipment 21 Terminal communication unit 22 Terminal memory section 23 Display section 24 Input section 25 Terminal control unit 142 Automatic reaction related data storage unit 144 User information storage unit 146 Avatar Information Storage Unit 150 User input acquisition unit 152 Avatar Processing Unit 1521 Avatar movement processing unit 1522 drawing processing unit 153 Correspondence processing unit 154 AC Restriction Processing Unit 155 Activation condition determination section 1550 Judgment object determination unit 15501 First Judgment Object Determination Unit 15502 Second Judgment Object Determination Unit 1551 1st Judgment Department 1552 Second Judgment Department 1553 Third Judgment Department 156 reaction type determination unit (first and second predetermined action determination unit) 158 Parameter Calculation Unit 1581 First Calculation Section 1582 Second Calculation Unit 160 Visual field processing section 1601 First field of view change processing section 1602 Second field of view change processing section 162 User information disclosure control section

Claims

1. an avatar processing unit that processes the movements of each avatar, including specific actions and automatic reactions, in the virtual space; an association processing unit that associates, with each avatar, character information designated by a corresponding user or character information associated with the avatar itself; an activation condition determination unit that determines whether a predetermined activation condition for activating the automatic reaction between avatars is satisfied when a relationship between an attribute of the specific action or the character information associated with a first avatar and an attribute of the specific action or the character information associated with a second avatar satisfies a predetermined relationship; An information processing system in which, when the first avatar and the second avatar are present in a first predetermined positional relationship in the virtual space, the avatar processing unit causes the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction based on the determination result by the activation condition determination unit.

2. The information processing system of claim 1, further comprising a reaction type determination unit that determines the type of the automatic reaction from among a plurality of types based on at least one of the attributes of the specific action or the character information associated with the first avatar and the attributes of the specific action or the character information associated with the second avatar.

3. The information processing system according to claim 1 or 2, wherein the automatic reactions are composed of a plurality of types including, in addition to the movements of the first avatar and / or the second avatar, emotes for the first avatar and / or the second avatar, the appearance of particles, and the display of message information, tag information, and speech bubbles.

4. The information processing system of claim 1, wherein the avatar processing unit causes the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction when the activation condition determination unit determines, as a first activation condition among the specified activation conditions, that the relationship between the character information associated with the first avatar and the character information associated with the second avatar is a specified relationship.

5. The information processing system of claim 1, wherein the avatar processing unit causes the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction when the activation condition determination unit determines, as a second activation condition among the predetermined activation conditions, that the relationship between the attributes of the specific action performed by the first avatar and the text information associated with the second avatar is a predetermined relationship.

6. The information processing system of claim 1, wherein the avatar processing unit causes the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction when the activation condition determination unit determines, as a third activation condition among the predetermined activation conditions, that the relationship between the attributes of the specific action performed by the first avatar and the attributes of the specific action performed by the second avatar is a predetermined relationship.

7. a parameter calculation unit that calculates an inter-character distance between a character or the character information related to the specific action associated with the first avatar and a character or the character information related to the specific action associated with the second avatar; The information processing system according to claim 1 , wherein the activation condition determination unit determines whether or not the predetermined relationship exists based on the inter-character distance.

8. The information processing system according to claim 7 , wherein the predetermined relationship includes a relationship having a common point.

9. 9. The information processing system according to claim 7, wherein the predetermined relationship can be divided into a plurality of categories, and when the predetermined relationship is divided into a first predetermined relationship and a second predetermined relationship having different commonalities, the avatar processing unit varies the automatic reaction depending on the category.

10. A non-transitory computer-readable storage medium, comprising: Avatar processing that processes the movements of each avatar in the virtual space, including specific actions and automatic reactions; a correspondence process for associating each avatar with character information designated by a corresponding user or character information associated with the avatar itself; a program for causing a computer to execute an activation condition determination process for determining whether a predetermined activation condition for activating the automatic reaction between avatars is satisfied when a relationship between an attribute of the specific action or the character information associated with a first avatar and an attribute of the specific action or the character information associated with a second avatar satisfies a predetermined relationship; A storage medium, wherein the avatar processing includes, when the first avatar and the second avatar are present in a predetermined positional relationship in the virtual space, causing the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction based on the determination result of the activation condition determination processing.

11. Executing avatar processing to process the movements of each avatar, including specific actions and automatic reactions, in the virtual space; performing an association process for associating each avatar with character information designated by a corresponding user or character information associated with the avatar itself; a predetermined activation condition for activating the automatic reaction between avatars, the predetermined activation condition being satisfied when a relationship between an attribute of the specific action or the character information associated with a first avatar and an attribute of the specific action or the character information associated with a second avatar is a predetermined relationship; and The avatar processing is an information processing method executed by a computer, which includes, when the first avatar and the second avatar are present in a predetermined positional relationship in the virtual space, causing the second avatar, or both the first avatar and the second avatar, to execute the automatic reaction based on the determination result of the activation condition determination processing.

Images