Information processing system, information processing method, and information processing program

Abstract

We generate terminal images that effectively support various user activities via display media in virtual space. [Solution] An information processing system is disclosed that includes a drawing unit that draws a display image for a terminal that includes one or more display media located in a virtual space, an acquisition unit that acquires user input, a state switching unit that switches the state of the display medium between multiple states, the multiple states including a first state in which the position of the display medium can be changed from its current position and a second state in which the position of the display medium cannot be changed from its current position, a position changing unit that changes the position of the display medium associated with a single user in the virtual space based on a first input from the single user acquired by the acquisition unit when the state of the display medium is in the first state, and a viewpoint switching unit that switches the viewpoint from the first viewpoint to the second viewpoint when the state of the display medium is switched from the first state to the second state by the state switching unit.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] A technology is known in which an avatar and a virtual camera are placed in a virtual space and an image for a device is generated according to the settings of the virtual camera. In this case, the visual coordinate system of the virtual camera is linked to the visual coordinate system of the head-mounted display worn by the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-179184 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-described conventional technology, the viewpoint of the virtual camera is fixed and linked to the viewpoint of an avatar (an example of a display medium) in the virtual space, making it difficult to generate images for a device in a manner that can effectively support various activities of a user via an avatar in the virtual space according to the attributes of each activity. For example, it is difficult to effectively support avatar movement input in the virtual space while also effectively supporting avatar-mediated conversations between users in the virtual space.

[0005] Therefore, in one aspect, an object of the present invention is to generate a terminal image that effectively supports various activities of a user via a display medium in a virtual space. [Means for solving the problem]

[0006] On one side, a rendering unit that renders a display image for a terminal including one or more display media located in a virtual space; an acquisition unit that acquires input from a user; The state of the display medium associated with one user is At least a first state and a second state a state switching unit that switches between a plurality of states, the display medium is capable of rendering a state in which the user is seated on a specific object in a virtual space; the drawing unit draws the display medium in the second state in a manner representing the seated state. An information processing system is provided. [Effects of the Invention]

[0007] According to one aspect, the present invention makes it possible to generate a terminal image that effectively supports various activities of a user via a display medium in a virtual space. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a virtual reality generation system according to an embodiment of the present invention. [Figure 2] FIG. 1 is an explanatory diagram of an example of a virtual space that can be generated by a virtual reality generation system. [Figure 3] FIG. 10 is a diagram illustrating an example of a terminal image. [Figure 4] FIG. 2 is an explanatory diagram of camera parameters of a virtual camera. [Figure 5] FIG. 10 is a functional block diagram of a server device related to an interaction promotion function. [Figure 6] FIG. 2 is an explanatory diagram of data in a user database. [Figure 7] FIG. 2 is an explanatory diagram of data in an avatar database. [Figure 8] FIG. 10 is an explanatory diagram of data in a group state storage unit. [Figure 9] FIG. 10 is an explanatory diagram of position / orientation information of a user avatar. [Figure 10] FIG. 10 is a diagram showing an example of a terminal image depicting a user avatar in a seated position. [Figure 11] 4 is a diagram showing a group information display area, which is a part (part Q1) of the terminal image shown in FIG. 3. FIG. [Figure 12]10 is an explanatory diagram of an example of a method for calculating values of various camera parameters of a virtual camera by a normal parameter calculation unit. FIG. [Figure 13] 10A and 10B are diagrams illustrating an example of a method for calculating values of various camera parameters of a virtual camera by a sitting parameter calculation unit. [Figure 14] FIG. 10 is an explanatory diagram illustrating a case where the front direction is set based on the user avatar of the conversation partner. [Figure 15] FIG. 10 is an explanatory diagram of a terminal image based on a viewpoint in a third-person viewpoint mode. [Figure 16] 10 is an explanatory diagram of a terminal image in a transition state based on a calculation result by a transition parameter calculation unit. FIG. [Figure 17] FIG. 10 is an explanatory diagram of a terminal image based on a viewpoint in a first-person viewpoint mode. [Figure 18] 10 is a schematic flowchart illustrating an example of operations related to an interaction promotion function. [Figure 19] 18 is a schematic flowchart showing an example of a terminal image drawing process (step S1822 in FIG. 18). [Figure 20] FIG. 10 is a functional block diagram of a terminal device related to an avatar movement guidance function. [Figure 21] FIG. 10 is an explanatory diagram of a part direction operation input using a gesture. [Figure 22] 21 is a schematic flowchart showing an example of operation by the terminal device shown in FIG. 20, which is related to a terminal image generating unit. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Overview of Virtual Reality Generation System) An overview of a virtual reality generation system 1 according to one embodiment of the present invention will be described with reference to Figure 1. Figure 1 is a block diagram of the virtual reality generation system 1 according to this embodiment. The virtual reality generation system 1 includes a server device 10 and one or more terminal devices 20. For simplicity, Figure 1 illustrates three terminal devices 20, but the number of terminal devices 20 may be two or more.

[0010] The server device 10 is, for example, a server managed by an operator that provides one or more virtual realities. The terminal device 20 is a device used by a user, such as a mobile phone, a smartphone, a tablet terminal, a PC (Personal Computer), a head-mounted display, or a game device. A plurality of terminal devices 20 can be connected to the server device 10 via the network 3, typically in a different manner for each user.

[0011] The terminal device 20 is capable of executing a virtual reality application according to this embodiment. The virtual reality application may be received by the terminal device 20 from the server device 10 or a predetermined application distribution server via the network 3, or may be stored in advance in a storage device provided in the terminal device 20 or in a storage medium such as a memory card readable by the terminal device 20. The server device 10 and the terminal device 20 are communicatively connected via the network 3. For example, the server device 10 and the terminal device 20 cooperate to execute various processes related to virtual reality.

[0012] The network 3 may include a wireless communication network, the Internet, a Virtual Private Network (VPN), a Wide Area Network (WAN), a wired network, or any combination of these.

[0013] Here, an overview of virtual reality according to this embodiment will be described. The virtual reality according to this embodiment is a virtual reality for any reality, such as education, travel, role-playing, simulation, and entertainment such as games and concerts, and a virtual reality medium such as an avatar is used in executing the virtual reality. For example, the virtual reality according to this embodiment is realized by a three-dimensional virtual space, various virtual reality media appearing in the virtual space, and various contents provided in the virtual space.

[0014] Virtual reality media are electronic data used in virtual reality, and include any media, such as cards, items, points, in-service currency (or virtual reality currency), tokens (e.g., Non-Fungible Tokens (NFTs)), tickets, characters, avatars, parameters, etc. Virtual reality media may also be virtual reality-related information such as level information, status information, virtual reality parameter information (such as stamina and attack power), or ability information (skills, abilities, spells, jobs, etc.). Virtual reality media are electronic data that can be acquired, owned, used, managed, exchanged, synthesized, enhanced, sold, discarded, or donated by users within virtual reality, but the manner of use of virtual reality media is not limited to those explicitly described in this specification.

[0015] (Server device configuration) The configuration of the server device 10 will be specifically described. The server device 10 is configured by a server computer. The server device 10 may be realized by a plurality of server computers working together. For example, the server device 10 may be realized by a server computer that provides various types of content, a server computer that functions as various authentication servers, and the like working together. The server device 10 may also include a web server. In this case, some of the functions of the terminal device 20, which will be described later, may be realized by a browser processing HTML documents received from the web server and various programs (Javascript) associated with the HTML documents.

[0016] The server device 10 includes a server communication unit 11, a server storage unit 12, and a server control unit 13.

[0017] The server communication unit 11 includes an interface for communicating with an external device wirelessly or via a wired connection to send and receive information. The server communication unit 11 may include, for example, a wireless LAN (Local Area Network) communication module or a wired LAN communication module. The server communication unit 11 is capable of sending and receiving information to and from the terminal device 20 via the network 3.

[0018] The server storage unit 12 is, for example, a storage device, and stores various information and programs necessary for various processes related to virtual reality. For example, the server storage unit 12 stores a virtual reality application.

[0019] The server storage unit 12 also stores data for rendering virtual spaces, such as images of indoor spaces such as buildings, outdoor spaces, etc. Note that multiple types of data for rendering virtual spaces may be prepared for each virtual space and used separately.

[0020] The server storage unit 12 also stores various images (texture images) to be projected (texture mapped) onto various objects placed in the three-dimensional virtual space.

[0021] For example, the server storage unit 12 stores drawing information of a user avatar M1 (an example of a display medium) as a virtual reality medium associated with each user. Note that a user is a user of the virtual reality generation system 1. In addition to general users, users may include staff users who operate avatars in association with the operator of the virtual reality generation system 1, guest users who provide content in the virtual space, and the like. The user avatar M1 is drawn in the virtual space based on the drawing information of the user avatar M1.

[0022] The server storage unit 12 also stores drawing information related to various objects different from the user avatar M1, such as buildings, walls, trees, NPCs (Non-Player Characters), etc. The various objects in the virtual space are drawn based on the drawing information.

[0023] Hereinafter, an object corresponding to any virtual reality medium (e.g., a building, a wall, a tree, or an NPC) different from the user avatar M1 and drawn in the virtual space will also be referred to as a second object M3. In this embodiment, the second object M3 may include an object that is fixed in the virtual space or an object that is movable in the virtual space. The second object M3 may also include an object that is always placed in the virtual space or an object that is placed only when a predetermined placement condition is met.

[0024] The server control unit 13 may include a dedicated microprocessor or a CPU (Central Processing Unit) that implements specific functions by loading specific programs, a GPU (Graphics Processing Unit), etc. For example, the server control unit 13 cooperates with the terminal device 20 to execute a virtual reality application in response to a user operation on the display unit 23 of the terminal device 20. The server control unit 13 also executes various processes related to virtual reality. Specific details of the processes performed by the server control unit 13 will be described later.

[0025] (Terminal Device Configuration) The following describes the configuration of the terminal device 20. As shown in Fig. 1, the terminal device 20 includes a terminal communication unit 21, a terminal storage unit 22, a display unit 23, an input unit 24, and a terminal control unit 25.

[0026] The terminal communication unit 21 includes an interface for communicating with an external device wirelessly or via a wired connection to transmit and receive information. The terminal communication unit 21 may include a wireless communication module, a wireless LAN communication module, or a wired LAN communication module that supports mobile communication standards such as LTE (Long Term Evolution) (registered trademark), LTE-A (LTE-Advanced), a fifth-generation mobile communication system, or UMB (Ultra Mobile Broadband). The terminal communication unit 21 can transmit and receive information to and from the server device 10 via the network 3.

[0027] The terminal storage unit 22 includes, for example, a primary storage unit and a secondary storage unit. For example, the terminal storage unit 22 may include a semiconductor memory, a magnetic memory, an optical memory, or the like. The terminal storage unit 22 stores various information and programs used in processing virtual reality received from the server device 10. The information and programs used in processing virtual reality may be acquired from an external device via the terminal communication unit 21. For example, a virtual reality application program may be acquired from a predetermined application distribution server. Hereinafter, the application program may also be simply referred to as an application. Furthermore, for example, some or all of the above-mentioned information about the user and information about other users' virtual reality media may be acquired from the server device 10.

[0028] The display unit 23 includes a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 23 is capable of displaying a variety of images. The display unit 23 is configured, for example, with a touch panel, and functions as an interface that detects a variety of user operations. The display unit 23 may also be in the form of a head-mounted display.

[0029] The input unit 24 includes an input interface including, for example, a touch panel provided integrally with the display unit 23. The input unit 24 is capable of accepting user input to the terminal device 20. The input unit 24 may include physical keys or may further include any input interface including a pointing device such as a mouse. The input unit 24 may also be capable of accepting non-contact user input such as voice input or gesture input. Note that a sensor (such as an image sensor, acceleration sensor, or distance sensor) for detecting the user's body movement may be used for gesture input. In this case, the input unit 24 may be realized by an acceleration sensor, gyro sensor, or the like built into the terminal device 20.

[0030] The terminal control unit 25 includes one or more processors and controls the overall operation of the terminal device 20.

[0031] The terminal control unit 25 transmits and receives information via the terminal communication unit 21. For example, the terminal control unit 25 receives various information and programs used for various processes related to virtual reality from at least one of the server device 10 and another external server. The terminal control unit 25 stores the received information and programs in the terminal storage unit 22. For example, the terminal storage unit 22 may store a browser (Internet browser) for connecting to a web server.

[0032] The terminal control unit 25 launches a virtual reality application in response to a user operation. The terminal control unit 25 cooperates with the server device 10 to execute various processes related to virtual reality. For example, the terminal control unit 25 causes an image of a virtual space to be displayed on the display unit 23. For example, a GUI (Graphical User Interface) that detects a user operation may be displayed on the screen. The terminal control unit 25 can detect a user operation on the screen via the input unit 24. For example, the terminal control unit 25 can detect a user's tap operation, long tap operation, flick operation, swipe operation, etc. A tap operation is an operation in which a user touches the display unit 23 with a finger and then releases the finger. The terminal control unit 25 transmits operation information to the server device 10.

[0033] (Example of virtual space) The server control unit 13, in cooperation with the terminal device 20, displays an image of the virtual space on the display unit 23 and updates the image of the virtual space in accordance with the progress of the virtual reality and the user's operations. In this embodiment, the server control unit 13, in cooperation with the terminal device 20, renders objects placed in the three-dimensional virtual space as if viewed from a virtual camera placed in the virtual space.

[0034] The drawing process described below is realized by the server control unit 13, but in other embodiments, part or all of the drawing process described below may be realized by the server control unit 13. In the following description, at least part of the image of the virtual space displayed on the terminal device 20 may be a web display that is displayed on the terminal device 20 based on data generated by the server device 10, and at least part of the image may be a native display that is displayed by a native application installed on the terminal device 20.

[0035] FIG. 2 is an explanatory diagram of an example of a virtual space that can be generated by the virtual reality generation system 1. As shown in FIG.

[0036] In this embodiment, the virtual space may include multiple spatial sections. Each of the multiple spatial sections is a spatial section that the user avatar M1 can enter, and each may be capable of providing unique content. Each of the multiple spatial sections may be generated in a manner that forms a continuous space in the virtual space, similar to various spaces in reality. Alternatively, some or all of the multiple spatial sections may be discontinuous with each other. Discontinuity refers to a relationship in which the spatial sections are connected in a manner that violates the laws of physics in reality, such as a relationship between spatial sections that allows movement in a manner of teleportation such as warp.

[0037] 2, the virtual space includes a free space section 71 and a space section 70 for providing a plurality of contents. In the free space section 71, the user avatar M1 can basically move freely. Note that content (for example, various contents to be described later, such as those provided in the space section 70) may also be provided in the free space section 71 as appropriate.

[0038] The space section 70 may be a space section that is at least partially separated from the free space section 71 by a wall (an example of the second object M3) or a movement prohibition section (an example of the second object M3). For example, the space section 70 may have an entrance / exit (e.g., a second object M3 such as a hole or a door) through which the user avatar M1 can enter and exit the free space section 71. In the space section 70, content may be provided to the user avatar M1 located in the space section 70.

[0039] The types and number of contents (contents provided in virtual reality) provided in the space unit 70 are arbitrary. In the present embodiment, as an example, the contents provided in the space unit 70 include digital content such as various types of video. The video may be real-time video or non-real-time video. Furthermore, the video may be video based on real images or video based on CG (Computer Graphics). The video may be video for providing information. In this case, the video may be related to an information providing service in a specific genre (a service providing information on travel, housing, food, fashion, health, beauty, etc.), a broadcasting service by a specific user (for example, YouTube (registered trademark)), etc.

[0040] The content provided in the space section 70 may be various items (an example of the second object M3) that can be used in the virtual space, and the various items may include, for example, specific objects described below. In this case, the space section 70 that provides the various items may be in the form of a sales outlet. Alternatively, the content provided in the space section 70 may be authorization to acquire items that are available in reality, tokens, etc. Some of the multiple space sections 70 may be space sections that do not provide content.

[0041] Each of the space sections 70 may be operated by a different entity, similar to a real-world brick-and-mortar store. In this case, the operator of each space section 70 may use the corresponding space section 70 by paying a store opening fee or the like to the operator of the virtual reality generation system 1.

[0042] The virtual space may be expandable as the number of space sections 70 increases. Alternatively, multiple virtual spaces may be set for each attribute of the content provided in the space section 70. In this case, the virtual spaces may be discontinuous or continuous with each other as "space sections."

[0043] (Drawing function in virtual space) The server control unit 13 cooperates with the terminal device 20 to display a display image for the terminal (hereinafter also simply referred to as a "terminal image") on the display unit 23, and updates the terminal image. In a modified example, the terminal image may be drawn by the terminal device 20 (see FIG. 24, etc.).

[0044] FIG. 3 is an explanatory diagram of a terminal image, showing an example of the terminal image. In FIG. 3, a part of the virtual space is depicted together with a user avatar M1 (user name "User A"). The terminal image may be depicted as an image from a virtual camera 60 placed in the virtual space. In this case, a virtual camera 60 may be set for each user avatar M1. Furthermore, the virtual camera 60 may include a camera installed at a fixed point in addition to a camera for each user avatar M1.

[0045] FIG. 4 is an explanatory diagram of camera parameters of the virtual camera 60. FIG. 4 shows a field surface 40 positioned in a global coordinate system. The global coordinate system is a coordinate system fixedly associated with the virtual space. Unless otherwise specified, the field surface 40 refers to the field surface 40 in a state where a field image is projected (the field surface 40 of a field object). The field surface 40 represents a field in the virtual space. Note that for each of two or more discontinuous space portions, a field surface 40 that is discontinuous with each other in the global coordinate system may be set.

[0046] In this embodiment, the camera parameters include two position parameters (X, Y), a distance parameter A2, an orientation parameter θ, and an angle-of-attack parameter ψ. Once the values of all these parameters are determined, the virtual camera 60 can be uniquely positioned with respect to the global coordinate system. Note that when the angle-of-attack parameter ψ is approximately 90 degrees, a bird's-eye view display becomes possible.

[0047] The position parameter X is the x coordinate of the intersection of the line of sight V on the xy plane, the position parameter Y is the y coordinate of the intersection of the line of sight V on the xy plane, and the distance parameter A2 is the distance from the intersection of the line of sight V on the xy plane to the virtual camera 60 (the distance along the line of sight V). The orientation parameter θ is the angle between the x axis and the projection vector V' of the line of sight V on the xy plane. The angle of attack parameter ψ is the angle between the line of sight V and the xy plane. Note that in this embodiment, the angle of attack parameter ψ is used, but the angle of attack parameter ψ may be omitted. In other words, the angle of attack parameter ψ may have a constant value (fixed value).

[0048] Some or all of the values of these various camera parameters may be changed in conjunction with the values of parameters related to the user avatar M1 (e.g., the position or state of the user avatar M1) and / or may be changed in response to input from the user. For example, the values of two position parameters (X, Y) may correspond to the position of the user avatar M1. Note that these camera parameters are merely an example, and different parameters may be equivalently used in actual processing. For example, the camera parameters may include height relative to the xy plane and rotation parameters around three orthogonal axes (i.e., yaw, roll, and pitch). The camera parameters may also include other parameters such as focal length.

[0049] (Details of the interaction promotion function) In this embodiment, the virtual space can also function as a place for users to interact with each other via the user avatar M1. In this case, for example, multiple users can make an appointment in advance and receive content in a specific space section 70 at a predetermined time. In this case, multiple users can interact with each other through the provision of content. Alternatively, multiple users can make an appointment in advance and gather in a specific space section 70 at a predetermined time to enjoy dialogue or conversation (hereinafter referred to as "dialogue").

[0050] In order to efficiently support interactions between users via their respective user avatars M1 in a virtual space, it is useful to effectively support input of avatar movement operations while also effectively supporting conversations between users via their avatars in the virtual space.

[0051] Therefore, in this embodiment, the virtual reality generation system 1 has an interaction promotion function that efficiently supports interaction between users via each user avatar M1 by appropriately switching the viewpoint of the virtual camera 60 associated with one user avatar M1, as will be described in detail below. Note that, hereinafter, the viewpoint of the virtual camera 60 associated with one user avatar M1 means a relative viewpoint based on the one user avatar M1, and the relative viewpoint is uniquely determined based on the position / orientation information of the one user avatar M1.

[0052] In the following, the server device 10 associated with the interaction promotion function realizes an example of an information processing system, but as will be described later, each element of a specific terminal device 20 (see the terminal communication unit 21 to the terminal control unit 25 in FIG. 1) may realize an example of an information processing system, or multiple terminal devices 20 may cooperate to realize an example of an information processing system. Also, the server device 10 and one or more terminal devices 20 may cooperate to realize an example of an information processing system.

[0053] Fig. 5 is an example of a functional block diagram of the server device 10 related to the interaction promotion function. Fig. 6 is an explanatory diagram of data in the user database 140. Fig. 7 is an explanatory diagram of data in the avatar database 142. Fig. 8 is an explanatory diagram of data in the group state storage unit 146. In Figs. 6 to 8, "***" indicates that some information is stored, "-" indicates that no information is stored, and "..." indicates a similar repetition.

[0054] As shown in FIG. 5 , the server device 10 includes a user database 140, an avatar database 142, a group state storage unit 146, a group setting unit 150, a user avatar processing unit 152, a terminal image generation unit 158, a dialogue processing unit 160 (an example of an output unit), an invalidation processing unit 162, and a parameter update unit 170. Note that some or all of the functions of the server device 10 described below may be implemented by the terminal device 20 as appropriate (see FIG. 20 ). The division between the user database 140 and the group state storage unit 146 and the division between the group setting unit 150 and the parameter update unit 170 are for convenience of explanation, and some functional units may implement the functions of other functional units. For example, the functions of the group setting unit 150, the user avatar processing unit 152, the terminal image generation unit 158, and the dialogue processing unit 160 may be implemented by the terminal device 20. Also, for example, some or all of the data in user database 140 may be integrated with the data in avatar database 142, or may be stored in a separate database.

[0055] Note that the user database 140 to the group state storage unit 146 can be realized by the server storage unit 12 shown in Fig. 1, and the group setting unit 150 to the parameter update unit 170 can be realized by the server control unit 13 shown in Fig. 1. Also, part of the group setting unit 150 to the parameter update unit 170 (functional units that communicate with the terminal device 20) can be realized by the server communication unit 11 together with the server control unit 13 shown in Fig. 1.

[0056] User information is stored in the user database 140. In the example shown in Fig. 6, the user information includes user information 600 relating to a user.

[0057] In the user information 600, each user ID is associated with a user name, user authentication information, user avatar ID, position / orientation information, friend information, specific object ID, etc. The user name is a name registered by the user himself / herself and is arbitrary. The user authentication information is information for identifying the user as a legitimate user and may include, for example, a password, an email address, a date of birth, a password, biometric information, etc.

[0058] A user avatar ID is an ID for identifying a user avatar. In this embodiment, one user avatar ID is associated with each user ID. Therefore, in the following description, "associated with a user (or user ID)" or similar expressions are synonymous with "associated with a user avatar ID" or similar expressions. However, in other embodiments, multiple user avatar IDs may be associated with one user ID.

[0059] The position / orientation information includes position information and orientation information of the user avatar M1. The position / orientation information will be described in detail later. The friend information may include information (e.g., a user ID) that identifies a user who is in a friend relationship. Hereinafter, each user avatar M1 associated with each user who is in a friend relationship will also be referred to as a "friend avatar." The specific object ID is information that identifies a specific object, which will be described later. Multiple specific object IDs may be associated with one user avatar ID.

[0060] The avatar database 142 stores avatar information about the user avatar M1.

[0061] In the example shown in FIG. 7 , the avatar information 700 associates each user avatar ID with a facial part ID, a hairstyle part ID, a clothing part ID, and the like. Appearance-related part information, such as the facial part ID, hairstyle part ID, and clothing part ID, is a parameter that characterizes the user avatar M1 and may be selected by each corresponding user. For example, multiple types of appearance-related information, such as the facial part ID, hairstyle part ID, and clothing part ID, associated with the user avatar M1 may be prepared. Furthermore, with regard to the facial part ID, a part ID may be prepared for each type, such as face shape, eyes, mouth, and nose, and the information related to the facial part ID may be managed as a combination of IDs of each part constituting the face. In this case, each user avatar M1 can be drawn not only on the server device 10 but also on the terminal device 20 side based on the appearance-related IDs associated with each avatar ID.

[0062] 7, the avatar information 700 associates state information indicating whether the user avatar M1 is in a seated state and a value of a seated parameter with each user avatar ID. In this embodiment, the state of the user avatar M1 is either a seated state (an example of a second state) or a non-seated state, but the non-seated state may be further subdivided.

[0063] In this embodiment, the non-seated state includes a movable state (an example of a first state) and a transition state, and the transition state includes a transition state to sitting and a transition state to standing, as described below. The movable state may also be further subdivided. For example, the movable state may include a moving state, a stationary state, etc. The seating parameter is any parameter related to sitting and may include multiple parameters. The seating parameter may include, for example, a parameter representing the sitting time or a parameter representing the sitting frequency.

[0064] The group state storage unit 146 stores group state information relating to the state of a group active in the virtual space. Groups are set by the group setting unit 150, which will be described later. In the example shown in FIG. 8, the group state information 800 associates, for each group ID, a corresponding group name and a user ID (user ID relating to a user belonging to the corresponding group). Note that one group ID can be associated with multiple user IDs. Note that a group may also be called a party.

[0065] The group setting unit 150 sets a group consisting of one or more users who will interact in the virtual space. For example, when each user enters the virtual space via the user avatar M1, the user inputs a group name. In this case, the group setting unit 150 may set a group ID for each group name and set users who input the same virtual space name as the same group. In this case, a virtual space may be generated for each group so that each user in the group can share the same virtual space. As a result, for example, multiple users who wish to interact in the virtual space can interact in the same virtual space without interacting with other users (users in different groups) by inputting a common virtual space name that they have notified each other of in advance. Furthermore, since virtual spaces can be managed for each group, the amount of information about other users transmitted to the terminal device 20 associated with one user can be reduced compared to when multiple users share one virtual space, thereby reducing the communication load on the virtual reality generation system 1 as a whole. In a modified example, one virtual space may be simultaneously available to users belonging to multiple groups. Hereinafter, unless otherwise specified, each user avatar M1 is considered to belong to the same group.

[0066] Furthermore, when a user enters the virtual space via the user avatar M1, the group setting unit 150 may cooperate with the terminal image generation unit 158 to display group information currently being set on the terminal device 20 associated with the user. In this case, the group information may include information representing the group name and its members (user names, etc.). The display of the group name may also function as a selection button. In this case, a user who finds a display of a desired group name can easily join the desired group by operating the corresponding selection button. Note that permission from users who are members of the group may be required to join the group.

[0067] In another embodiment, the group setting unit 150 may assign each user to one of multiple groups without relying on input from the user. In this case, the assignment may be realized so that the number of users belonging to each group is equalized. This makes it possible to equalize the balance of the processing load for each group. In this case, the group setting unit 150 may assign users who enter the room at similar times to the same group, or may realize assignment based on user attribute information (age, gender, preferences, etc.).

[0068] The user avatar processing unit 152 executes various processes related to each user avatar M1. The user avatar processing unit 152 includes an operation input acquisition unit 1521 (an example of an acquisition unit) and a user action processing unit 1522 for each user avatar M1.

[0069] The operation input acquisition unit 1521 acquires operation input information generated in response to various operations by the user. Note that the operation input information by the user is generated via the input unit 24 of the terminal device 20 described above.

[0070] In this embodiment, the operation input information may include an operation input (an example of a first input) that changes the position of the user avatar M1 in the virtual space, an operation input that changes the value of other parameters (parameters for avatar actions related to things other than movement) such as the orientation of the user avatar M1, an operation input generated via a user interface (UI) drawn by the user interface drawing unit 1582, and an input for dialogue (an example of a third input) such as voice or text (including characters, symbols, pictures, etc.) used by the dialogue processing unit 160. The operation input generated via the user interface includes an operation input (an example of a second input) via a chair button 301, which will be described later.

[0071] Hereinafter, an operation input that changes the position of the user avatar M1 in the virtual space is an operation input that moves the user avatar M1, and is hereinafter also referred to as a “movement operation input.” Also, hereinafter, an operation input that changes the orientation of one user avatar M1 as a whole is also referred to as an “entire orientation operation input,” and an operation input (an example of a fourth input) that changes the orientation of a specific part (described later) among all parts that form one user avatar M1 is also referred to as a “part orientation operation input.”

[0072] The movement operation input, whole direction operation input, part direction operation input, etc. may be generated by operating specific keys (e.g., "WASD" keys), may be generated via a user interface including arrow buttons, etc., or may be generated by movements such as voice or gestures. In the following, as an example, it is assumed that the movement operation input, whole direction operation input, and part direction operation input are generated by operating specific keys (e.g., "WASD" keys).

[0073] In this embodiment, as an example, a movement operation input, a whole-body direction operation input, and a part-direction operation input are each treated as different inputs, but are generated via the same user interface. Specifically, the specific "WASD" keys may function as a movement operation input or a whole-body direction operation input when the user avatar M1 is in a movable state (described later), and may function as a part-direction operation input when the user avatar M1 is in a seated state (described later). This makes it possible to efficiently increase the variety of operation inputs while reducing the complexity of the user interface.

[0074] In this embodiment, the user action processing unit 1522 includes a state switching unit 15220 and a basic action processing unit 15221 (an example of a position changing unit).

[0075] The state switching unit 15220 switches the state of each user avatar M1 between a seated state and a movable state. In a modified example, the movable state may be further subdivided as described above, in which case the state switching unit 15220 may switch the state of each user avatar M1 between three or more states. In this embodiment, when switching the state of each user avatar M1 between the seated state and the movable state, the state switching unit 15220 switches via a transition state. The transition state is a state related to the transition between the seated state and the movable state, and will be described in detail later.

[0076] The state switching unit 15220 switches the state of the user avatar M1, which is in a movable state, to a seated state when a predetermined seating condition (an example of a predetermined first switching condition) is met. The predetermined seating condition may be determined for each user avatar M1, or may be determined for each of a plurality of user avatars M1. In the present embodiment, as an example, the predetermined seating condition is determined for each user avatar M1.

[0077] The predetermined seating condition is arbitrary. In the present embodiment, as an example, the predetermined seating condition for one user avatar M1 is satisfied when a seating instruction for a user associated with the user avatar M1 is generated. The seating instruction for the user may be generated via a user interface (chair button 301) generated by the user interface rendering unit 1582 (described later). In this case, the predetermined seating condition for one user avatar M1 may be satisfied regardless of the position of the user avatar M1. In this case, the user can switch the user avatar M1 to a seated state at any location. This allows the user to seat the user avatar M1 and engage in a conversation with a user associated with another user avatar M1 on the spot, without having to move the user avatar M1 to a dedicated conversation space that may be set in the virtual space, for example, thereby improving convenience.

[0078] In another embodiment, when a user avatar M1 is located within a specific area in the virtual space, the state switching unit 15220 may switch the state of the user avatar M1, which is in a movable state, to a seated state in response to a sitting instruction for a user associated with the user avatar M1. In this case, the specific area is a portion of the virtual space and may be a fixed area or a variable area. For example, the specific area may include an area of the entire virtual space other than the no-sitting area. In this case, the no-sitting area may include an area that obstructs the field of view (the area displayed in the terminal image) of each user avatar M1. Note that in this case, the no-sitting area may change in response to a change in the field of view of each user avatar M1. Alternatively, the specific area may include an area of the entire virtual space in which the user avatar M1 can be located. In this case, the condition regarding the position of the user avatar M1 is essentially satisfied unconditionally.

[0079] The state switching unit 15220 switches the state of the user avatar M1, which is in a seated state, to a movable state when a predetermined seating release condition (an example of a predetermined second switching condition) is met. The predetermined seating release condition may be determined for each user avatar M1, or may be determined for each of a plurality of user avatars M1. In the present embodiment, as an example, the predetermined seating release condition is determined for each user avatar M1.

[0080] The predetermined seating release condition is arbitrary and may be determined based on, for example, seating parameters of the user avatar M1 in a seated state. The predetermined seating release condition may also be determined based on a dialogue status of the user avatar M1 in a seated state. For example, the predetermined seating release condition may be satisfied when no dialogue is performed by the user avatar M1 in a seated state for a certain period of time or more. In this case, the presence or absence of dialogue may be determined based on the presence or absence of an utterance, or the presence or absence of a user avatar M1 as a dialogue partner. In the present embodiment, as an example, the predetermined seating release condition for one user avatar M1 is satisfied when a release instruction for a user associated with the one user avatar M1 is generated. The release instruction for the user may be generated via a user interface (chair button 301) generated by the user interface rendering unit 1582 (described later).

[0081] The basic action processing unit 15221 determines the position and orientation of each user avatar M1 in the virtual space based on the operation input information (movement operation input, whole orientation operation input, part orientation operation input) acquired by the operation input acquisition unit 1521. The position / orientation information of each user avatar M1 may be information based on a global coordinate system (see FIG. 4). The position / orientation information of each user avatar M1 determined by the basic action processing unit 15221 may be stored (updated) in association with the corresponding user ID (see FIG. 6). Furthermore, the basic action processing unit 15221 may determine various movements of the hands, face, etc. of the user avatar M1 based on the operation input information.

[0082] In this embodiment, the position / orientation information of one user avatar M1 may include only information representing the position and orientation of the user avatar M1 as a whole (hereinafter, for clarity, this information may also be referred to as "position / orientation information related to the representative point of the user avatar M1"). Alternatively, the position / orientation information of one user avatar M1 may include information representing the positions and orientations of each of a plurality of features of the user avatar M1 (hereinafter, for clarity, this information may also be referred to as "position / orientation information related to the features of the user avatar M1"). For example, if the user avatar M1 has one or more specific features whose orientations can be changed, the position / orientation information of the user avatar M1 may include information representing the orientations of the one or more specific features. In this case, the specific features may be, for example, the face, upper body, eyes, etc.

[0083] For example, FIG. 9 illustrates a local coordinate system (x1 axis, y1 axis, z1 axis) related to the orientation of facial features as an explanatory diagram of the position / orientation information of user avatar M1. The origin of the local coordinate system may be set to the center (e.g., center of gravity) of the facial features of user avatar M1. In this case, the orientation of the facial features may be defined by the rotation angle around each of the x1 axis, y1 axis, and z1 axis. Note that the variable range of the rotation angle around each of the x1 axis, y1 axis, and z1 axis may be set according to the attribute of a specific feature.

[0084] From a similar perspective, the position / orientation information of one user avatar M1 may include position information of various joints of the one user avatar M1. The various joints of the user avatar M1 may include joints corresponding to some or all of the joints of a human (joints related to the limbs, neck, fingers, etc.). Note that the position information of the various joints of the user avatar M1 may be generated based on position information of various joints of the corresponding user. In this case, the position information of the various joints of the user may be generated as operation input information in the terminal device 20 using motion capture technology.

[0085] The terminal image generation unit 158 renders each virtual reality medium (e.g., user avatar M1) that can move within the virtual space. Specifically, the terminal image generation unit 158 generates a terminal image to be displayed on the terminal device 20 of each user based on the avatar information 700 (see FIG. 7) and the position / orientation information of each user avatar M1.

[0086] For example, for each user avatar M1, the terminal image generation unit 158 generates, based on the position / orientation information of the user avatar M1, an image (terminal image) to be displayed on the terminal device 20 of the user associated with the user avatar M1. Specifically, based on the position / orientation information of the user avatar M1, the terminal image generation unit 158 generates, as the terminal image, an image of the virtual space viewed from the virtual camera 60 at a position and orientation corresponding to the position / orientation information. In this case, since the position / orientation information of each user avatar M1 is different from one another, the terminal image is different for each user associated with each user avatar M1. In the following, taking this into consideration, a terminal image generated based on the position / orientation information of the user avatar M1 of a certain user may be referred to as a terminal image for the certain user. Unless otherwise specified, the following describes the function of the terminal image generation unit 158 when generating a terminal image for one user (and the user avatar M1 associated with the user avatar M1). However, the same applies to the case where terminal images for other users are generated.

[0087] In this embodiment, the terminal image generation unit 158 has multiple modes, such as a first-person perspective mode and a third-person perspective mode. For example, in the first-person perspective mode, the terminal image generation unit 158 adjusts the position and orientation (each value of the camera parameters) of the virtual camera 60 to a position and orientation corresponding to the position / orientation information of a single user avatar M1. In this case, the field of view of the virtual camera 60 substantially matches the field of view of the single user avatar M1. Note that in this case, the user avatar M1 is not visible in the field of view of the virtual camera 60. This allows the user associated with the single user avatar M1 to understand the state of their surroundings (e.g., a friend avatar) from the perspective of the single user avatar M1. As a result, it becomes easier to concentrate on a specific activity, such as concentrating on a conversation or watching content.

[0088] On the other hand, in the third-person perspective mode, the terminal image generation unit 158 adjusts the position of the virtual camera 60 to a position slightly away from the position of the one user avatar M1. At this time, the terminal image generation unit 158 may determine the values of other camera parameters of the virtual camera 60 according to the position / orientation information of the one user avatar M1. At this time, the viewpoint of the virtual camera 60 may be set to a position slightly away behind or to the side of the user avatar M1 so as to generate a terminal image in which the user avatar M1 is displayed. This allows the user associated with the one user avatar M1 to grasp the state of the one user avatar M1 in the virtual space from a third-person perspective. As a result, the state of the one user avatar M1 can be confirmed from a relatively wide field of view, making it easier to operate the one user avatar M1 (for example, to move in a desired direction).

[0089] The first-person viewpoint mode and the third-person viewpoint mode will be described in detail later in relation to the viewpoint switching unit 1702.

[0090] It should be noted that modes other than the first-person perspective mode and the third-person perspective mode may be set, and for example, in other modes, the values of various camera parameters of the virtual camera 60 may be arbitrarily adjustable by the corresponding user.

[0091] When another user avatar M1 is located within the field of view of the virtual camera 60, the terminal image generation unit 158 generates a terminal image including the other user avatar M1. In this case, however, in order to reduce the load of the drawing process, the other user avatar M1 may be drawn in a relatively simple manner (for example, in the form of a two-dimensional sprite).

[0092] The terminal image generation unit 158 may draw each user avatar M1 in association with username information 3002 indicating the username (for example, "User A" in FIG. 3). The username information 3002 may optionally include the text of the username. This allows each user to identify the user avatar M1 of a desired user based on the username. Note that the username may be hidden based on a setting made by the user.

[0093] In this embodiment, terminal image generation unit 158 includes an avatar image drawing unit 1580 , a base image drawing unit 1581 , a user interface drawing unit 1582 , and a speaker information drawing unit 1583 .

[0094] Avatar image drawing unit 1580 draws a portion of the terminal image described above that relates to user avatar M1. Specifically, avatar image drawing unit 1580 draws one or more user avatars M1 that may be located within the field of view of virtual camera 60, based on the values of each camera parameter of virtual camera 60, position / orientation information of each user avatar M1, avatar information 700 (see FIG. 7), etc. Note that the user avatar M1 drawn by avatar image drawing unit 1580 may be superimposed on a base image, which will be described later.

[0095] As described above, if the position / orientation information of one user avatar M1 includes information representing the positions and orientations of multiple features of the user avatar M1, the avatar image drawing unit 1580 may represent the positions and orientations of each of the multiple features of the user avatar M1 based on such information. This allows the movement of the user avatar M1 to be expressed more naturally. For example, if the specific feature is the upper body, it may be possible to represent a movement in which the upper body twists relative to the lower body.

[0096] Furthermore, as described above, if the position / orientation information of a user avatar M1 includes position information of various joints of the user avatar M1, the avatar image drawing unit 1580 may draw the user avatar M1 so that the positions of the various joints of the user avatar M1 match such information.

[0097] Avatar image rendering unit 1580 renders user avatar M1 located within the field of view of virtual camera 60 in different ways depending on the state of user avatar M1 (seated state or non-seated state).

[0098] Specifically, when the user avatar M1 is in a seated state, the avatar image drawing unit 1580 draws the user avatar M1 in a manner that represents the seated state of the user avatar M1. More specifically, as shown in FIG. 10 , the avatar image drawing unit 1580 draws the user avatar M1 sitting on a specific object M4, which is a second object M3. In this case, the specific object M4 may be any second object M3 corresponding to an object on which a real person can sit, such as a chair or sofa. That is, the specific object M4 has the form of a chair or similar. However, the specific object M4 may also be a second object M3 corresponding to an animal, such as a horse or a camel, or a second object M3 corresponding to the ground (field object) or a rug on it.

[0099] When the user avatar M1 is in a non-seated state (e.g., a movable state or a transition state), the avatar image drawing unit 1580 draws the user avatar M1 in a manner that represents the non-seated state of the user avatar M1. For example, the avatar image drawing unit 1580 may draw the user avatar M1 in a manner that represents a moving state in which the user avatar M1 is moving, based on operation input information including a movement operation input. In this case, the avatar image drawing unit 1580 may represent the movement of the user avatar M1's limbs. Furthermore, when no operation input is acquired, the avatar image drawing unit 1580 may draw the user avatar M1 in a manner that represents a stationary state in which the user avatar M1 is not moving. However, the avatar image drawing unit 1580 may impart slight movement to the user avatar M1 so that a stationary state for a certain period of time or longer does not appear unnatural.

[0100] Note that, when user avatar M1 is not seated, avatar image rendering unit 1580 may render one user avatar M1 without considering position / orientation information related to the features of that user avatar M1. This is because position / orientation information related to the features of user avatar M1 is less important when user avatar M1 is not seated than when user avatar M1 is seated. In particular, when a movement operation input related to user avatar M1 has been acquired, the moving state of user avatar M1 may be rendered based only on position / orientation information related to the representative point of user avatar M1. This allows for efficient reduction in the load of the rendering process without significantly reducing the amount of information in the device image.

[0101] Furthermore, when one user avatar M1 is in a seated state, the avatar image drawing unit 1580 may change the orientation of a specific feature of the one user avatar M1 based on the above-described feature orientation operation input for the one user avatar M1 (i.e., position / orientation information related to the specific feature of the one user avatar M1). This allows the user to communicate with other user avatars M1 (friend avatars, etc.) not only with words but also with the movements of the user avatar M1.

[0102] Furthermore, instead of or in addition to using part orientation operation input, when one user avatar M1 is in a speaking state, the avatar image drawing unit 1580 may identify a conversation partner based on an analysis of the content of the conversation up to that point, and change the orientation of a specific part of the one user avatar M1 so that the specific part faces the user avatar M1 associated with the identified conversation partner.

[0103] For example, in the following order: 1. User avatar M1 associated with username "User A" sends "Hello" 2. User avatar M1 associated with the username "User B" sends a message saying, "Hello, Mr. A." 3. User avatar M1, who is associated with the username "User C," says, "Mr. B, it's been a while." 4. User avatar M1, who has the username "User A," asks, "What should I do today?" In the above conversation, when the user avatar M1 associated with the username "User B" speaks in 2., since this is after the user avatar M1 associated with the username "User A", the entire user avatar M1 associated with the username "User B" or specific parts of the user avatar M1 may be directed toward the user avatar M1 associated with the username "User A" and / or may be moved in a specific motion. When moving in a specific motion, for example, the hand parts of the user avatar M1 associated with the username "User B" may be moved in a manner similar to a person waving their hands, thereby showing a reaction to the user avatar M1 associated with the speaker. The same applies when the user avatar M1 associated with the username "User C" speaks in 3. Furthermore, the recognition results of keywords (e.g., usernames, etc.) included in the content of the utterance may be used. For example, when user avatar M1 associated with the username "User B" speaks in 2., since the speech includes "Mr. A," the entire user avatar M1 associated with the username "User B" or specific parts of the user avatar M1 associated with the username "User A" may be directed toward the user avatar M1 associated with the username "User A."

[0104] In the present embodiment, avatar image drawing unit 1580 includes a sitting action drawing unit 15801 and a standing action drawing unit 15802 .

[0105] When the state switching unit 15220 of the user action processing unit 1522 switches the state of one user avatar M1 from a movable state to a seated state, the sitting action drawing unit 15801 draws the user avatar M1 in a transition state relating to the transition from the movable state to the seated state. Specifically, the sitting action drawing unit 15801 draws an animation of the user avatar M1 sitting down to a seated posture. In this embodiment, the sitting action drawing unit 15801 draws an animation of the sitting action while transitioning from the third-person perspective mode to the first-person perspective mode. At this time, the sitting action drawing unit 15801 may draw an animation of the sitting action while causing a specific object M4 to appear as a seated object. Examples of animations of the sitting action will be described later in relation to the viewpoint switching unit 1702 with reference to FIGS. 15 to 17 .

[0106] The sitting action drawing unit 15801 may determine the sitting position and sitting posture (orientation) based on the position / orientation information of the representative point of the user avatar M1 when the sitting instruction is generated. Therefore, the user can achieve a seated state in a desired sitting position and sitting posture (orientation) by adjusting the overall position and orientation of the user avatar M1. This makes it easier for the individuality of the user avatar M1 to be expressed, and promotes interaction between users, compared to when the sitting posture (orientation) is automatically determined according to the sitting position, for example.

[0107] When the state switching unit 15220 of the user movement processing unit 1522 switches the state of one user avatar M1 from a seated state to a movable state, the standing-up movement drawing unit 15802 draws the user avatar M1 in a transition state relating to the transition from the seated state to the movable state. Specifically, the standing-up movement drawing unit 15802 draws an animation of the user avatar M1 standing up from a seated posture. In this embodiment, the standing-up movement drawing unit 15802 draws an animation of the standing-up movement while transitioning from the first-person perspective mode to the third-person perspective mode. Note that the standing-up movement may be drawn in a manner that is substantially the reverse of the above-described sitting movement.

[0108] The base image drawing unit 1581 draws the basic part of the terminal image as described above. That is, the base image drawing unit 1581 draws the basic part before the drawings by the avatar image drawing unit 1580 and the user interface drawing unit 1582 are superimposed. For example, the base image drawing unit 1581 draws the virtual space itself (a portion excluding the second object M3, etc.) and the second object M3 in the virtual space within the field of view of the virtual camera 60, based on the drawing information of the virtual space, the values of each camera parameter of the virtual camera 60, various information related to the second object M3, etc. Note that the drawing information of the virtual space may be prepared in advance or may be updated later or dynamically. Each position in the virtual space may be defined in a global coordinate system (see FIG. 4). Note that the virtual space may be drawn by any method, and may be realized, for example, by mapping a field object or a background object onto an appropriate plane or curved surface.

[0109] The user interface drawing unit 1582 draws a user interface that allows the user to perform various operations. The items that can be operated via the user interface are arbitrary. For example, in the example shown in FIG. 3, the user interface includes a main interface 300, which includes a chair button 301, a like button 302, a ticket management button 303, a friend management button 304, and an exit button 305. Also, in the example shown in FIG. 3, the terminal image includes an interaction interface 309, which is another user interface.

[0110] The chair button 301 is operated when switching the state of the user avatar M1 between the seated state and the non-seated state described above. For example, when each user wants to have a long conversation via the user avatar M1, the user can generate the above-mentioned seated instruction by operating the chair button 301.

[0111] In this embodiment, when the chair button 301 is operated while the user avatar M1 is in a seated state, the above-described release instruction is generated. In this case, the chair button 301 generates different instructions (a seated instruction or a release instruction) depending on whether the user avatar M1 is in a seated state or a movable state.

[0112] The chair button 301 may take any shape, but in the example shown in Fig. 3 it takes the shape of a chair, which makes it possible to realize an intuitively easy-to-understand user interface.

[0113] The user interface rendering unit 1582 may render the chair button 301 for one user avatar M1 in a different manner when the user avatar M1 is in a seated state and when the user avatar M1 is in a movable state. For example, the user interface rendering unit 1582 may render the chair button 301 in a different color or shape when the user avatar M1 is in a seated state and when the user avatar M1 is in a movable state. Alternatively, in a modified example, a button for instructing the user to sit and a button for instructing the user to release may be rendered separately. In this case, the button for instructing the user to sit may be rendered operable when the user avatar M1 is in a movable state and inoperable when the user avatar M1 is in a seated state. Furthermore, the button for instructing the user to release may be rendered inoperable when the user avatar M1 is in a movable state and inoperable when the user avatar M1 is in a seated state.

[0114] The like button 302 is operated when giving a good rating, a gift, or the like to another user avatar M1 via the user avatar M1.

[0115] A ticket management button 303 is operated to output a ticket management screen (not shown) on which various ticket statuses can be viewed.

[0116] The friend management button 304 is operated when outputting a friend management screen (not shown) related to other user avatars M1 who are friends with the user.

[0117] The exit button 305 is operated when the user avatar M1 is to exit the virtual space.

[0118] The speaker information drawing unit 1583, in cooperation with the dialogue processing unit 160 (to be described later), draws information suggesting or indicating which user avatar M1 is speaking. That is, the speaker information drawing unit 1583, in cooperation with the dialogue processing unit 160 (to be described later), expresses the speaking state of the user avatar M1.

[0119] Specifically, the speaker information drawing unit 1583 may use a predetermined image associated with each user avatar M1 to emphasize the predetermined image associated with one user avatar M1 in a speaking state compared to the predetermined images associated with other user avatars M1 (for example, user avatars M1 located in the vicinity of the one user avatar M1).

[0120] The predetermined image associated with each user avatar M1 is an image drawn at a position away from the corresponding user avatar M1, and may be in the form of an avatar icon representing each user avatar M1, for example. FIG. 11 shows a group information display area 3000, which is a part (part Q1) of the terminal image shown in FIG. 3. The groups displayed in the group information display area 3000 correspond to the groups set by the group setting unit 150. The group information display area 3000 includes avatar icons 350, 351, and 352 of each user avatar M1 in the same group. Each of the avatar icons 350, 351, and 352 is associated with a corresponding user name (e.g., "User A," "User B," etc.). In this case, when the user avatar M1 associated with the user name "User A" is in a speaking state, the avatar icon 350 may be highlighted. The highlighting method is arbitrary, but may be realized, for example, by changing the brightness or color of the outline or outside of the avatar icon related to user avatar M1 in a speaking state, by flashing the outline of the avatar icon related to user avatar M1 in a speaking state, by increasing the thickness of the line of the outline of the avatar icon related to user avatar M1 in a speaking state, or by any combination of these.

[0121] Alternatively, the predetermined image may be another icon that may be drawn in association with the avatar icon. In this case, the other icon may be microphone icons 360, 361, and 362 as shown in FIG. 11. In this case, among the microphone icons 360, 361, and 362, the microphone icon corresponding to the user avatar M1 in the speaking state may be emphasized in a similar manner (for example, by enlarging the size, blinking, coloring, etc.). In this case, the size of the microphone icon may be changed depending on the volume of the voice.

[0122] Furthermore, the predetermined image may be drawn in association with the user avatar M1. For example, when one user avatar M1 transitions to a speaking state, a predetermined image may be drawn in association with the one user avatar M1. For example, a predetermined image in the form of a megaphone may be drawn above the head of the one user avatar M1. In this case, the predetermined image may be hidden when the one user avatar M1 finishes speaking.

[0123] The predetermined image may be username information 3002 (see FIG. 3) representing the username. In this case, any method of highlighting may be used, but may be realized, for example, by changing the brightness or color of the outline or the outside of the outline of the username information 3002 related to the user avatar M1 in the speaking state, by blinking the outline of the username information 3002 related to the user avatar M1 in the speaking state, by increasing the thickness of the line of the outline of the username information 3002 related to the user avatar M1 in the speaking state, or by any combination thereof.

[0124] In another embodiment, when one user avatar M1 is in a speaking state, a predetermined image in the form of a line or an arrow may be drawn toward the user avatar M1 of the conversation partner. In this case, the predetermined image in the form of a line or an arrow may be drawn such that the starting point side corresponds to the one user avatar M1 and the ending point side corresponds to the user avatar M1 of the conversation partner. Note that the conversation partner user avatar M1 may be estimated based on an analysis of the conversation content, as described above. Alternatively, a collider (collision determination) may be adjusted based on the volume of the voice, and the user avatar M1 determined to be in contact may be identified as the conversation partner.

[0125] Alternatively, the speaker information drawing unit 1583 may be realized by the avatar image drawing unit 1580. In this case, the avatar image drawing unit 1580 may express the speaking state of the user avatar M1 by moving the face (e.g., mouth) or limbs of the user avatar M1. In this case, the movements of the face (e.g., mouth) or limbs of the user avatar M1 may be fixed by animation or may be realized in a manner based on a gesture input by the user. In the case of gesture input, various facial movements of the user may be recognized based on a facial image of the user using a camera (an example of the input unit 24) of the terminal device 20. In this case, the recognition of various facial movements of the user may be realized by a face tracking function that may be built into the terminal device 20. The movements of the face (e.g., mouth) or limbs of the user avatar M1 to express such a speaking state may be drawn only when the user avatar M1 is in a seated state. In this case, the processing load on terminal device 20 and the processing load of the drawing process on avatar image drawing unit 1580 can be reduced compared to, for example, when the face tracking function is always operating. Furthermore, when one user avatar M1 is in a speaking state, avatar image drawing unit 1580 may emphasize the one user avatar M1 compared to other user avatars M1 (e.g., user avatars M1 located around the one user avatar M1). The emphasis method is arbitrary, and may be realized, for example, by changing the brightness or color of the outline or the outside of the outline of the user avatar M1 in the speaking state, blinking the outline of the user avatar M1 in the speaking state, increasing the thickness of the line of the outline of the user avatar M1 in the speaking state, or any combination thereof.

[0126] The dialogue processor 160 executes dialogue processing related to dialogue between users via the network 3 based on input from multiple users. Dialogue between users may be realized in a text and / or voice chat format via their respective user avatars M1. For example, the dialogue interface 309 of the terminal image shown in FIG. 3 may be used for dialogue input. In this case, a user can input voice by operating a microphone icon 3091 and speaking, or can input text by entering text in a text input area 3092. This enables dialogue between users. Note that text may be drawn on each terminal image (each terminal image associated with each user engaged in dialogue) in a dialogue format that leaves a certain number of histories. In this case, for example, the text may be output separately from the image related to the virtual space, or may be output superimposed on the image related to the virtual space.

[0127] The dialogue processor 160 may execute dialogue processing for each group so that dialogue is realized only within the same group. In this case, each user can enjoy dialogue with peace of mind because the content of their utterances will not be known to users outside the group. Furthermore, when the user avatar M1 takes a seat, the dialogue processor 160 may mute or reduce the volume of sounds in the virtual space (for example, predetermined music that is constantly being played) to make voice chat (audio dialogue) easier to hear.

[0128] In the present embodiment, the dialogue processing unit 160 may implement dialogue processing between user avatars M1 in a seated state and dialogue processing between other user avatars M1 (for example, dialogue processing between a user avatar M1 in a seated state and a user avatar M1 in a movable state, or dialogue processing between user avatars M1 in a movable state) in different modes. For example, in the case of dialogue processing between user avatars M1 in a seated state, the dialogue processing unit 160 may increase the output level of the voice of the utterance and / or decrease the output level of the ambient sound (external volume) compared to dialogue processing between other user avatars M1. This makes it easier for the user to concentrate on the dialogue when the user avatar M1 is in a seated state. Alternatively, the dialogue processing unit 160 may implement dialogue processing only between user avatars M1 in a seated state. Alternatively, the dialogue processing unit 160 may implement voice-based dialogue processing only between user avatars M1 in a seated state, and implement text-based dialogue processing for dialogue processing between other user avatars M1.

[0129] The invalidation processing unit 162 determines whether a predetermined seating invalidation condition (one example of a predetermined invalidation condition) is met, and invalidates the chair button 301 if the predetermined seating invalidation condition is met. The predetermined seating invalidation condition is arbitrary, but may be met, for example, when the user avatar M1 is located within the above-described seating prohibition area. In this case, the predetermined seating invalidation condition may be determined for each user avatar M1. Alternatively, the predetermined seating invalidation condition may be determined based on the value of a seating parameter.

[0130] The chair button 301 may be disabled by not drawing the chair button 301 in cooperation with the user interface drawing unit 1582, or by drawing the chair button 301 in an inoperable mode. For example, the chair button 301 may be drawn with significantly lower brightness (or a different color from normal) in an inoperable display mode than in an operable display mode. In this case, the user can easily understand whether the chair button 301 is in an active state. When the user operates the chair button 301 drawn in an operable display mode, a sit instruction is normally generated. Also, when the user operates the chair button 301 drawn in an inoperable display mode, no sit instruction is generated.

[0131] In a modified example, the invalidation processing unit 162 may invalidate a sit instruction from one user via the chair button 301, instead of invalidating the chair button 301. In this case, even if a sit instruction is generated, it is invalidated. For example, the sit instruction is not generated by the operation input acquisition unit 1521, or is not normally processed by the state switching unit 15220 of the user movement processing unit 1522.

[0132] Furthermore, the disabling processing unit 162 may determine whether a predetermined standing-up disabling condition is met, and may disable the chair button 301 if the predetermined standing-up disabling condition is met. The predetermined standing-up disabling condition is arbitrary, but may be met, for example, when a content is being viewed simultaneously by multiple user avatars M1 and the user avatar M1 is seated in a position where standing up would bring the user avatar M1 into the field of view of the other user avatars M1 viewing the content. The predetermined standing-up disabling condition may also be determined based on the value of a seating parameter.

[0133] The parameter update unit 170 updates the values of the various camera parameters (see FIG. 4) of the virtual camera 60. As described above, the values of the various camera parameters of the virtual camera 60 may differ for each terminal image (each user avatar M1). Unless otherwise specified, the following description will be directed to a representative configuration for one user avatar M1 (one virtual camera 60).

[0134] In this embodiment, the parameter update unit 170 includes a normal parameter calculation unit 1700 , a sitting parameter calculation unit 1701 , a viewpoint switching unit 1702 , and a transition parameter calculation unit 1703 .

[0135] The normal parameter calculation unit 1700 functions when one user avatar M1 is in a movable state. The normal parameter calculation unit 1700 calculates the values of various camera parameters of the virtual camera 60 associated with one user avatar M1 based on the position / orientation information of the one user avatar M1. The normal parameter calculation unit 1700 calculates the values of various camera parameters of the virtual camera 60 so that the third-person perspective mode of the terminal image generation unit 158 described above is realized.

[0136] FIG. 12 is an explanatory diagram of an example of a method for calculating the values of various camera parameters of the virtual camera 60 by the normal parameter calculation unit 1700. FIG. 12 schematically illustrates a side view of one user avatar M1 (user avatar M1 associated with the user name "User A") movable in a virtual space and the virtual camera 60. An arrow R12 from the virtual camera 60 corresponds to the line of sight V illustrated in FIG. 4. When using the various camera parameters illustrated in FIG. 4, the normal parameter calculation unit 1700 may correspond the position parameter (X, Y) to the position (x, y) of the user avatar M1 based on position / orientation information associated with a representative point of the user avatar M1. Furthermore, the normal parameter calculation unit 1700 may correspond the orientation parameter θ to the orientation (front direction) of the user avatar M1 based on position / orientation information associated with a representative point of the user avatar M1. Note that the distance parameter A2 and the angle of attack parameter ψ may be fixed to values such that the entire user avatar M1 is depicted in the terminal image. Furthermore, the values of the distance parameter A2 and the angle of attack parameter ψ may be adjustable by the user.

[0137] The normal parameter calculation unit 1700 may calculate the values of various camera parameters of the virtual camera 60 without considering position / orientation information related to the features of one user avatar M1. This is because when the user avatar M1 is in a movable state, the position / orientation information related to the features of the user avatar M1 is less important than when the user avatar M1 is in a seated state. In particular, when a user associated with one user avatar M1 is performing a movement operation input, the moving state of the one user avatar M1 may be drawn based only on position / orientation information related to the representative point of the one user avatar M1. This makes it possible to efficiently reduce the load of the drawing process without significantly reducing the amount of information in the terminal image.

[0138] However, when the user associated with one user avatar M1 is not performing a movement operation input (i.e., when the user is stationary) or when the movement speed is relatively slow, the normal parameter calculation unit 1700 may calculate the values of various camera parameters of the virtual camera 60 associated with the one user avatar M1 based on the part orientation operation input (particularly, the part orientation operation input for the facial part) by the user associated with the one user avatar M1. For example, the normal parameter calculation unit 1700 may realize the viewpoint of the virtual camera 60 that matches the facial orientation of the user avatar M1 by calculating the value of the orientation parameter θ shown in FIG. 4 based on the part orientation operation input (particularly, the part orientation operation input for the facial part) associated with the one user avatar M1. This makes it easier for the user to observe the virtual space or find a desired route, for example, based on the terminal image that changes as if looking around the surroundings while the user avatar M1 is in a movable state.

[0139] The sitting parameter calculation unit 1701 functions when one user avatar M1 is in a seated state. The sitting parameter calculation unit 1701 calculates the values of various camera parameters of the virtual camera 60 so that the first-person perspective mode of the terminal image generation unit 158 described above is realized.

[0140] 13 is an explanatory diagram of an example of a method for calculating the values of various camera parameters of the virtual camera 60 by the sitting parameter calculation unit 1701. FIG. 13 schematically shows a side view of one user avatar M1 (user avatar M1 associated with the user name "User A") in a seated state in the virtual space and the virtual camera 60. An arrow R13 from the virtual camera 60 corresponds to the line of sight direction V shown in FIG. 4. As schematically shown in FIG. 13, the sitting parameter calculation unit 1701 may calculate the values of the various camera parameters of the virtual camera 60 so that the viewpoint of the virtual camera 60 corresponds to the viewpoint of one user avatar M1. 4, the sitting parameter calculation unit 1701 may calculate the position parameters (X, Y), distance parameter A2, and angle of attack parameter ψ of the virtual camera 60 based on position / orientation information related to the representative point of the user avatar M1 so that the virtual camera 60 is positioned in front of (for example, in front of) one user avatar M1. Furthermore, the sitting parameter calculation unit 1701 may calculate the value of orientation parameter θ corresponding to the orientation of one user avatar M1 based on the position / orientation information related to the representative point of the user avatar M1.

[0141] In contrast to the normal parameter calculation unit 1700, the sitting parameter calculation unit 1701 may preferably calculate the values of various camera parameters of the virtual camera 60 associated with one user avatar M1 based on position / orientation information related to the features of the user avatar M1. That is, the sitting parameter calculation unit 1701 may calculate the values of various camera parameters of the virtual camera 60 associated with one user avatar M1 based on a feature orientation operation input (particularly a feature orientation operation input for a facial feature) related to the user avatar M1. This allows, for example, a user to enjoy a conversation with multiple friend avatars based on a terminal image that can appropriately and selectively display multiple friend avatars in front of the user avatar M1 while the user avatar M1 is in a sitting state.

[0142] In a modified example, the sitting parameter calculation unit 1701 may position the virtual camera 60 in front of one user avatar M1 (e.g., in front of the user's eyes), and then use the rotation angles around the x1, y1, and z1 axes in a local coordinate system as shown in FIG. 9 as various camera parameters of the virtual camera 60. In this case, the sitting parameter calculation unit 1701 may calculate the values of the various camera parameters of the virtual camera 60 related to one user avatar M1 based on a feature orientation operation input (particularly a feature orientation operation input for a facial feature) related to the one user avatar M1. In this case, the local coordinate system may be set such that the direction indicated by the position / orientation information related to the representative point of the user avatar M1 is the front direction. Alternatively, as shown in FIG. 14, when the user avatar M1 of the conversation partner (user avatar M1 with the username "User B" in FIG. 14) is identified, the local coordinate system may be set such that the direction facing the user avatar M1 of the conversation partner is the front direction.

[0143] The viewpoint switching unit 1702 switches the viewpoint (the relative viewpoint described above) of the virtual camera 60 relating to the terminal image for the user of one user avatar M1 between a viewpoint relating to the third-person viewpoint mode (an example of the first viewpoint) and a viewpoint relating to the first-person viewpoint mode (an example of the second viewpoint).

[0144] That is, the viewpoint switching unit 1702 updates the values of the various camera parameters of the virtual camera 60 based on either the normal parameter calculation unit 1700 or the sitting parameter calculation unit 1701. Specifically, when one user avatar M1 is in a movable state, the viewpoint switching unit 1702 updates the values of the various camera parameters of the virtual camera 60 based on the normal parameter calculation unit 1700. Furthermore, when one user avatar M1 is in a seated state, the viewpoint switching unit 1702 updates the values of the various camera parameters of the virtual camera 60 based on the sitting parameter calculation unit 1701.

[0145] The transition parameter calculation unit 1703 temporarily functions in place of the viewpoint switching unit 1702 when the viewpoint switching unit 1702 switches between the viewpoint related to the third-person viewpoint mode and the viewpoint related to the first-person viewpoint mode (hereinafter also referred to as "viewpoint switching"). That is, the transition parameter calculation unit 1703 functions when the state of the user avatar M1 is in a transition state. The transition parameter calculation unit 1703 calculates and updates the values of the various camera parameters of the virtual camera 60 in the transition state described above so that a sudden change in the values of the various camera parameters of the virtual camera 60 caused by the viewpoint switching by the viewpoint switching unit 1702 does not occur.

[0146] For example, when the above-described sitting instruction is generated, the transition parameter calculation unit 1703 updates the values of the various camera parameters of the virtual camera 60 in a manner that gradually changes over a predetermined time from the values of the various camera parameters of the virtual camera 60 calculated by the normal parameter calculation unit 1700 to the values of the various camera parameters of the virtual camera 60 calculated by the sitting parameter calculation unit 1701. The predetermined time (i.e., the time related to the transition state) may be adjusted to a length that does not cause a sense of incongruity due to a sudden change. Note that the animation by the above-described sitting action drawing unit 15801 is realized based on the values of the various camera parameters of the virtual camera 60 updated by the transition parameter calculation unit 1703. In this case, when the values of the various camera parameters of the virtual camera 60 match the values of the various camera parameters of the virtual camera 60 calculated by the sitting parameter calculation unit 1701, the transition state ends and the sitting state begins. That is, the transition to a state (first-person viewpoint mode related to a sitting state) in which the values of the various camera parameters of the virtual camera 60 are updated by the viewpoint switching unit 1702 based on the calculation results of the sitting parameter calculation unit 1701 is completed.

[0147] Similarly, when the above-described release instruction is generated, the transition parameter calculation unit 1703 updates the values of the various camera parameters of the virtual camera 60 in a manner that gradually changes over a predetermined time from the values of the various camera parameters of the virtual camera 60 calculated by the sitting parameter calculation unit 1701 to the values of the various camera parameters of the virtual camera 60 calculated by the normal parameter calculation unit 1700. The animation by the above-described standing-up action drawing unit 15802 is realized based on the values of the various camera parameters of the virtual camera 60 updated by the transition parameter calculation unit 1703. In this case, when the values of the various camera parameters of the virtual camera 60 match the values of the various camera parameters of the virtual camera 60 calculated by the normal parameter calculation unit 1700, the transition state ends and a movable state begins. That is, the transition to a state (third-person perspective mode related to the movable state) in which the values of the various camera parameters of the virtual camera 60 are updated by the viewpoint switching unit 1702 based on the calculation results of the normal parameter calculation unit 1700 is completed.

[0148] Figures 15 to 17 are explanatory diagrams of the function of the transition parameter calculation unit 1703, and show examples of terminal images generated when switching between a viewpoint related to the third-person perspective mode and a viewpoint related to the first-person perspective mode.

[0149] Fig. 15 shows a terminal image G1500 based on a viewpoint related to the third-person perspective mode, Fig. 16 shows a terminal image G1600 in a transition state based on the calculation result by the transition parameter calculation unit 1703, and Fig. 17 shows a terminal image G1700 based on a viewpoint related to the first-person perspective mode. It is assumed that Figs. 15 to 17 are terminal images for a user related to user avatar M1 associated with the username "User A."

[0150] In the examples shown in FIGS. 15 to 17 , a user associated with user avatar M1 associated with the username “User A” operates chair button 301 to sit at the position of user avatar M1 shown in FIG. 15 . This activates transition parameter calculation unit 1703 and, accordingly, seated action drawing unit 15801, to generate terminal image G1700 shown in FIG. 17 via terminal image G1600 shown in FIG. 16 . Note that in this transition state, the overall position and orientation of user avatar M1 do not substantially change, but by switching to the first-person perspective mode, the user avatar M1 of the conversation partner (here, user avatar M1 with username “User B”) is depicted larger in terminal image G1700, making it easier to see the movements of user avatar M1. This makes it possible, for example, to enjoy a thorough conversation with the conversation partner, and effective interaction between users through the conversation can be expected. Furthermore, in the first-person perspective mode, the conversation partner's user avatar M1 (here, the user avatar M1 with the username "User B") is rendered large, making it easy to grasp the movements of the conversation partner's user avatar M1 even on a relatively small screen (e.g., a smartphone screen). This also contributes to effective interaction between users through dialogue. Furthermore, in the first-person perspective mode, the user's own user avatar M1 does not need to be rendered, so a reduction in the load of the rendering process can be expected. In the example shown in FIGS. 15 to 17, the specific object M4 appears at the start of the transition state (see FIG. 16), and at the end of the transition state, the specific object M4 is outside the field of view of the virtual camera 60 associated with the seated user avatar M1 (the user avatar M1 with the username "User A").

[0151] Next, an example of the operation of the virtual reality generation system 1 will be described with reference to Figures 18 and 19. In the following processing flow diagrams (flowcharts), the processing order of each step may be changed as long as the relationship between input and output of each step is not impaired.

[0152] Fig. 18 is a schematic flowchart showing an example of an operation related to the interaction promotion function. The process shown in Fig. 18 shows a process for a certain user, and may be started when a virtual reality application is started in the terminal device 20 related to the certain user, and may be repeatedly executed at predetermined intervals until the virtual reality application is turned off. Note that the process shown in Fig. 18 may be executed in parallel for each user.

[0153] In step S1800, the server device 10 acquires operation input information related to the user.

[0154] In step S1802, the server device 10 determines whether the operation input information acquired in step S1800 includes an operation input of the chair button 301. If the determination result is "YES", the process proceeds to step S1804; otherwise, the process proceeds to step S1810.

[0155] In step S1804, the server device 10 determines whether the state of the user avatar M1 is not seated. If the determination result is "YES", the process proceeds to step S1806, otherwise the process proceeds to step S1808.

[0156] In step S1806, the server device 10 generates a seat instruction based on the operation input information (operation input of the chair button 301) acquired in step S1800.

[0157] In step S1808, the server device 10 generates a release instruction based on the operation input information (operation input of the chair button 301) acquired in step S1800.

[0158] In step S1810, the server device 10 determines whether or not the operation input information acquired in step S1800 includes an operation input of a specific key (for example, the "WASD" keys). If the determination result is "YES", the process proceeds to step S1812; otherwise, the process proceeds to step S1818.

[0159] In step S1812, the server device 10 determines whether the state of the user avatar M1 is not seated. If the determination result is "YES", the process proceeds to step S1814; otherwise, the process proceeds to step S1816.

[0160] In step S1814, the server device 10 treats the operation input of a specific key included in the operation input information acquired in step S1800 as a movement operation input or a whole-orientation operation input, and updates the position / orientation information of the user avatar M1 based on the movement operation input or the whole-orientation operation input.

[0161] In step S1816, the server device 10 treats the operation input of a specific key included in the operation input information acquired in step S1800 as a part orientation operation input, and updates the position / orientation information of the user avatar M1 based on the part orientation operation input.

[0162] In step S1818, the server device 10 determines whether or not another operation input is included in the operation input information acquired in step S1800. If the determination result is "YES", the process proceeds to step S1820; otherwise, the process proceeds to step S1822.

[0163] In step S1820, the server device 10 performs processing in response to the other operation input. Note that the other operation input is optional, and the processing in response to the other operation input is also optional. For example, the other operation input may be an operation input for causing the user avatar M1 to jump. In this case, the processing in response to the other operation input may be processing for updating the position / orientation information of the user avatar M1 so that the jumping action of the user avatar M1 is reflected in the terminal image drawing processing in the next step S1822.

[0164] In step S1822, the server device 10 executes a terminal image drawing process for drawing the above-described terminal image other than the portion related to the user interface, based on the processing results of steps S1800 to S1820. An example of the terminal image drawing process will be described later with reference to FIG.

[0165] In step S1824, the server device 10 performs dialogue processing based on dialogue input that may be included in the operation input information acquired in step S1800. Note that, in a processing cycle in which dialogue input is not included in the operation input information acquired in step S1800, step S1824 may be skipped.

[0166] In step S1826, the server device 10 determines whether a predetermined seating invalidation condition or a predetermined standing-up invalidation condition is satisfied. The predetermined seating invalidation condition or the predetermined standing-up invalidation condition is as described above. If either the predetermined seating invalidation condition or the predetermined standing-up invalidation condition is satisfied, the process proceeds to step S1828. If neither the predetermined seating invalidation condition nor the predetermined standing-up invalidation condition is satisfied, the process proceeds to step S1830.

[0167] In step S1828, the server device 10 renders a user interface (see the main interface 300 and the interaction interface 309 in FIG. 3). At this time, the server device 10 renders the chair button 301 in an inoperable state.

[0168] In step S1830, the server device 10 renders a user interface (see the main interface 300 and the interaction interface 309 in FIG. 3). At this time, the server device 10 renders the chair button 301 in an operable manner.

[0169] In step S1832, the server device 10 displays the terminal image thus drawn on the terminal device 20 associated with the user.

[0170] FIG. 19 is a schematic flowchart showing an example of the terminal image drawing process (step S1822 in FIG. 18).

[0171] In step S1902, the server device 10 determines whether the user avatar M1 is in a seated state based on the avatar information 700 (see FIG. 7). If the determination result is "YES", the process proceeds to step S1904; otherwise, the process proceeds to step S1910.

[0172] In step S1904, the server device 10 updates the values of the various camera parameters based on the viewpoint in the first-person viewpoint mode (the calculation results by the above-described sitting parameter calculation unit 1701). The various camera parameters are as described above with reference to FIG.

[0173] In step S1906, the server device 10 determines whether or not a release instruction has been generated. The release instruction is generated in step S1808 of Fig. 18 described above. If the determination result is "YES", the process proceeds to step S1908; otherwise, the process proceeds to step S1932.

[0174] In step S1908, the server device 10 sets the state of the user avatar M1 to a "transition state" to standing up, and updates the avatar information 700 (see FIG. 7).

[0175] In step S1910, server device 10 determines whether or not user avatar M1 is in a "transition state" to standing up, based on avatar information 700 (see FIG. 7). If the determination result is "YES," the process proceeds to step S1912; otherwise, the process proceeds to step S1918.

[0176] In step S1912, the server device 10 updates the values of the various camera parameters based on the viewpoint in the "transition state" to startup (the calculation results by the transition parameter calculation unit 1703 described above). The various camera parameters are as described above with reference to FIG.

[0177] In step S1914, the server device 10 determines whether or not the termination condition for the "transition state" to the startup has been met. The termination condition for the "transition state" to the startup is met when the values of the various camera parameters match the viewpoint related to the third-person viewpoint mode (the calculation result by the normal parameter calculation unit 1700 described above). If the determination result is "YES", the process proceeds to step S1916; otherwise, the process proceeds to step S1932.

[0178] In step S1916, the server device 10 sets the state of the user avatar M1 to "movable state" and updates the avatar information 700 (see FIG. 7).

[0179] In step S1918, the server device 10 determines whether the state of the user avatar M1 is in a "transition state" to sitting, based on the avatar information 700 (see FIG. 7). If the determination result is "YES", the process proceeds to step S1920; otherwise (i.e., if the user avatar M1 is in a movable state), the process proceeds to step S1926.

[0180] In step S1920, the server device 10 updates the values of the various camera parameters based on the viewpoint in the "transition state" to sitting (the calculation result by the transition parameter calculation unit 1703 described above). The various camera parameters are as described above with reference to FIG.

[0181] In step S1922, the server device 10 determines whether the termination condition for the "transition state" to sitting has been met. The termination condition for the "transition state" to sitting is met when the values of the various camera parameters match the viewpoint associated with the first-person viewpoint mode. If the determination result is "YES," the process proceeds to step S1924; otherwise, the process proceeds to step S1932.

[0182] In step S1924, the server device 10 sets the state of the user avatar M1 to "seated state" and updates the avatar information 700 (see FIG. 7).

[0183] In step S1926, the server device 10 updates the values of the various camera parameters based on the viewpoint related to the third-person viewpoint mode (the calculation results by the above-described normal parameter calculation unit 1700). The various camera parameters are as described above with reference to FIG.

[0184] In step S1928, the server device 10 determines whether or not a seat instruction has been generated. The seat instruction is generated in step S1806 of Fig. 18. If the determination result is "YES", the process proceeds to step S1930; otherwise, the process proceeds to step S1932.

[0185] In step S1930, the server device 10 sets the state of the user avatar M1 to a "transition state" to sitting, and updates the avatar information 700 (see FIG. 7).

[0186] In step S1932, the server device 10 draws the above-described terminal image except for the portion related to the user interface, based on the values of the various camera parameters updated in the above-described manner.

[0187] In this way, according to the processing shown in Figures 18 and 19, by setting the viewpoint of the virtual camera 60 associated with one user avatar M1 according to various states of the user avatar M1, it is possible to generate a terminal image that effectively supports various activities of the user via the user avatar M1 in the virtual space (for example, interactions with users associated with other user avatars M1).

[0188] 5 to 19, the server device 10 comprehensively realizes various functions, but some or all of the various functions of the server device 10 described above can also be realized by the terminal device 20 instead of the server device 10. In the following, as an example, a configuration in which some of the various functions of the server device 10 are realized by the terminal device 20 will be described.

[0189] 20 is an example of a functional block diagram of a terminal device 20 related to the avatar movement guidance function. In the following, one terminal device 20 will be described as a representative, and the term "user" refers to the user associated with the terminal device 20 unless otherwise specified.

[0190] In the example shown in FIG. 20, the terminal device 20 includes a user database 240, an avatar database 242, a group state memory unit 246, an operation input generation unit 250, a server information acquisition unit 251, a user avatar processing unit 252 (an example of an acquisition unit and an example of a position change unit), a friend avatar processing unit 254, an operation input transmission unit 255, a terminal image generation unit 258, a dialogue processing unit 260 (an example of an output unit), an invalidation processing unit 262, and a parameter update unit 270.

[0191] Note that the user database 240 to the group state storage unit 246 can be realized by the terminal storage unit 22 shown in Fig. 1, and the operation input generation unit 250 to the parameter update unit 270 can be realized by the terminal control unit 25 shown in Fig. 1. Also, part of the operation input generation unit 250 to the parameter update unit 270 (functional units that communicate with the server device 10) can be realized by the terminal communication unit 21 together with the terminal control unit 25 shown in Fig. 1.

[0192] The data stored in the user database 240 and the group state storage unit 246 may be substantially the same as the data stored in the user database 140 and the group state storage unit 146 of the server device 10 described above. However, the data stored in the user database 240 may only be data related to the user and their friend users (users related to friend avatars in the same group; the same applies below) among the data stored in the user database 140 of the server device 10 described above. The same applies to the group state storage unit 246.

[0193] The operation input generating unit 250 generates the above-mentioned operation input information based on various inputs from the user (various inputs via the input unit 24). Note that the various inputs from the user are as described above, and may include movement operation input by operating a specific key or the like, whole direction operation input, part direction operation input, input for dialogue, operation input of the chair button 301, etc.

[0194] Note that the part orientation operation input may be realized by gestures, etc., as described above. For example, FIG. 21 is an explanatory diagram of part orientation operation input by gestures. FIG. 21 illustrates a state in which a user performs a part orientation operation input by changing the orientation of their face while holding the terminal device 20 in their hand. In this case, the terminal device 20 identifies the user's face based on a facial image of the user input via the terminal camera 24A (an example of the input unit 24), and generates operation input information including a part orientation operation input corresponding to the identified facial orientation. Alternatively, the user may change the orientation of the terminal device 20 while holding the terminal device 20 in their hand. In this case, the terminal device 20 may generate operation input information including a part orientation operation input corresponding to the orientation of the terminal device 20 based on an acceleration sensor 24B built into the terminal device 20.

[0195] The server information acquisition unit 251 acquires various data stored in the user database 240 and each of the group state storage units 246 from the server device 10. The timing of data acquisition by the server information acquisition unit 251 is arbitrary, and may be when the virtual reality application is updated, for example. However, the timing of acquisition (updating) of the data stored in the group state storage unit 246 may be when the users constituting the group change. In this way, the various data stored in the group state storage unit 246 from the user database 240 may be updated as appropriate based on the data acquired by the server information acquisition unit 251.

[0196] The user avatar processing unit 252 has substantially the same functions as the user avatar processing unit 152 of the server device 10. However, the user avatar M1 to be processed by the user avatar processing unit 252 may be only the user avatar M1 associated with the user of the terminal device 20.

[0197] The friend avatar processing unit 254 has substantially the same functions as the above-described user avatar processing unit 152 of the server device 10. However, the user avatar M1 to be processed by the friend avatar processing unit 254 may be only the friend avatar for the user avatar M1 associated with the user of the terminal device 20.

[0198] The user avatar processing unit 252 and the friend avatar processing unit 254, like the user avatar processing unit 152 of the server device 10 described above, each perform various processes, such as movement processing based on movement operation input, for the user avatar M1 to be processed. The user avatar processing unit 252 may perform various processes based on operation input information related to the user, and the friend avatar processing unit 254 may perform various processes based on operation input information related to the friend user. This updates the position / orientation information of each user avatar M1.

[0199] The operation input transmitting unit 255 transmits the operation input information generated by the operation input generating unit 250 to the server device 10. Note that the operation input transmitting unit 255 may transmit, instead of the operation input information, position / orientation information of the user avatar M1 updated by the user avatar processing unit 252 based on the operation input information to the server device 10. Furthermore, the operation input transmitting unit 255 may transmit the operation input information to the server device 10 only when there is another user avatar M1 (friend avatar) in the virtual space in which the user avatar M1 related to the user is active.

[0200] The terminal image generation unit 258 generates a terminal image for the terminal device 20. The terminal image may be as described above. In this case, for example, the terminal image generation unit 258 may draw an image of each friend avatar based on the position / orientation information of the friend avatar acquired or generated by the friend avatar processing unit 254, information that can identify the friend avatar to be drawn (e.g., a user avatar ID), and avatar information 700 (see FIG. 7) related to the friend avatar to be drawn. In this case, the terminal device 20 may store part information for drawing each part of the avatar in the device storage unit 22 (avatar database 242), and draw each friend avatar based on the part information and the position / orientation information, etc. of the friend avatar acquired from the server device 10.

[0201] Specifically, terminal image generation unit 258 includes avatar image drawing unit 2580, base image drawing unit 2581, and user interface drawing unit 2582. Avatar image drawing unit 2580, base image drawing unit 2581, and user interface drawing unit 2582 may be similar to avatar image drawing unit 1580, base image drawing unit 1581, and user interface drawing unit 1582 of server device 10, respectively, described above. However, the terminal image to be drawn is only the terminal image for one terminal device 20.

[0202] The dialogue processing unit 260 has substantially the same functions as the dialogue processing unit 160 of the above-described server device 10. The dialogue processing unit 260 executes dialogue processing relating to a dialogue between users in the same group, based on dialogue inputs from the user and his / her friend users.

[0203] The invalidation processing unit 262 has substantially the same function as the invalidation processing unit 162 of the above-described server device 10. However, the processing target of the invalidation processing unit 262 may be only the chair button 301 associated with the user of the terminal device 20.

[0204] The parameter update unit 270 updates the values of the various parameters (see FIG. 4) of the virtual camera 60 associated with the user avatar M1.

[0205] Fig. 22 is a schematic flowchart showing an example of operation by the terminal device 20 shown in Fig. 20, and an example of operation related to the terminal image generation unit 258. In Fig. 22, terminal device 20C represents the terminal device 20 shown in Fig. 20 associated with a user, and terminal device 20D represents the terminal device 20 shown in Fig. 20 associated with a user associated with one friend avatar in the same group. Note that here, there is one user associated with terminal device 20D, but the same applies to two or more users. That is, in this case, terminal device 20C and each of the multiple terminal devices 20D may form a pair to realize the example of operation shown in Fig. 22.

[0206] Each of the terminal device 20C and the terminal device 20D generates operation input information based on various inputs by the corresponding user (steps S2500 and S2501) and transmits the generated operation input information to the server device 10 (steps S2502 and S2508). The server device 10 transfers the operation input information received from the terminal devices 20 of each user in the same group (here, the terminal devices 20C and 20D) (steps S2504 and S2510). At this time, the server device 10 may transfer the operation input information as is, or may perform predetermined processing before transmitting. For example, the operation input information may be converted into position / orientation information of each user avatar M1 before transmitting. In this way, the operation input information (operation input information related to the friend avatar) is received by each of the terminal devices 20C and 20D (steps S2512 and S2506).

[0207] In terminal device 20C, the position / orientation information of each user avatar M1 is updated and a terminal image is drawn (step S2514) based on the operation input information generated in step S2500 and the operation input information (operation input information related to the friend avatar) received in step S2512. Similarly, in terminal device 20D, the position / orientation information of each user avatar M1 is updated and a terminal image is drawn (step S2516) based on the operation input information generated in step S2501 and the operation input information (operation input information related to the friend avatar) received in step S2506.

[0208] Such an operation is repeated in each of the terminal devices 20C and 20D until the corresponding user avatar M1 exits the virtual space ("YES" in step S2518, "YES" in step S2520).

[0209] Although not shown in FIG. 22, the above-mentioned dialogue processing (dialogue processing between user avatars M1 in the same group) may also be realized by exchanging voice inputs, etc. generated in each of terminal device 20C and terminal device 20D between terminal device 20C and terminal device 20D in a manner similar to that of the above-mentioned operation input information.

[0210] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0211] For example, in the above-described embodiment, when a predetermined forced sitting condition for a user is satisfied, the state switching unit 15220 may forcibly switch the state of the user avatar M1 for that user from a non-seated state to a seated state, regardless of whether or not a seating instruction for that user has been issued. In this case, the predetermined forced sitting condition is arbitrary, but may be satisfied when, in a specification in which a seating area (e.g., a space dedicated to conversation) is set in the virtual space, the non-seated state (e.g., a movable state) is maintained within the seating area for a predetermined period of time or more, the movable user avatar M1 remains stationary without moving for a predetermined period of time or more, or conversation in the movable state is maintained for a predetermined period of time or more. The predetermined forced sitting condition may also be determined based on the value of the above-described sitting parameter. A predetermined forced standing condition may also be set instead of or in addition to the predetermined forced sitting condition. In this case, when a predetermined forced standing-up condition for a user is met, the state switching unit 15220 may forcibly switch the state of the user avatar M1 for that user from the seated state to the non-seated state, regardless of whether a cancellation instruction for that user has been issued. In this case, the predetermined forced standing-up condition is arbitrary, but may be determined based on the value of the sitting parameter described above, similar to the predetermined forced sitting condition.

[0212] In addition, in the above-described embodiment, a user associated with multiple specific object IDs (see FIG. 6) may be able to specify a specific object M4 on which to seat the user avatar M1, together with or separately from the sitting instruction.

[0213] In the above-described embodiment, a predetermined specific object M4 may be prepared on which multiple user avatars M1 can sit simultaneously. In this case, when a sitting instruction is generated for a user associated with one user avatar M1, and the user avatar M1 is located near the predetermined specific object M4 and there is an empty space on the predetermined specific object M4, the user avatar M1 may be placed in a seated state on the predetermined specific object M4.

[0214] In the above-described embodiment, the difference between the first-person perspective mode and the third-person perspective mode is realized by calculating the values of the various parameters of the virtual camera 60 in different ways, but this is not limited to this. For example, the difference between the first-person perspective mode and the third-person perspective mode may be realized equivalently by using multiple types of cameras with different characteristics as the virtual camera 60. For example, a virtual camera for the first-person perspective mode and a virtual camera for the third-person perspective mode may be prepared, and in this case, the virtual camera for the third-person perspective mode may be set to have a wider angle of view than the virtual camera for the first-person perspective mode. For example, the virtual camera for the third-person perspective mode may have lens characteristics corresponding to a fisheye lens.

[0215] The following additional notes are provided regarding the above-described embodiments of the present invention.

[0216] [Appendix 1] a rendering unit that renders a display image for a terminal including one or more display media located in a virtual space; an acquisition unit that acquires input from a user; a state switching unit that switches a state of the display medium associated with one user between a plurality of states, the plurality of states including a first state in which the position of the display medium can be changed from a current position, and a second state in which the position of the display medium cannot be changed from the current position; a position change unit that changes a position of the display medium associated with the one user in a virtual space based on a first input from the one user acquired by the acquisition unit when the state of the display medium is the first state; an information processing system including a viewpoint switching unit that switches a viewpoint from a first viewpoint to a second viewpoint when the drawing unit draws the display image associated with the one user when the state of the display medium associated with the one user is switched from the first state to the second state by the state switching unit.

[0217] [Appendix 2] 2. The information processing system according to claim 1, wherein the drawing unit draws the display medium in a different manner when the display medium is in the first state and when the display medium is in the second state.

[0218] [Appendix 3] The information processing system according to claim 1 or 2, wherein the state switching unit switches the state of one of the display media, which is in the first state, to the second state when a predetermined first switching condition is met.

[0219] [Appendix 4] The information processing system described in Appendix 3, wherein the state switching unit switches the state of one of the display media, which is in the first state, to the second state in response to a second input from the one user acquired by the acquisition unit, when the display medium associated with the one user is located within a specific area in the virtual space.

[0220] [Appendix 5] The information processing system according to claim 4, wherein the drawing unit draws a user interface for inputting the second input while the display medium is in the first state.

[0221] [Appendix 6] The information processing system described in Appendix 5, further comprising a disabling processing unit that disables the user interface or disables the second input via the user interface when a predetermined disabling condition is met.

[0222] [Appendix 7] 7. The information processing system according to any one of appendices 4 to 6, wherein the specific area includes any current position of the display medium in the first state.

[0223] [Appendix 8] the display medium is capable of rendering a state in which the user is seated on a specific object in a virtual space; 8. The information processing system according to claim 1, wherein the drawing unit draws the display medium in the second state in a manner that represents the seated state.

[0224] [Appendix 9] the specific object is in the form of a chair or the like, The information processing system described in Appendix 8, wherein the drawing unit newly draws the specific object associated with the display medium when switching the state of the display medium from the first state to the second state.

[0225] [Appendix 10] An information processing system as described in any one of Appendices 1 to 9, further comprising an output unit that, when a third input from the one user corresponding to one of the display media in the second state is acquired by the acquisition unit, outputs characters or audio that can be seen or heard by other users other than the one user based on the third input.

[0226] [Appendix 11] the display medium is capable of rendering in a manner that represents an utterance state; The information processing system described in Appendix 10, wherein when the output unit outputs the character or the voice based on the third input from the one user, the drawing unit draws the one display medium associated with the one user in a manner representing the speech state.

[0227] [Appendix 12] The information processing system described in Appendix 10, wherein when the output unit outputs the characters or the audio based on the third input from the one user, the drawing unit emphasizes the one display medium or a predetermined image associated with the one display medium compared to other display media or the predetermined image associated with the other display media.

[0228] [Appendix 13] The display medium has a form including specific parts whose orientation can be changed, An information processing system described in any one of Appendices 1 to 12, wherein when a fourth input from the one user associated with one of the display media in the second state is acquired by the acquisition unit, the drawing unit changes the orientation of the specific part on the one of the display media associated with the one user based on the fourth input.

[0229] [Appendix 14] The information processing system of claim 13, wherein the specific features include a face, an upper body, or an eye.

[0230] [Appendix 15] The information processing system described in Appendix 13 or 14, wherein the viewpoint switching unit changes the viewpoint when the drawing unit draws the display image associated with the one user based on the fourth input.

[0231] [Appendix 16] 16. The information processing system according to any one of appendices 13 to 15, wherein the fourth input and the first input are generated based on the same input by a user.

[0232] [Appendix 17] the first viewpoint includes a viewpoint at which the display medium associated with the one user is rendered; 17. The information processing system according to any one of appendices 1 to 16, wherein the second viewpoint includes a viewpoint seen from the display medium associated with the one user.

[0233] [Appendix 18] An information processing system described in any one of Appendices 1 to 17, wherein the state switching unit switches the state of one of the display media, which is in the second state, to the first state when a predetermined second switching condition is met.

[0234] [Appendix 19] An information processing system described in any one of Appendices 1 to 18, wherein the position change unit invalidates the first input from the one user when the state of the display medium associated with the one user is the second state.

[0235] [Appendix 20] Rendering a display image for a device including one or more display media located in a virtual space; Get input from the user, switching a state of the display medium associated with one user between a plurality of states including a first state in which the position of the display medium can be changed from a current position and a second state in which the position of the display medium cannot be changed from the current position; changing a position of the display medium associated with the one user in a virtual space based on a first input from the one user acquired when the state of the display medium is the first state; When a state of the display medium associated with the one user is switched from the first state to the second state, a viewpoint when drawing the display image associated with the one user is switched from a first viewpoint to a second viewpoint. An information processing program that causes a computer to execute processing.

[0236] [Appendix 21] a rendering unit that renders a display image for a terminal including one or more display media located in a virtual space; an acquisition step of acquiring input from a user; switching a state of the display medium associated with one user between a plurality of states including a first state in which the position of the display medium can be changed from a current position and a second state in which the position of the display medium cannot be changed from a current position; a position change step of changing a position of the display medium associated with the one user in a virtual space based on a first input from the one user acquired by the acquisition step when the state of the display medium is the first state; an information processing method executed by a computer, the method including a step of switching a viewpoint when drawing the display image associated with the one user from a first viewpoint to a second viewpoint when the state of the display medium associated with the one user is switched from the first state to the second state. [Explanation of symbols]

[0237] 1 Virtual reality generation system 3 Network 10 Server device 20 Terminal equipment 60 Virtual Camera 70 Space section 71 Free space section 140 User Database 142 Avatar Database 146 Group State Memory 150 Group setting section 152 User avatar processing unit 1521 Operation input acquisition unit 1522 user action processing section 15220 State switching unit 15221 Basic operation processing unit 158 Terminal image generation unit 1580 Avatar Image Drawing Section 15801 Seated motion drawing section 15802 Rising motion drawing section 1581 Base Image Drawing Unit 1582 User Interface Drawing Unit 1583 Speaker Information Drawing Unit 160 Dialogue processing unit 162 Invalidation processing unit 170 Parameter Update Unit 1700 Normal parameter calculation unit 1701 Seating parameter calculation unit 1702 Viewpoint switching section 1703 Transition parameter calculation unit 240 user database 242 Avatar Database 246 Group State Memory 250 Operation input generation unit 251 Server Information Acquisition Unit 252 User avatar processing unit 254 Friend Avatar Processing Unit 255 Operation input transmission unit 258 Terminal Image Generation Unit 2580 Avatar Image Drawing Section 2581 Base Image Drawing Unit 2582 User Interface Drawing Unit 260 Dialogue Processing Unit 262 Invalidation processing unit 270 Parameter Update Unit 300 Main Interface 301 Chair Button

Claims

1. a rendering unit that renders a display image for a terminal including one or more display media located in a virtual space; an acquisition unit that acquires input from a user; a state switching unit that switches a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; a viewpoint switching unit that switches a viewpoint when the drawing unit draws the display image associated with the one user from a third-person viewpoint mode to a first-person viewpoint mode when the state of the display medium is switched from the first state to the second state by the state switching unit; Equipped with the display medium is capable of rendering a state in which the user is seated on a specific object in a virtual space; The drawing unit draws the display medium, which is in the second state when an end condition for a transition state from the first state to the second state is met, in a manner representing the seated state.

2. a rendering unit that renders a display image for a terminal including one or more display media located in a virtual space; an acquisition unit that acquires input from a user; a state switching unit that switches a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; an information processing system comprising: a viewpoint switching unit that, when the state of the display medium is switched from the first state to the second state by the state switching unit, switches a viewpoint when the drawing unit draws the display image associated with the one user from a third-person viewpoint mode to a first-person viewpoint mode; and an output unit that, when a third input from the one user associated with one of the display media in the second state is acquired by the acquisition unit, outputs characters or audio that can be seen or heard by another user different from the one user based on the third input, in a manner depending on whether the other display medium associated with the other user is in the first state or the second state.

3. The information processing system according to claim 1 , wherein the drawing unit draws the image on one of the display media in different modes when the display media is in the first state and when the display media is in the second state.

4. the display medium is capable of rendering a state in which the seated person is not seated; The information processing system according to claim 1 , wherein the drawing unit draws the display medium in the first state in a manner that represents the non-seated state.

5. the display medium can be rendered in a manner representing a non-seated state or a manner representing a seated state in which the user is seated on a specific object in a virtual space; 4. The information processing system according to claim 2, wherein the drawing unit draws the display medium in the first state in a manner representing the non-seated state, and draws the display medium in the second state in a manner representing the seated state.

6. The information processing system according to claim 1 , wherein the state switching unit switches the state of one of the display media, which is in the first state, to the second state when a predetermined first switching condition is met.

7. The information processing system of claim 6, wherein the state switching unit switches the state of the display medium, which is in the first state, to the second state in response to a second input from the user acquired by the acquisition unit when the display medium associated with the user is located within a specific area in the virtual space.

8. The information processing system according to claim 1 , wherein the state switching unit switches the state of one of the display media, which is in the second state, to the first state when a predetermined second switching condition is met.

9. The information processing system according to claim 1 , wherein the display medium has a configuration including a specific part whose orientation can be changed in the second state.

10. 10. The information processing system of claim 9, wherein when a fourth input from the one user associated with the one display medium is acquired by the acquisition unit, the drawing unit changes the orientation of the specific part on the one display medium associated with the one user based on the fourth input.

11. the display medium is capable of rendering in a manner that represents an utterance state; 10. The information processing system of claim 9, wherein when one of the display media is in the speaking state, the drawing unit identifies the display medium related to a dialogue partner based on an analysis of the dialogue content of the one of the display media, and changes the orientation of the specific part of the one of the display media so that the specific part faces the display medium related to the identified dialogue partner.

12. the display medium is capable of rendering in a manner that represents an utterance state; The information processing system of claim 9, wherein the drawing unit, when representing the speech state of the one of the display media, uses a face tracking function to recognize various movements of the face of the one of the users associated with the one of the display media.

13. The information processing system of claim 12 , wherein the face tracking function operates in the second state and does not operate in the first state.

14. the third-person viewpoint mode includes a viewpoint at which the display medium associated with the one user is rendered; The information processing system according to claim 1 , wherein the first-person perspective mode includes a perspective seen from the display medium associated with the one user.

15. The information processing system according to claim 1 , wherein the termination condition is met when a viewpoint when the drawing unit draws is switched from a third-person viewpoint mode to a first-person viewpoint mode.

16. Rendering a display image for a terminal including one or more display media located in a virtual space; Get input from the user, switching a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; when the state of the display medium is switched from the first state to the second state, switching a viewpoint when drawing the display image associated with the one user from a third-person viewpoint mode to a first-person viewpoint mode; the display medium is capable of rendering a state in which the user is seated on a specific object in a virtual space; an end condition for a transition state from the first state to the second state is satisfied, and the display medium in the second state is drawn in a manner representing the seated state.

17. a rendering step of rendering a display image for a terminal including one or more display media located in a virtual space; an acquisition step of acquiring input from a user; switching a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; when the state of the display medium is switched from the first state to the second state, switching a viewpoint when drawing the display image associated with the one user in the drawing step from a third-person viewpoint mode to a first-person viewpoint mode; Equipped with the display medium is capable of rendering a state in which the user is seated on a specific object in a virtual space; an information processing method executed by a computer, wherein in the drawing step, an end condition of the transition state from the first state to the second state is met and the display medium in the second state is drawn in a manner representing the seated state.

18. Rendering a display image for a terminal including one or more display media located in a virtual space; Get input from the user, switching a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; when the state of the display medium is switched from the first state to the second state, switching a viewpoint when drawing the display image associated with the one user from a third-person viewpoint mode to a first-person viewpoint mode; An information processing program that causes a computer to execute a process in which, when a third input is obtained from the one user associated with one of the display media in the second state, characters or audio that can be seen or heard by another user different from the one user are output based on the third input in a manner depending on whether the other display medium associated with the other user is in the first state or the second state.

19. a rendering step of rendering a display image for a terminal including one or more display media located in a virtual space; an acquisition step of acquiring input from a user; switching a state of the display medium associated with one user between a plurality of states including at least a first state and a second state; when the state of the display medium is switched from the first state to the second state, switching a viewpoint when drawing the display image associated with the one user in the drawing step from a third-person viewpoint mode to a first-person viewpoint mode; An information processing method executed by a computer, comprising: when a third input is obtained from the one user associated with one of the display media in the second state, an output step of outputting, based on the third input, characters or audio that can be seen or heard by another user different from the one user, in a manner depending on whether the other display medium associated with the other user is in the first state or the second state.

Images