Information processing system, information processing method, and program for communication point on event

Abstract

There is provided an information processing apparatus including circuitry configured to provide a user with information regarding a selection of entering a communication room based on an avatar operated by the user entering a first area that is a communication point at which communication between users is performed, where the communication room is related to the communication point, and where the communication room is not related to the communication point. The communication system includes a virtual object that includes a display object that displays information about an event occurring in a real space or a virtual space, the event being a subject of a conversation in a communication room, and provides information about communication based on an avatar entering a second area located outside the first area, the virtual object including a display object that displays information about the event occurring in the real space or the virtual space, and provides information about the communication based on the avatar entering a second area located outside the first area.

Description

[0001] Cross - reference to related applications

[0002] This application claims the benefit of Japanese Priority Patent Application JP 2022 - 181346, filed on November 11, 2022, the entire content of which is incorporated herein by reference. Technical field

[0003] The present disclosure relates to an information processing system, an information processing method, and a program. Background art

[0004] In recent years, as one of real - time communications using images, audio, or characters, a system that can perform communication through avatars operated by each user in a virtual space provided with a three - dimensional model has been widely used. For example, in such a system, a video is generated from an arbitrary perspective (free perspective) according to the user's operation and provided to the user as a video of the virtual space. The video of the virtual space is provided through a display device such as a head - mounted display (HMD), a smart phone, a tablet terminal, or a personal computer (PC) that covers the entire field of view of the user.

[0005] Regarding communication between users performed in a virtual space, for example, the following PTL 1 discloses a technique for establishing communication between communication terminals used by respective users in a case where scheduled users gather at a predetermined space in the virtual space at a predetermined time.

[0006] Citation list

[0007] Patent literature

[0008] PTL 1: WO 2013 / 054748 Summary of the invention

[0009] Technical problem

[0010] However, in the above - mentioned technology, it is necessary to pre - determine the participants, space, and time of communication. Further convenience of communication between some users in the virtual space may be required.

[0011] Therefore, the present disclosure proposes an information processing system, an information processing method, and a program that can further enhance the convenience of communication between some users at a predetermined location in a virtual space.

[0012] Solution to the problem

[0013] According to one aspect of the present disclosure, there is provided an information processing apparatus including: a circuitry configured to provide information about a choice of entering a communication room to a user based on an avatar operated by the user entering a first area that is a communication point for performing communication between users, wherein the communication room is related to the communication point and includes a virtual object including a display object that displays information about an event occurring in a real space or a virtual space; and provide information about communication based on the avatar entering a second area located outside the first area, where the event is a topic of conversation in the communication room.

[0014] In addition, according to another aspect of the present disclosure, there is provided an information processing method including: providing information about a choice of entering a communication room to the user based on an avatar operated by the user entering a first area that is a communication point for performing communication between users, wherein the communication room is related to the communication point and includes a virtual object including a display object that displays information about an event occurring in a real space or a virtual space; and providing information about communication based on the avatar entering a second area located outside the first area, where the event is a topic of conversation in the communication room.

[0015] In addition, according to another aspect of the present disclosure, there is provided an information processing apparatus including: a circuitry configured to:

[0016] provide a virtual object including a display object that displays information about an event occurring in a real space or a virtual space in a communication room related to a communication point for performing communication between users, and provide information about a choice of entering the communication room to the user based on the avatar operated by the user entering a first area that is the communication point for performing communication between users,

[0017] wherein information about communication is provided based on the avatar entering a second area located outside the first area, and the event is a topic of conversation in the communication room.

[0018] In addition, according to another aspect of the present disclosure, there is provided an information processing method including: providing a virtual object including a display object that displays information about an event occurring in a real space or a virtual space in a communication room related to a communication point for performing communication between users, providing information about a choice of entering the communication room to the user based on the avatar operated by the user entering a first area that is the communication point for performing communication between users, providing information about communication based on the avatar entering a second area located outside the first area, and the event is a topic of conversation in the communication room. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Figure 1 is a diagram showing the overall configuration of a virtual space providing system according to an embodiment of the present disclosure.

[0020] Figure 2 Figure 2 is a diagram for explaining each space included in the virtual space S.

[0021] Figure 3 Figure 3 is a diagram for explaining the spaces by type included in the motion space S3 according to an embodiment of the present disclosure.

[0022] Figure 4 Figure 4 is a block diagram showing an example of the configuration of the virtual space management server 200 according to an embodiment of the present disclosure.

[0023] Figure 5 Figure 5 is a diagram showing a specific example of the service server included in the backend system 30 according to an embodiment of the present disclosure.

[0024] Figure 6 Figure 6 is a block diagram showing an example of the configuration of the client terminal 10 according to an embodiment of the present disclosure.

[0025] Figure 7 Figure 7 is a sequence diagram showing an example of the process of connection processing to a virtual space according to an embodiment of the present disclosure.

[0026] Figure 8 Figure 8 is a diagram showing an example of a display screen according to an embodiment of the present disclosure.

[0027] Figure 9 Figure 9 is a diagram showing an example of a display screen according to an embodiment of the present disclosure.

[0028] Figure 10 Figure 10 is a diagram showing an example of a simple map of an IP content specific area according to an embodiment of the present disclosure.

[0029] Figure 11 Figure 11 is a diagram of a mechanism for reproducing a game using the skeletal data of a player according to an embodiment of the present disclosure.

[0030] Figure 12 ​​​​​​​​​​​​​​​​​​​​​​​​Figure 12 It is a view showing an interior view of a bar with a conversation table 600 provided thereon according to an embodiment of the present disclosure.

[0031] Figure 13 Figure 13 It is a view for explaining an example of the shape of the conversation table 600.

[0032] Figure 14 Figure 14 It is a view for describing a situation of disclosing a state in a conversation room according to a distance to the conversation table 600 according to an embodiment of the present disclosure.

[0033] Figure 15 Figure 15 It is a view showing a display screen of a terminal (client terminal 10) used by a user operating a user avatar when the position of the user avatar in the virtual space includes the area E2.

[0034] Figure 16 Figure 16 It is a view showing a display screen of a terminal (client terminal 10) used by a user operating a user avatar when the position of the user avatar in the virtual space includes the area E1.

[0035] Figure 17 Figure 17 It is a view showing an example of the effect display of the conversation table 600 according to the excitement level in the conversation room according to an embodiment of the present disclosure.

[0036] Figure 18 Figure 18 It is a sequence diagram showing an example of an operation process from the generation to the entry of the conversation table 600 according to an embodiment of the present disclosure.

[0037] Figure 19 Figure 19 It is a view showing details of a communication connection between the client terminal 10 and each server when entering the conversation room according to an embodiment of the present disclosure.

[0038] Figure 20 Figure 20 It is a view showing an example of a conversation room screen according to an embodiment of the present disclosure.

[0039] Figure 21 Figure 21 It is a view showing another example of a conversation room screen according to an embodiment of the present disclosure.

[0040] Figure 22 Figure 22 ​​​​​​​​​​​​​​​​​​​​It is a diagram for explaining a mechanism for event coordination according to an embodiment of the present disclosure.

[0041] Figure 23 Figure 23 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0042] Figure 24 Figure 24 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0043] Figure 25 Figure 25 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0044] Figure 26 Figure 26 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0045] Figure 27 Figure 27 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0046] Figure 28 Figure 28 It is a diagram showing an example of a display screen in a live stadium according to an embodiment of the present disclosure.

[0047] Figure 29 Figure 29 It is a diagram showing an example of the position of a camera device according to an embodiment of the present disclosure.

[0048] Figure 30 Figure 30 It is a diagram showing Figure 29 the game reproduction UI in the basic case of the camera device position 810 shown.

[0049] Figure 31 Figure 31 It is a diagram showing Figure 29 the game reproduction UI in the bird's-eye view case of the camera device position 811 shown.

[0050] Figure 32 Figure 32 It is a diagram for describing an overview of the screen of the game reproduction UI according to an embodiment of the present disclosure.

[0051] Figure 33 Figure 33 It is a diagram showing an example of the display of a chapter list according to an embodiment of the present disclosure.​​​​​​​​​​​​​​​​​​​​​​

[0052] Figure 34 Figure 34 is a diagram showing a display example of a game reproduction UI in the case of showing an avatar mode according to an embodiment of the present disclosure.

[0053] Figure 35 Figure 35 is a diagram for explaining a highlight playback of a game according to an embodiment of the present disclosure. Detailed Embodiments

[0054] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0055] In addition, the description is given in the following order.

[0056] 1. Overall configuration of a virtual space providing system according to an embodiment of the present disclosure

[0057] 2. Configuration examples

[0058] 2-1. Configuration example of the virtual space management server 200

[0059] 2-2. Configuration example of the backend system 30

[0060] 2-3. Configuration example of the client terminal 10

[0061] 3. Connection to the virtual space

[0062] 4. Display screen examples

[0063] 5. Specific examples of the virtual space

[0064] 5-1. IP content area

[0065] 5-2. Conversation table

[0066] 5-3. Event coordination

[0067] 5-4. Game reproduction UI

[0068] 6. Supplementary

[0069] <<1. Overall configuration of a virtual space providing system according to an embodiment of the present disclosure>>

[0070] Figure 1 ​​​​This is a diagram showing the overall configuration of a virtual space providing system according to an embodiment of the present disclosure. The virtual space providing system (an example of an information processing system) according to an embodiment of the present disclosure is an information processing system including one or more virtual space management servers 200 (an example of an information processing device), client terminals 10 (an example of a terminal) used by each user, and a backend system 30.

[0071] The virtual space management server 200 constructs (generates) and manages a virtual space (VR) in which virtual objects are set, and provides virtual space information to one or more client terminals 10 communicably connected via a network 40. For example, the virtual space information includes information about virtual objects set in the virtual space, map information about the virtual space, information about events (hereinafter also referred to as virtual events) occurring in the virtual space, position information about each user in the virtual space, and the like. Each client terminal 10 can reproduce (duplicate) the virtual space based on this information. Specifically, each client terminal 10 arranges virtual objects based on the map information about the virtual space, and draws a video of the virtual space according to the user's viewpoint (user avatar viewpoint). Note that the virtual space management server 200 can generate a video from any viewpoint (e.g., the user's viewpoint) in the virtual space to send the video to the client terminal 10.

[0072] In the virtual space, for example, virtual objects formed by 3DCG models are set. The virtual space can be a space in which the actions of the user are restricted to a pre-prepared behavior scenario such as a game, or can be a space in which the user can act freely without being restricted to a behavior scenario. In addition, the virtual space can be a space in which interaction with other users can be performed by two-way transmission and reception of audio, characters, etc. In addition, the virtual space can be a space designed according to an original worldview, or can be a space called a so-called mirror world, which faithfully reproduces buildings, facilities, roads, cities, streets, etc. existing in the real space. In addition, in the virtual space, interaction with other users, exchange of education, social activities such as work, and economic activities such as selling or buying products can also be performed. The virtual space can be a space called the metaverse.

[0073] In a virtual space, there can exist a user avatar that serves as the virtual self of a user. The user avatar is generated, for example, by a character 3D model. Each user can arbitrarily customize the appearance of his / her character 3D model. The user avatar can be operated by the user and moved in the virtual space. In addition, the gestures or facial expressions of the user avatar can also be operated by the user. In this specification, the position information of the user in the virtual space can be information indicating the position of the user avatar. In addition, the user viewpoint in the virtual space can be the viewpoint of the user avatar or can be the viewpoint within the viewing range that includes the user avatar. The user viewpoint can be arbitrarily switched by the user operation.

[0074] The user avatar is one of the virtual objects set in the virtual space. Information about the user avatar of each user is also shared as virtual space information by a large number of client terminals 10 communicably connected to the virtual space management server 200. In addition, each user can communicate through audio, characters, or images in the virtual space. Communication can be performed via the virtual space management server 200 or via the backend system 30. The backend system 30 includes servers (service servers 31, 32, 33,...) that provide various services to users using the virtual space. Examples of the servers include a server that provides audio communication, a server that provides character communication, and a server that provides image (moving image and still image) communication. The client terminal 10 obtains communication channel information (an example of connection information) from the virtual space management server 200 and is communicably connected to a predetermined service server. Then, the client terminal 10 obtains communication channel information via the service server and can communicate with another client terminal 10 communicably connected to the predetermined service server.

[0075] In addition, in the virtual space providing system according to an embodiment of the present disclosure, in order to distribute the processing load of the virtual space management server 200, servers can be multiplexed according to the number of users in the virtual space (i.e., the number of terminals connected to the virtual space management server 200). In Figure 1In the illustrated example, the virtual space system 20 includes a plurality of virtual space management servers 200 (200a, 200b, 200c, ......). For example, when it is desired to allow 1000 people to use a certain virtual space, 10 virtual space management servers 200 that can be connected to 100 terminals are prepared, and each virtual space management server 200 provides the same virtual space (the same virtual objects and the same map information). Each virtual space management server 200 manages position information about users corresponding to 100 terminals connected for communication (position information about 100 people). Therefore, the same virtual space can be provided for a total of 1000 users, with 100 users per virtual space. In each virtual space, rough elements such as the flow of time and the execution of virtual events are synchronized, but interactions between users can be performed for each virtual space.

[0076] Reusing servers according to the number of connected terminals - that is, server increase / decrease management - can be performed by the service provided by the backend system 30. Specifically, when the number of terminals connected to one virtual space management server 200 exceeds a predetermined number (e.g., 100), the backend system 30 can construct a new virtual space management server 200 that provides the same virtual space (the same virtual objects and the same map information) as one virtual space management server 200. Note that a predetermined number of virtual space management servers 200 can be constructed in advance.

[0077] There is no particular limitation on the server environment. For example, the virtual space management server 200 can be a physical server or a virtual server executed on a physical server. The physical server or virtual server described here can be a server provided by a hosting service or can be the own server prepared by a business operator that provides a service for providing virtual spaces. In addition, the functions of one virtual space management server 200 can be implemented by multiple physical servers or multiple virtual servers. Each virtual space management server 200 manages position information about users corresponding to a plurality of client terminals 10 connected for communication.

[0078] The client terminal 10 is a terminal used by a user. For example, the client terminal 10 is implemented by a smart phone, a tablet terminal, a personal computer (PC), a head-mounted display (HMD) covering the entire field of view, a glasses-type device, a projector, a console game machine, or the like. The client terminal 10 reproduces a virtual space based on information about the virtual space received from the virtual space management server 200. In addition, the client terminal 10 generates a video of the user's viewpoint in the virtual space, and displays and outputs the video. In addition, the client terminal 10 appropriately outputs audio in the virtual space. Note that the client terminal 10 may receive a video according to the user's viewpoint generated by the virtual space management server 200 from the virtual space management server 200, and display and output the video.

[0079] The client terminal 10 can control virtual objects set in the virtual space according to the user's operations. For example, the client terminal 10 controls the position, gestures, facial expressions, etc. of the user avatar according to the user's operations. In addition, the client terminal 10 sends information about the virtual objects changed according to the user's operations (for example, the position, gestures, facial expressions, etc. of the user avatar) to the virtual space management server 200 in real time.

[0080] In addition, the client terminal 10 can send various types of information to the virtual space management server 200, another client terminal 10, or the backend system 30 according to the user's operations. For example, the client terminal 10 can send login information to the virtual space management server 200 or the backend system 30 to request user authentication. In addition, the client terminal 10 can send the user's audio, input character information, selected images, etc. as communication information to the virtual space management server 200, another client terminal 10, or the backend system 30.

[0081] (Content-specific virtual space)

[0082] As an example of the virtual space, in the embodiments of the present disclosure, there is a space where users interact with each other centering on various contents such as movies, music, sports, or animations (including IP contents described later). Figure 2 is a diagram for explaining each space included in the virtual space S. As Figure 2As shown, the virtual space S provided in the virtual space system 20 according to an embodiment of the present disclosure includes a user personal space S1 as a virtual space for an individual user, a public space S2 as a virtual space in which users interact with each other regardless of the content, and each virtual space in which users interact with each other regarding each content. Examples of each virtual space in which users interact with each other regarding each content include: a sports space S3, which is a virtual space in which users interact with each other regarding sports; a music space S4, which is a virtual space in which users interact with each other regarding music; a movie space S5, which is a virtual space in which users interact with each other regarding movies; and an animation space S6, which is a virtual space in which users interact with each other regarding animations. The content mentioned here is an example, and the content according to the present disclosure is not limited thereto.

[0083] In addition, each virtual space in which users interact with each other regarding each content may be each type of virtual space that is further hierarchical. Figure 3 is a diagram for explaining the space by type included in the sports space S3 according to an embodiment of the present disclosure. As Figure 3 shown, examples thereof include a football space S3-1 in which users interact with each other particularly regarding football, a basketball space S3-2 in which users interact with each other particularly regarding basketball, and a tennis space S3-3 in which users interact with each other particularly regarding tennis. In addition, Figure 3 examples of the virtual space in the lower hierarchical structure of the space by each sports type shown in may include a virtual space in which users (fans) of each sports team interact with each other. Note that the virtual space is not limited to being hierarchical by type. The virtual space may be a space in which each space corresponding to various IP contents exists in parallel. For example, the virtual space according to an embodiment of the present disclosure may be a virtual space in which spaces corresponding to each of a certain baseball team, a certain football team, and a certain animation exist in parallel.

[0084] In this way, as a virtual space by content, a virtual space in which users interested in the content want to interact with each other can be considered. A user can have a conversation with another user having the same preference and the same interest regarding the content, and enjoy the content more.

[0085] The virtual space system 20 can provide Figure 2 and Figure 3 all or some of the virtual spaces shown in. For example, the virtual space system 20 can prepare a user personal space S1, a public space S2, and a virtual space for interaction between fans of a specific sports team. In an embodiment of the present disclosure, as an example, the virtual space system 20 provides a virtual space for interaction between fans of a specific football team.

[0086] <<2. Configuration Example>>

[0087] <2-1. Configuration Example of Virtual Space Management Server 200>

[0088] Figure 4 is a block diagram showing an example of the configuration of the virtual space management server 200 according to an embodiment of the present disclosure. As Figure 4 shown, the virtual space management server 200 includes a communication unit 210, a control unit 220, and a storage unit 230.

[0089] (Communication Unit 210)

[0090] The communication unit 210 transmits data to an external device and receives data from the external device in a wired or wireless manner. The communication unit 210 is communicably connected to each of the client terminal 10 and the backend system 30 by using, for example, a wired / wireless local area network (LAN), Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network (Long Term Evolution (LTE), Fourth Generation Mobile Communication System (4G), Fifth Generation Mobile Communication System (5G)), etc.

[0091] (Control Unit 220)

[0092] The control unit 220 serves as an arithmetic processing device and a control device, and controls the overall operation in the virtual space management server 200 according to various programs. For example, the control unit 220 is implemented by an electronic circuit such as a central processing unit (CPU) or a microprocessor. In addition, the control unit 220 may include a read-only memory (ROM) that stores programs, operation parameters, etc. to be used, and a random access memory (RAM) that temporarily stores appropriately changed parameters, etc.

[0093] The control unit 220 appropriately performs processing based on the data received from the external device, and controls the storage in the storage unit 230 and the transmission of data to the external device.

[0094] The control unit 220 also serves as a virtual space management unit 221, a content management unit 222, and a user management unit 223.

[0095] The virtual space management unit 221 manages the virtual space. Specifically, the virtual space management unit 221 performs generation, acquisition, addition, update, etc. on various types of information for constructing (generating) the virtual space. The various types of information are virtual space information, for example, information about a 3DCG model of a virtual object set in the virtual space, color settings, images, map information, virtual event information, sound effects, and background music (BGM).

[0096] The virtual objects include user-operable objects, user-inoperable objects, background images, performance effects, etc. The virtual objects also include information about each user avatar set in the virtual space. Examples of information about each user avatar include appearance information of the user avatar, real-time user location information, real-time facial expression or gesture information of the user avatar, and the profile of the corresponding user. The information about each user avatar is managed by the user management unit 223 described later.

[0097] The virtual event information includes information about the time, progress, performance, etc. of the virtual event. The virtual space management unit 221 can notify the client terminal 10 to execute a predetermined virtual event at a preset time, or can notify the client terminal 10 to execute a virtual event corresponding to the situation where a predetermined trigger is detected in the virtual space. The virtual event information includes the information required to execute the virtual event. In addition, the virtual event can be content. Notifying the client terminal 10 of the virtual event may include information indicating to obtain predetermined content from the backend system 30 and execute the content at a predetermined location in the virtual space.

[0098] The virtual space management unit 221 appropriately sends the virtual space information to the client terminal 10. In addition, the virtual space management unit 221 can also enable communication between users sharing the virtual space by sending communication information such as characters, audio, or images received from the client terminal 10 to another client terminal 10.

[0099] The virtual space management unit 221 receives information about user operations from the client terminal 10 to send the information to other client terminals 10 in real time as virtual space information. In addition, the virtual space management unit 221 performs a process of detecting computational conflicts based on the information about user operations received from the client terminal 10 to send the calculation results to each client terminal 10 at an appropriate time. Thus, the states of the virtual spaces reproduced by the corresponding client terminals 10 are synchronized. The information about user operations may include: operations related to the movement of the user avatar, such as the movement and jump of the user avatar; and operations related to the emotional expression of the user avatar, such as making the user avatar perform poses, gestures, etc.; and stamps displayed above the head of the user avatar.

[0100] The content management unit 222 manages information about the content shared in the virtual space. For example, the content management unit 222 manages information indicating the content shared in the content sharing area currently provided in the virtual space. For example, the content sharing area may be a virtual object of one or more large displays set in the virtual space, or may be a stage, forum, venue, etc. on which 3D content or the like is reproduced. The content itself can be sent from the backend system 30 to the client terminal 10. Such content management can be managed by the virtual space management unit 221 as one of the above virtual event information.

[0101] The user management unit 223 manages information about the users who use the virtual space provided by the virtual space management server 200. More specifically, the user management unit 223 manages information about the users corresponding to the client terminals 10 to be connected for communication. Information about the users includes, for example, user profiles (user names, user IDs, icon images, etc.), appearance information about user avatars (images, character types, parts, etc.), status information about user avatars, information about virtual items owned by the users, information about virtual currency or points available to the users in the virtual space, real-time user location information in the virtual space (i.e., location information about user avatars), and motion information about the real-time gestures, facial expressions, postures, or motion states of user avatars. Information about the users is transmitted from the client terminal 10. In addition, a part of the information about the users can be sent from the backend system 30.

[0102] (Storage unit 230)

[0103] The storage unit 230 is implemented by a ROM that stores programs, operation parameters, etc. for the processing of the control unit 220 and a RAM that temporarily stores appropriately changed parameters, etc. The storage unit 230 stores virtual space information.

[0104] <2-2. Configuration example of the backend system 30>

[0105] The backend system 30 includes servers that provide various services. Each server can be provided by a different business operator. In addition, the services provided by each server can be appropriately customized and used by the business operator providing the virtual space.

[0106] Each server of the backend system 30 can be communicably connected to the virtual space management server 200 to send and receive information, or can communicate with the client terminal 10 to send and receive information. The virtual space management server 200 can cooperate with each server to provide various services to the users who use the virtual space.

[0107] Figure 5It is a diagram showing a specific example of a service server included in the backend system 30 according to an embodiment of the present disclosure. Figure 5 The service server shown in Figure 5 is an example, and the present disclosure is not limited thereto. Each service server will be described below.

[0108] The user information management server 301 manages information about users who use the virtual space. For example, the user information management server 301 stores in a data memory (storage unit) information about a user's account, the user's profile, user characteristics (age, gender, country, language used, history, likes / preferences, viewing history of content, participation history in virtual events, interaction history in the virtual space, etc.), information about items owned by the user, appearance information about the user's avatar, status information about the user's avatar, etc. In addition, the user information management server 301 can also perform authentication on the user using the user's account information. Although details will be described later with reference to Figure 7 the user information management server 301 performs user authentication in response to a request from the client terminal 10, and in the case of successful authentication, calls user information such as the user's profile and appearance information about the user's avatar from the data memory to send the user information to the client terminal 10.

[0109] The hosting server 302 performs server multiplexing, that is, server increase / decrease management, according to the number of terminals connected to the virtual space system 20. For example, when the number of terminals connected to one virtual space management server 200 exceeds a predetermined number (e.g., 100), the hosting server 302 constructs a new virtual space management server 200 that provides the same virtual space (the same virtual objects and the same map information) as one virtual space management server 200.

[0110] In the case where there are multiple virtual space management servers 200 providing the same virtual space, the matching server 303 determines to which virtual space management server 200 the client terminal 10 that makes a connection request after completing user authentication is to be connected. As described above, for example, in the case where the connection number is limited for each virtual space management server 200, the matching server 303 obtains information about the current connection number of each virtual space management server 200 from the virtual space system 20, and instructs the client terminal 10 to communicatively connect to the virtual space management server 200 having free space. In addition, in addition to the number of connections, the matching server 303 can also determine the virtual space management server 200 to be connected according to user information. For example, the matching server 303 can instruct users with similar user characteristics to communicatively connect to the same virtual space management server 200.

[0111] The text chat server 304 is a server that provides a mechanism in which users of the virtual space use text (character information) as a means of communication to have conversations. Specifically, the text chat server 304 creates virtual conversation rooms (text channels) and performs control so that character information can be exchanged between the users in the conversation rooms. The conversation rooms are generated in response to requests from the virtual space management server 200 or the client terminal 10. In addition, the text chat server 304 returns session information (connection information) for connecting to the generated conversation rooms to the virtual space management server 200 or the client terminal 10. The client terminal 10 is communicably connected to the text chat server 304 based on the session information and performs text chat.

[0112] The voice chat server 305 is a server that provides a mechanism in which users of the virtual space use voice (audio information) as a means of communication to have conversations. Specifically, the voice chat server 305 creates virtual conversation rooms (voice channels) and performs control so that audio information can be exchanged between the users in the conversation rooms. The conversation rooms are generated in response to requests from the virtual space management server 200 or the client terminal 10. In addition, the voice chat server 305 returns session information (connection information) for connecting to the generated conversation rooms to the virtual space management server 200 or the client terminal 10. The client terminal 10 is communicably connected to the voice chat server 305 based on the session information and performs voice chat.

[0113] The means of communication between users of the virtual space is not limited to the above-mentioned character information and audio information, but for example, it is also possible to assume the use of moving images (video / streaming video). Although not shown in Figure 5 , for example, a server that provides a mechanism for having conversations using moving images (video / streaming video) can be included in the backend system 30.

[0114] The content distribution server 306 controls the distribution of the content viewed in the virtual space. The content distribution server 306 can obtain and distribute content from various content servers (servers that store content). In an embodiment of the present disclosure, an application (also referred to as a client application) operating in each client terminal 10 reproduces the virtual space (sets virtual objects based on map information) and generates a virtual space video according to the user's viewpoint. The content distribution server 306 is communicably connected to the client terminal 10 to send the content included in the virtual space reproduced by the client terminal 10 to the client terminal 10. For example, the content included in the virtual space is a moving image displayed on a large display (virtual object) set in the virtual space. In addition, the content included in the virtual space is, for example, a 3D video displayed in a predetermined venue in the virtual space.

[0115] The conversation table server 307 manages and sets up a conversation room (communication room) associated with a conversation table (virtual object) at any location in the virtual space. In the conversation room, each user can have a conversation using one or more different types of communication means (e.g., text chat and voice chat). The generation of the conversation room is executed in response to a request from the virtual space management server 200 or the client terminal 10. In addition, the conversation table server 307 returns session information (connection information) for connecting to the generated conversation room to the virtual space management server 200 or the client terminal 10. Details of the conversation room will be described below with reference to Figures 12 to 19 Note that the conversation table is an example of a virtual object that is set at a communication point as a marker where multiple users gather to interact in the virtual space, and the communication point is set at any location in the virtual space. Any virtual object used as a marker can be set at the communication point, and for example, it can also be a chair, pole, parasol, stage, etc. other than a table.

[0116] The event coordination server 308 controls the performances in the virtual space according to the development of events occurring in the real space. The performances in the virtual space can include virtual events. For example, the event coordination server 308 can coordinate the performances in the virtual space with the events in the real space by executing corresponding performances in the virtual space in real time according to the development of a competition being executed in the real space. More specifically, the event coordination server 308 instructs the client terminal 10 to execute the performance.

[0117] Therefore, users can, while watching a competition (an example of an event in the real space) being executed in the real space, for example, enjoy an exciting experience with other users who are watching the same competition in the virtual space. Users can watch the competition in the real space on TV, the Internet, radio, etc., or can watch the competition in the virtual space. Details of event coordination will be described below with reference to Figures 20 to 26 Note that the conversation table is an example of a virtual object that is set at a communication point as a marker where multiple users gather to interact in the virtual space, and the communication point is set at any location in the virtual space. Any virtual object used as a marker can be set at the communication point, and for example, it can also be a chair, pole, parasol, stage, etc. other than a table.

[0118] The transaction management server 309 implements transactions of items, products, tickets, etc. in the virtual space and stores data regarding the transactions. For example, virtual currency can be used for transactions in the virtual space. The transaction management server 309 can be a server that provides a charging system.

[0119] The data analysis server 310 analyzes various data, such as the behavior of each user avatar in the virtual space, the user characteristics of the user avatars that perform specific actions, etc. The analysis results can be presented to business operators who provide the virtual space, etc.

[0120] <2-3. Configuration example of the client terminal 10>

[0121] Figure 6 is a block diagram showing an example of the configuration of the client terminal 10 according to an embodiment of the present disclosure. As Figure 6 shown, the client terminal 10 includes a communication unit 110, a control unit 120, an operation input unit 130, a sensor 140, a display unit 150, an audio output unit 160, and a storage unit 170.

[0122] (Communication unit 110)

[0123] The communication unit 110 is communicably connected to the virtual space server 200 in a wired or wireless manner to send and receive data. The communication unit 110 can perform communication using, for example, a wired / wireless local area network LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), infrared communication, a mobile communication network (fourth-generation mobile communication system (4G), fifth-generation mobile communication system (5G)), etc.

[0124] (Control unit 120)

[0125] The control unit 120 serves as an arithmetic processing device and a control device, and controls the overall operation in the client terminal 10 according to various programs. The control unit 120 is implemented by an electronic circuit such as a CPU or a microprocessor, for example. In addition, the control unit 120 may include a ROM that stores programs, operation parameters, etc. to be used, and a RAM that temporarily stores appropriately changed parameters, etc.

[0126] The control unit 120 also serves as a virtual space processing unit 121 and a display control unit 122.

[0127] The virtual space processing unit 121 performs various processes for providing a virtual space experience to the user in appropriate cooperation with the virtual space management server 200 and the backend system 30. Such processes can be executed by a client application pre-downloaded by the client terminal 10. For example, the virtual space processing unit 121 executes a user registration process and a login process to use the virtual space. For example, in the user registration process and the login process, data is appropriately sent to and received from the user information management server 301 via the communication unit 110. The virtual space processing unit 121 generates a virtual space based on the virtual space information received from the virtual space management server 200. Specifically, the virtual space processing unit 121 arranges virtual objects based on the map information received from the virtual space management server 200 and reproduces the virtual space.

[0128] In addition, the virtual space processing unit 121 can also incorporate various information obtained from the backend system 30 into the virtual space. For example, the virtual space processing unit 121 can display the video distributed from the content distribution server 306 on a large display provided in the virtual space.

[0129] In addition, the virtual space processing unit 121 controls the position, facial expression of the face, gestures (emotional expressions made by hands or the whole body), and motion states (sitting, standing, jumping, running, etc.) of the user avatar provided in the virtual space according to user operations. The virtual space processing unit 121 continuously sends user operation information to the virtual space management server 200 in real time. The virtual space processing unit 121 reflects the information about another user avatar continuously sent from the virtual space management server 200 on another user avatar provided in the virtual space in real time. The virtual space processing unit 121 can realize audio conversations between users by sending the audio of the user obtained by the sensor 140 to the virtual space management server 200 and outputting the audio of another user received from the virtual space management server 200 from the audio output unit 160.

[0130] In addition, the virtual space processing unit 121 generates a video (display screen) of the user's viewpoint in the virtual space. In addition, the virtual space processing unit 121 can also generate a display screen in which user avatars operate buttons, setting screens, notification screens, menu screens, etc. are superimposed on the video from the user's viewpoint. In addition, the virtual space processing unit 121 can display various information obtained from the backend system 30 on the display screen. For example, the virtual space processing unit 121 can display the screen of the text chat between users executed via the text chat server 304 based on the information received from the text chat server 304. The text chat screen can be updated in real time.

[0131] In addition, the virtual space processing unit 121 can realize audio conversations between users via the voice chat server 305 by sending the audio of the user obtained by the sensor 140 to the voice chat server 305 and outputting the audio of another user received from the voice chat server 305 from the audio output unit 160.

[0132] Note that in the embodiments of the present disclosure, the case where the virtual space is reproduced in the virtual space processing unit 121 will be described, but the present disclosure is not limited thereto. For example, the virtual space management server 200 can generate and send (stream) a video according to the user's viewpoint in the virtual space, and the virtual space processing unit 121 can display the video received from the virtual space management server 200 on the display unit 150, so that a virtual space experience can be provided to the user.

[0133] The display control unit 122 performs control to display an image on the display unit 150. For example, the display control unit 122 performs control to display a display screen generated by the virtual space processing unit 121 on the display unit 150.

[0134] (Operation input unit 130)

[0135] The operation input unit 130 receives an operation instruction made by the user and outputs the content of the operation to the control unit 120. For example, the operation input unit 130 may be a touch sensor, a pressure sensor, or a proximity sensor. In addition, the operation input unit 130 may have a physical configuration such as buttons, switches, and joysticks. The user can use the operation input unit 130 to operate the user avatar in the virtual space.

[0136] (Sensor 140)

[0137] The sensor 140 has a function of detecting (acquiring) various types of information about the user or the user's surroundings. Figure 6 The sensor 140 shown in may include several sensors. For example, the sensor 140 may be a microphone that collects sound. In addition, the sensor 140 may be an imaging device that images the user or the user's surroundings. In addition, the sensor 140 may be a position information measurement unit that measures the position (absolute position or relative position) of the user. In addition, the sensor 140 may be various sensors (imaging devices, acceleration sensors, angular velocity sensors, geomagnetic sensors, infrared sensors, depth sensors, and biometric sensors) that detect the user's facial expression, emotion, line of sight direction, posture, limb movement, head direction, biometric information, etc. In addition, the sensor 140 may include a sensor capable of detecting a total of nine axes including a three-axis gyro sensor, a three-axis acceleration sensor, and a three-axis geomagnetic sensor.

[0138] Note that the information detected by the sensor 140 can be used as information about the user operation in the virtual space processing unit 121. For example, the virtual space processing unit 121 may control the user avatar in the virtual space according to the information detected by the sensor 140.

[0139] (Display unit 150)

[0140] The display unit 150 has a function of displaying an image under the control of the display control unit 122. For example, the display unit 150 may be a display panel such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display.

[0141] In addition, the display unit 150 may be implemented by an HMD that covers the entire field of view of the user. In this case, the display unit 150 displays a left-eye image and a right-eye image on a left screen and a right screen respectively fixed to the user's left eye and right eye. The screen of the display unit 150 includes, for example, a display panel such as a liquid crystal display (LCD) or an organic EL display, or a laser scanning display such as a retinal direct drawing display. In addition, the display unit 150 may include an imaging optical system that magnifies and projects the display screen to form a magnified virtual image with a predetermined viewing angle on the user's pupil.

[0142] (Audio output unit 160)

[0143] The audio output unit 160 outputs audio information under the control of the control unit 120. For example, when the client terminal 10 is an HMD, the audio output unit 160 may be configured as a head-mounted earphone worn on the user's head, or may be implemented by earphones or bone conduction speakers.

[0144] (Storage unit 170)

[0145] The storage unit 170 is implemented by a ROM that stores programs, operation parameters, etc. for the processing of the control unit 120, and a RAM that temporarily stores appropriately changed parameters, etc. For example, the storage unit 170 stores a client application that executes various processes related to the virtual space. In addition, the storage unit 170 may store user information such as user profiles and user avatar information, and virtual space information received from the virtual space management server 200.

[0146] Although the configuration of the client terminal 10 has been specifically described above, the configuration of the mobile terminal 10 according to the present disclosure is not limited to Figure 6 the example shown. For example, the client terminal 10 may be implemented by multiple devices. Each configuration included in the client terminal 10 is not limited to being integrally provided in one housing, and they may be communicably connected by wire or wirelessly.

[0147] In addition, the client terminal 10 may be implemented by a system configuration that includes an output device (corresponding at least to the display unit 150 and the audio output unit 160) implemented by an HMD or the like, and a processing device (corresponding at least to the control unit 120) implemented by a smart phone, a tablet terminal, a PC, or the like.

[0148] In addition, the client terminal 10 may be a non-wearable device such as a smart phone, a tablet terminal, or a PC.

[0149] In addition, the client terminal 10 can also use information received from external devices such as a controller held by the user, a sensor worn by the user, or a sensor set around (environmentally) the user as information about user operations.

[0150] In addition, the client terminal 10 can also be communicably connected to an external display device such as a projector, a television device, or a monitor, and display a video of the virtual space through the display control unit 122.

[0151] <<3. Connection to the Virtual Space>>

[0152] Next, the connection of the client terminal 10 to the virtual space will be described with reference to Figure 7 Here, it is assumed that user registration has been completed in advance.

[0153] Figure 7 is a sequence diagram showing an example of the flow of connection processing to the virtual space according to an embodiment of the present disclosure. As Figure 7 shown, first, the client terminal 10 activates the client application (step S103). The operation processing of the client terminal 10 described below is executed by the client application. Note that the functions of the client application include the functions of the above-described virtual space processing unit 121.

[0154] Next, the client terminal 10 requests the user information management server 301 to perform user authentication (step S106). Specifically, the client terminal 10 sends the user ID, authentication information, etc. to the user information management server 301.

[0155] Next, the user information management server 301 performs user authentication in response to the request from the client terminal 10 (step S109). In the case where user authentication is successful, the operation proceeds to the next process. In the case where user authentication fails, error processing is executed.

[0156] Next, the user information management server 301 sends the user information stored in the data memory to the client terminal 10 (step S112). The user information includes the user's profile, information about the user avatar (appearance information, status information), information about the items owned by the user, etc.

[0157] Next, the client terminal 10 sets the received user information in the client application (step S115). Thus, regardless of which client terminal 10 the user uses, the user environment can be obtained. For example, regardless of whether the user accesses from a smart phone or a home PC, the latest user environment can be reflected in the used terminal.

[0158] Next, the client terminal 10 queries the matching server 303 for the connection destination server (step S118).

[0159] Next, the matching server 303 performs matching processing between the client terminal 10 and the connection destination server (virtual space management server 200) (step S121). Specifically, the matching server 303 determines the virtual space management server 200 with free space as the connection destination server based on the current number of terminals connected to each virtual space management server 200. In addition, the matching server 303 can also determine the virtual space management server 200 that the client terminal 10 previously connected to corresponding to the user ID as the connection destination server based on the user ID.

[0160] Next, the matching server 303 sends data (e.g., Internet Protocol (IP) address) indicating the connection destination server to the client terminal 10 (step S124).

[0161] Subsequently, the client terminal 10 sends a connection request to the matched virtual space management server 200 (step S127), establishes communication with the virtual space management server 200, and starts two-way communication (step S130).

[0162] By establishing communication with the virtual space management server 200, the client terminal 10 can receive virtual space information (map information, virtual objects, information about other user avatars, etc.) from the virtual space management server 200 and reproduce the virtual space locally. The reproduction of the virtual space can be performed by setting virtual objects based on the map information. In addition, the client terminal 10 also sends user information, user operation information, etc. to the virtual space management server 200. Establishing communication with the virtual space management server 200 can be logging in to the virtual space. The client terminal 10 can store the data of the connection destination server, and for example, can log in to the same virtual space again even when temporarily logging out from the virtual space.

[0163] <<4. Display Screen Example>>

[0164] Next, the display screen displayed on the client terminal 10 logged in to the virtual space will be described. When logging in to the virtual space, the user avatar is set at a predetermined position (e.g., Figure 10 the starting area 410 shown in) in the virtual space. The virtual space processing unit 121 of the client terminal 10 generates a video including the user avatar in the perspective as the video of the user's perspective and displays the video on the display unit 150.

[0165] In addition, the virtual space processing unit 121 may include, in the display screen, a setting screen for the user avatar, various operation buttons for operating the user avatar, a display button for text chat being executed nearby, a follow and follower display button, a notification display button for the user, a map display button, an owned item display button, and the like.

[0166] Figure 8 FIG. is an example of a display screen according to an embodiment of the present disclosure. As Figure 8 shown, a video 511 of the virtual space and an operation screen 512 are displayed on the display screen 510. On the operation screen 512, an image of the user avatar, buttons for editing the user avatar and the user profile, a region-specific menu display button 513, and a region-general menu button 514 are displayed.

[0167] The region-specific menu display button 513 is a button for displaying a menu specific to the region (virtual space) where the user avatar currently exists. In an embodiment of the present disclosure, a region dedicated to content is assumed, and an experience menu related to the corresponding content is assumed to be the region-specific menu. Note that the content here may be intellectual property (IP) content. In an embodiment of the present disclosure, a region of a specific football team is assumed as an example of such a region. Here, interactions between fans of the specific football team are performed. At various locations in this region, a logo of the football team, a merchandise store, a virtual stadium reproducing the stadium existing in the real space, and the like are set. The user can also move to a region dedicated to another content. At this time, the client terminal 10 may newly obtain data of a connection destination server (which provides a region (virtual space) dedicated to another IP content) from the matching server 303. In addition, one virtual space management server 200 may provide multiple different IP content regions. In this case, the client terminal 10 can move regions without switching the connection destination server.

[0168] The region-general menu button 514 is a menu button that can be used for any IP content region. For example, the avatar matching button is a button for matching with another user avatar in the region. Another user avatar with similar user characteristics may be presented as a recommendation for an interaction partner. In addition, setting buttons for displaying expressions and stamps, where the expression is an action performed by the avatar to further enrich the emotional expression of the avatar, and the stamp is a small image temporarily displayed above the avatar's head to express the avatar's feeling.

[0169] Figure 9 FIG. is an example of a display screen according to an embodiment of the present disclosure. As Figure 9 shown, the display screen 520 displays a video of the virtual space and various operation buttons. In Figure 9In the example shown, the user avatar U1 is in front of a stage provided with a large display 529. For example, a real-space relay video is displayed on the large display 529. The content distribution server 306 acquires the relay video from an external server (relay video distribution server) and distributes the relay video to each client terminal 10 in real time.

[0170] As an example of operation buttons, a controller 521 for controlling the movement of the user avatar in the virtual space (area) is displayed at the left end of the display screen 520, and a jump icon 522 is displayed at the right end of the display screen 520.

[0171] When the user slides the cursor displayed at the center of the controller 521 up, down, left, and right, the virtual space processing unit 121 makes the user avatar walk forward, backward, left, and right according to the sliding direction. Note that when the sliding amount or sliding speed is equal to or greater than the set value, the virtual space processing unit 121 makes the user avatar run forward, backward, left, and right. When the jump icon 522 is clicked, the user avatar jumps. Such user operation information regarding movements such as the movement or jump of the user avatar is sent to the client terminal 10 of each of the other users in real time via the virtual space management server 200 and is reflected on the user avatars set in the virtual space reproduced in the client terminal 10 of each of the other users. Thus, the movement of the user avatar is shared with other users.

[0172] In addition, the user can move the user's line of sight in the virtual space by dragging any location (locations other than the locations of the operation buttons) up, down, left, and right on the display screen 520 with one finger or the like. In addition, the user can also reduce or enlarge the video in the virtual space by pinching or stretching any location (locations other than the locations of the operation buttons) on the display screen 520 with two fingers.

[0173] The microphone icon 523 is a button for switching the input on (ON) and off (OFF) of the user's audio according to a click operation. When the microphone icon 523 is in the input on state, the user's audio is sent to another user via the virtual space management server 200 or the voice chat server 305. Voice chat can be performed with another user avatar located around the user avatar, can be performed with a specific other user avatar permitted by the user, or can be performed among the participating users in a voice chat group.

[0174] The text input icon 524 is a button for accepting the input of text chat. When the text input icon 524 is clicked, a text input UI (user interface) is displayed, and the user inputs text into the text input UI. For example, as Figure 9As shown, the input text is displayed as a balloon image T1 above the head of user avatar U1. A balloon image T2 is also displayed above the head of another user avatar U2 near user avatar U1. As Figure 9 shown, 3D representation can be used to display the balloon images of other user avatars U2 and U3 located on both sides of user avatar U1. In addition, the virtual space processing unit 121 can also hide the balloon images of other user avatars U4 and U5 located at positions far from user avatar U1 so that they cannot be visually recognized by the user. Note that text chat is not limited to the method via the balloon image displayed above the head of the user avatar. The virtual space processing unit 121 can display the surrounding chat window in response to a click operation on the surrounding chat window display switch button 527. The surrounding chat window is a screen that displays text chat between user avatars around the user avatar (e.g., in the part where the user avatar is located). Therefore, the user can perform text chat with another nearby user. Note that text chat can be executed via the virtual space management server 200 or the text chat server 304.

[0175] The stamp icon 525 is a button for displaying the stamp menu screen. The user can select any stamp from the displayed stamp menu screen by clicking the stamp icon 525 and display the stamp above the head of user avatar U1. The emoticon icon 526 is a button for displaying the emoticon menu screen. The user can select any emoticon from the displayed emoticon menu screen by clicking the emoticon icon 526 and can operate the facial expression, posture, gesture, etc. of user avatar U1.

[0176] Although various operation buttons have been described above with reference to Figure 9 the arrangement and functions of the various operation buttons are only examples, and the present disclosure is not limited thereto.

[0177] <<5. Specific Examples of Virtual Space>>

[0178] <5-1. IP Content Area>

[0179] Examples of the virtual space according to an embodiment of the present disclosure include areas dedicated to IP content. For example, this area can be an area conceptually based on the interaction among fans of a specific football team. In this case, for example, football team logos, merchandise stores, virtual stadiums that reproduce the stadiums existing in the real space, etc. are set at different locations in this area. Figure 10 is an example of a simple map showing an IP content specific area according to an embodiment of the present disclosure. As Figure 10As shown, the area 400 includes a starting area 410 where the user avatar is first placed when entering the area, a merchandise store 420, a stadium 430, and a bar 440. The area 400 according to an embodiment of the present disclosure may be a mirror world in which the corresponding virtual facilities (merchandise store 420, stadium 430, etc.) to be set are similar to the facilities in the real space. The user can enjoy the atmosphere around the actual stadium by moving the user avatar in the area 400.

[0180] In addition, in the stadium 430, as an example of content distribution in the virtual space, a football team's game is replayed. The game video can be displayed on a large display (streaming distribution of 2D moving images) set in the stadium 430, or the football players and the ball can be reproduced by 3DCG, and each football player and the ball can be moved based on the motion data in the actual game to reproduce the game.

[0181] Figure 11 It is a diagram for describing a mechanism for game reproduction using the skeletal data of players according to an embodiment of the present disclosure. As Figure 11 shown, first, the virtual space management server 200a sends specification information about the distributed content to each client terminal 10. The distributed content specification information is information indicating the game content reproduced in the stadium 430.

[0182] Each client terminal 10 requests the content distribution server 306 of the backend system 30 to distribute the game content based on the specification information received from the virtual space management server 200a.

[0183] Next, the content distribution server 306 requests the game data server 50, which is an external device, to distribute the game content. The game data server 50 is a device that stores the tracking data of each player and the ball collected by the tracking system 51 in the game executed in the real space. The tracking data of each player is, for example, skeletal data. The game data server 50 can store the 3DCG of each player. After adjusting the format of the tracking data obtained from the tracking system 51 and appropriately deleting unnecessary data, the game data server 50 generates game content data. In response to the request from the content distribution server 306, the game data server 50 sends the game content data to the content distribution server 306.

[0184] Next, the content distribution server 306 sends the game content data to each client terminal 10. The game content data includes the 3DCG of each player (also referred to as a player object), the skeletal data of each player in the game, the 3DCG of the ball, the tracking data of the ball in the game, and the like. Based on the game content data, the client terminal 10 moves the player objects in the stadium 430 based on the skeletal data and can reproduce the game more realistically in 3D. The user can operate the user avatar to enter the stadium 430 and enjoy the game in the stadium 430. In addition, the user can move the user avatar onto the field (where a football uniform is worn) and watch the movement of each player from various nearby angles. The content distribution server 306 can also synchronize the timing of the game content data reproduced in each client terminal 10 (the playback time of the game reproduced in the stadium 430). This will be described later with reference to Figures 27 to 32 The UI when reproducing a game in a virtual space using the tracking data of a game executed in the real space is described.

[0185] <5-2. Conversation Table>

[0186] At any location in the virtual space, a communication point associated with a communication room for performing communication between users is set. The communication point can be set by the virtual space management server 200 or by the user.

[0187] In the communication room, communication can be performed by one or more different communication means. One or more different communication means are, for example, communication means using information such as audio, characters, or images. The communication room can be provided by the backend system 30 (e.g., the conversation table server 307). The user can enter the communication room (participate in the communication) by moving the user avatar to the communication point where a conversation table is displayed as a virtual object. When entering the communication room, the display screen changes to the screen of the communication room. In addition, the communication performed in the communication room can be executed by the corresponding communication server (e.g., the text chat server 304 and the voice chat server 305).

[0188] A virtual object used as a marker can be set at the communication point. In an embodiment of the present disclosure, a table is used as an example of the virtual object used as a marker. Such a table is also referred to as a conversation table in this specification.

[0189] The conversation table is set at any location (communication point setting location) in the virtual space by an input operation to the client terminal 10 operated by the user or the virtual space management server 200. As an example, it can be set in the bar 440 described above with reference to Figure 10 described. Figure 12 FIG. is a display screen showing the inside of a bar where a conversation table 600 according to an embodiment of the present disclosure is set.Figure 12 The display screen 530 shown is an example of a video of the user's line of sight when the user avatar moves inside the bar 440. As described above with reference to Figure 10 the bar 440 is set as a place where users (here, fans of a football team) interact with each other. As shown in the display screen 530 of Figure 12 multiple conversation tables 600 are provided inside the bar 440. A conversation room is associated with each of the conversation tables 600. The conversation room is provided by the conversation table server 307.

[0190] The shape of the conversation table 600 is an example. When the user brings the user avatar close to the conversation table 600, information about the choice of entering the conversation room associated with the conversation table 600 is sent from the virtual space management server 200 to the client terminal 10. The user can make the user avatar enter (in other words, participate in the conversation table) the conversation room (communication room) by performing an operation to request entering the room on the client terminal 10.

[0191] The conversation table 600 is displayed at a communication point associated with a communication room where communication between users is performed. The communication point is set in a specific area on the virtual space, and when an avatar is included in the communication point, information about the room entrance choice is sent.

[0192] Figure 13 is a diagram for explaining an example of the shape of the conversation table 600. As shown in Figure 13 for example, the conversation table 600 includes a table 601 and a display 602. The display 602 is an example of a display object that displays "information about an event that occurs in the real space or virtual space", and the event is the subject of the conversation in the conversation room (communication room) associated with the conversation table 600. The subject of the conversation in the conversation room can be preset in the virtual space management server 200, or can be the current topic in the conversation room or the currently viewed video in the conversation room (for example, Figure 20 the topic window 542 shown). The display content of the display 602 is information about the subject of the conversation in the conversation room, and can be appropriately changed according to the change of the subject of the conversation in the conversation room. The display 602 and the table 601 are set as virtual objects in the virtual space.

[0193] For example, in the case where "a competition recently held in the real space" is set as the topic of conversation in the conversation room associated with the conversation table 600, information about the competition, such as a video of the competition, an interview video of the players participating in the competition, or the results of the competition, is displayed on the display 602. The 2D moving image displayed on the display 602 can be a moving image streamed from the content distribution server 306. The designation of the 2D moving image displayed on the display 602 can be performed by the virtual space management server 200. The virtual space management server 200 can specify the 2D moving image to be displayed on the display 602 according to the IP content associated with the area. The same 2D moving image can be played back at a plurality of conversation tables 600 provided inside the bar 440.

[0194] The conversation table 600 may further include a display 603 that displays information about the conversation table 600. For example, the display 603 displays the identification number of the conversation table 600 and the number of people currently entering the conversation room. By visually recognizing the display 603, the user can check how many people can enter the conversation room and how many people are currently in the conversation room.

[0195] (Method of entry)

[0196] By approaching the conversation table 600, the user can choose to participate (enter) as a speaker or as an audience. "Participation as a speaker" is participation in a state where speech (e.g., voice chat or text chat) can be carried out in the conversation room. "Participation as an audience" is participation in a state where speech is not allowed in the conversation room but viewing is allowed in the conversation room.

[0197] When the user avatar approaches the conversation table 600, the virtual space management server 200 sends information about the room entry selection to the client terminal 10 for the user to select whether to enter the conversation room associated with the conversation table 600. The information about the room entry selection indicates that the user can enter (participate) in the room in the form of a speaker or an audience. When the user operates the room entry (participation) request in the client terminal 10, the user can choose to participate in the form of a speaker or an audience.

[0198] Based on the information about room entry selection received from the virtual space management server 200, for example, when the user avatar moves within a certain distance range from the position 600 of the conversation table, the virtual space processing unit 121 displays a selection screen for allowing the user to select whether to enter the conversation room as a speaker or as an audience. Note that the participation / non-participation as an audience can be set when creating the conversation room. When the entry as a speaker or an audience is selected, an entry request is sent from the client terminal 10 to the virtual space management server 200, and each session information (connection information) corresponding to the conversation table 600 is returned from the virtual space management server 200. The client terminal 10 can establish a communication connection with the conversation table server 307, the text chat server 304, the voice chat server 305, etc. based on the received session information.

[0199] Note that each session information (connection information) corresponding to the conversation table 600 can be pre-included in the information about room entry selection. Therefore, in response to the operation of room entry selection performed by the user, the client terminal 10 issues a room entry request to the virtual space management server 200, and can establish a communication connection with the conversation table server 307, the text chat server 304, the voice chat server 305, etc. based on each partial information.

[0200] In addition, in the case where there are multiple conversation tables 600, the user may not know which one of the conversation tables 600 the user can participate in to perform the desired communication. Therefore, for example, the following method can be considered.

[0201] As an example, there is a method of changing the disclosure level of the state in the conversation room according to the distance between the conversation table 600 and the user avatar. Figure 14 It is a diagram for describing the case of disclosing the state in the conversation room according to the distance to the conversation table 600 according to an embodiment of the present disclosure.

[0202] As Figure 14 shown, when the virtual space management server 200 determines that the user avatar is located within the area E1 (second area) within the first distance from the position of the conversation table 600 (within the first distance centered on the communication point), the virtual space management server 200 sends information about the communication of the conversation table 600 to the client terminal 10. As an example of the information about the communication of the conversation table 600, there is audio information associated with the conversation table 600 in the communication room. Therefore, the user can grasp the conversation volume, atmosphere, etc. in the conversation room without entering the conversation room. In Figure 14In the example shown, the users of user avatars U4 and U5 are in a state of hearing the conversation in the conversation room. In addition, the virtual space management server 200 may instruct the client terminal 10 to increase the volume so that the conversation in the conversation room can be heard when the user avatar approaches the center (the conversation table 600).

[0203] On the other hand, as Figure 14 shown, when the virtual space management server 200 determines that the user avatar is in the area E2 (the first area) within a second distance shorter than the first distance from the conversation table 600, the virtual space management server 200 sends information about the choice of entering the conversation room associated with the conversation table 600 to the client terminal. The user can perform an operation of requesting to enter the room on the client terminal 10. At this time, the user can choose whether to participate (enter) in the conversation room in the form of a speaker or an audience (this can also be said to be the user input of the information about the room entry choice).

[0204] When the user enters the conversation room, the virtual space management server 200 may send the conversation in the conversation room (for example, voice chat information and text chat information), the video in the conversation room (for example, Figure 20 the state of the user avatar shown), the video shared (viewed) in the conversation room, or the agenda in the conversation room to the client terminal 10. The agenda can be extracted from the analysis of the voice chat or text chat in the conversation room. Such extraction of the agenda can be performed by the backend system 30. In addition, the category picked up in the conversation room to be described later in the topic window 542 (see Figure 20 ) can be extracted as the agenda. In addition, the conversation in the conversation room, the video in the conversation room (for example, Figure 20 the state of the user avatar shown), the video shared (viewed) in the conversation room, or the agenda in the conversation room can be sent to the client terminal 10 corresponding to the user avatar located in the area E1.

[0205] In Figure 14In the example shown, it is assumed that in the avatars located in area E2, user avatar U1 and user avatar U2 enter the conversation room as speakers, and user avatar U3 enters the conversation room as an audience. At this time, user avatars U4 and U5 located in area E1 can identify the topic of the conversation being held in the conversation room by visually recognizing the display content of the display 602. In addition, information about the communication in the conversation room (e.g., audio information in the conversation room) can be sent from the virtual space management server 200 to the client terminals 10 of each user who operates user avatars U4 and U5. Then, on the client terminal 10 of the user who operates the user avatar that newly enters area E2 or the user avatar closer to the conversation table 600, a selection screen for whether to enter the conversation room is presented.

[0206] As described above, in this system, the virtual space management server 200 arranges a virtual object (conversation table 600) in the virtual space, and this virtual object includes a display object (display 602) that displays "information about an event that occurs in the real space or virtual space" as the topic of the conversation in the conversation room (communication room). Therefore, the user who operates the user avatar existing around the virtual object can identify the topic of the conversation being held in the conversation room by visually recognizing the display content of the display object.

[0207] In addition, Figure 15 is a view of the display screen of the terminal (client terminal 10) used by the user who operates the user avatar when the position of the user avatar in the virtual space is included in area E2 (first area) associated with the communication point for communication between the executing users. The virtual space management server 200 sends information about the selection of entering the communication room to the terminal (client terminal 10) used by the user to display the selection screen SC on the display screen.

[0208] In addition, Figure 16 is a view of the display screen of the terminal (client terminal 10) used by the user who operates the user avatar when the position of the user avatar is included in area E1 (second area) located outside area E2 (first area). The virtual space management server 200 sends information about the communication to the terminal (client terminal 10) used by the user who operates the user avatar. For example, the audio information AI in the communication room associated with the conversation table 600 can be output by the client terminal 10.

[0209] Therefore, when operating an avatar in a virtual space, the user can determine the relationship with respect to the conversation room through avatar operations. The avatar operations include, for example, grasping the topic in the conversation room by visually recognizing a display object (display 602), checking the audio in the conversation room based on the distance between the user avatar operated by the user and a virtual object (conversation table 600), or participating in the conversation room by bringing the user avatar closer to the virtual object by a certain amount or more.

[0210] As another example, when the user avatar approaches the conversation table 600 within a certain distance, the virtual space processing unit 121 can change the disclosure level of the state in the conversation room according to the preference / preference information about the user. For example, when the conversation (agenda) in the conversation room matches the user's preference / preference, the virtual space processing unit 121 can play back the conversation in the conversation room in a loud voice, or display a more detailed agenda on the label display 605. On the other hand, when the conversation (agenda) in the conversation room does not match the user's preference / preference, the virtual space processing unit 121 can play back the conversation in the conversation room in a small voice, or may not play back the conversation, or may not display the agenda on the tab display 605.

[0211] In addition, as another example, the virtual space processing unit 121 can highlight the conversation table 600 of the agenda that matches the user's preference / preference. In addition, the virtual space processing unit 121 can highlight the conversation table 600 of the agenda that the user has mentioned so far in the conversation.

[0212] In addition, as another example, the virtual space processing unit 121 can highlight the conversation table 600 in which another user estimated to have preferences / preferences matching those of the user participates.

[0213] In addition, as another example, the virtual space processing unit 121 can present an effect under the table according to the excitement level in the conversation room. Figure 17 FIG. is an example showing the effect display of the conversation table 600 according to the excitement level in the conversation room according to an embodiment of the present disclosure. As Figure 17 shown, the virtual space processing unit 121 can present a writing effect under the table 601 included in the conversation table 600. The virtual space processing unit 121 can control the color, blinking, density, etc. of the effect according to the excitement level in the conversation room. Therefore, the user can grasp the excitement level in the conversation room.

[0214] In addition, the virtual space processing unit 121 can change the shape of the conversation table 600 according to the number of participants (the number of people entering the conversation room) of the conversation table 600. For example, as the number of participants increases, the table can change to a larger table.

[0215] In addition, in this system, there is no special restriction on entering the conversation room, and anyone can freely enter the conversation room. However, a function that allows a person who has entered the conversation room to optionally leave another participant can be provided.

[0216] In addition, in this system, a host of the conversation room may not be specifically set, and the user who first enters the room can be used as the host. In the case of creating a conversation room by a request from a user, the creator can be the host. The host can restrict entry into the conversation room, or the host can allow entry into the conversation room. The setting of the host can be executed by the conversation table server 307 that manages the conversation room. In addition, only the host can be given the permission to leave another participant. In addition, the authority of the host can be arbitrarily transferred to another participant.

[0217] In addition, the virtual space management server 200 can display a specific scene of a game that occurs in the real space or the virtual space on the display 602 as information about an event in the real space or the virtual space. The information about an event that occurs in the real space is not limited to a specific scene of a game, and only a specific scene of an event is desired, for example, a specific scene of a music live performance executed in the real space. In the case of an event that occurs in the virtual space, it can be a specific scene of an event such as a music live performance or an e-sports league executed in the virtual space.

[0218] For example, a specific scene of an event can be a highlighted video of a game in the real space. The virtual space management server 200 can execute control to display a highlighted video of a game that occurs in the real space on the display 602 (display object) as information about an event in the real space. As a highlighted video of a game, for example, a goal scene, a foul scene, etc. are assumed. In addition, a specific scene of an event can be a highlighted video of an event that occurs in the virtual space.

[0219] In addition, the virtual space management server 200 can use only the information about the team supported by a specific user group in an event as the "information about an event in the real space" to be displayed on the display 602 (display object). For example, it can be a highlighted video of a game played by a specific team. Therefore, before entering the room, the user can grasp which team communication is to be performed in the conversation room associated with the conversation table 600 including the display 602. For example, in this system, only the information about a specific team is displayed on the display 602 so that fans of the specific team can gather in the conversation room.

[0220] In addition, the virtual space management server 200 may send information regarding the room entry selection to the conversation room to only the users belonging to a specific user group among the users whose operating user avatars approach the conversation table 600. Therefore, in this system, the user group that can enter the conversation room can be restricted.

[0221] (Operation Processing)

[0222] Figure 18 is a sequence diagram showing an example of the operation processing from the generation to the entry of the conversation table 600 according to an embodiment of the present disclosure.

[0223] As Figure 18 shown, first, the conversation table server 307 generates a conversation room (step S206) in response to a request to generate a conversation room from the virtual space management server 200 (step S203). Then, the conversation table server 307 sends the session information of the generated conversation room to the virtual space management server 200 (step S209). The virtual space management server 200 associates the session information with the communication points set in the virtual space. Note that here, the generation request from the virtual space management server 200 is described as an example, but the present disclosure is not limited thereto, and the client terminal 10 may make a generation request. In this case, the conversation table server 307 sends the session information of the generated conversation room to the client terminal 10. The client terminal 10 associates the session information with the communication points set in the virtual space. In addition, the client terminal 10 sends the communication points and session information set in the virtual space to the virtual space management server 200 and shares the communication points and session information with other users.

[0224] Next, the virtual space management server 200 sends a text chat room generation request to the text chat server 304 (step S212). The text chat server 304 generates a text chat room (step S215) and sends the session information of the text chat room to the virtual space management server 200 (step S218).

[0225] Next, the virtual space management server 200 sends a voice chat room generation request to the voice chat server 305 (step S221). The voice chat server 305 generates a voice chat room (step S224) and sends the session information of the voice chat room to the virtual space management server 200 (step S227).

[0226] Next, the virtual space management server 200 stores each session information (step S230).

[0227] Next, when the user avatar moves closer to the conversation table 600 (within a certain distance from the conversation table 600) and the user selects to enter the conversation room (and the form of entry) through user operation (when performing user input of information regarding the entry selection), the client terminal 10 sends an entry request to the virtual space management server 200 (step S233). Note that when the user avatar moves closer to the conversation table 600, information regarding the selection of entering the communication room may be sent from the virtual space management server 200 to the client terminal 10. The client terminal 10 may display an entry selection screen based on the information regarding the room entry selection.

[0228] Next, the virtual space management server 200 identifies the user's intention to enter the room by receiving the room entry request from the client terminal 10 (step S236), and sends each session information (connection information) corresponding to the conversation table 600 to the client terminal 10 (step S239). In addition, the virtual space management server 200 may cause the client terminal 10 to transition to the conversation room screen (an example of the communication room screen) when the user enters the conversation room.

[0229] Note that here, as an example, it is described that an entry request is sent to the virtual space management server 200 in the case where the user selects to enter, but the present disclosure is not limited thereto. As described above, in the case where the client terminal 10 performs control based on the distance between the user avatar and the conversation table 600 to disclose the state in the conversation room, the client terminal may request the virtual space management server 200 to send each session information before obtaining the user's intention to enter the room. In addition, since the client terminal 10 continuously sends the user's position (the position of the user avatar) to the virtual space management server 200, in the case where the user avatar is positioned within a certain distance from the conversation table 600 (communication point), the virtual space management server 200 may recognize the intention to enter the room and send each session information to the client terminal 10. Alternatively, the virtual space management server 200 may include each session information corresponding to the conversation table 600 in the information regarding the selection to enter to be sent to the client terminal 10. After receiving each session information from the virtual space management server 200, the client terminal 10 may enable the user to select whether to enter the room. Therefore, before obtaining the user's intention to enter the conversation room, the client terminal 10 may perform control based on the distance between the conversation table 600 and the user avatar to disclose the state in the conversation room.

[0230] Then, the client terminal 10 displays the conversation room screen and starts communication connections with each server based on each session information (step S242). Next, the details of the communication connections with each server will be described with reference to Figure 19 Describe the details of the communication connections with each server.

[0231] Figure 19 FIG. is a diagram showing details of a communication connection between the client terminal 10 and each server when entering a conversation room according to an embodiment of the present disclosure. As Figure 19 shown, each client terminal 10 that has entered the conversation room performs two-way communication with the conversation table server 307 to obtain room entry user information. The room entry user information includes the user ID, user name, information about the user avatar, etc. of the user who entered the room.

[0232] Each client terminal 10 performs two-way communication with the text chat server 304 to send and receive text chats. This allows text chats to be performed in the conversation room.

[0233] Each client terminal 10 performs two-way communication with the voice chat server 305 to send and receive voice chats. Therefore, voice chats can be performed in the conversation room.

[0234] Each client terminal 10 performs two-way communication with the virtual space management server 200a to send and receive expressions, stamps, etc. in real time. This controls the expressions and stamps of each user avatar displayed in the conversation room in real time.

[0235] Each client terminal 10 is communicably connected to the content distribution server 306 and receives a streaming distribution of content (such as a moving image). Therefore, content (such as a moving image) can be viewed in the conversation room. Note that content specification information can be received from the conversation table server 307 or the virtual space management server 200a. The content can be set by the conversation table server 307 or the virtual space management server 200a, or can be set by the user who enters the room.

[0236] (Example of a conversation room screen)

[0237] Figure 20 FIG. is a diagram showing an example of a conversation room screen according to an embodiment of the present disclosure. When the user participates (i.e., enters the conversation room) in the conversation table, the virtual space processing unit 121 of the client terminal 10 causes the screen to change from Figure 14 the display screen shown to Figure 20 the conversation room screen 540 shown.

[0238] On the conversation room screen 540, the user avatars of the users who entered the room are displayed side by side. In addition, various operation buttons (stamp icon 546, expression icon 547, stamp icon 548, text input button 549, microphone on-off button 550, menu button 541) and a topic window 542 are displayed.

[0239] Above the head of each user avatar, a balloon image 544 and a stamp image 545 selected by the user can be displayed. The balloon image 544 displays text input. The virtual space processing unit 121 can display the balloon image 544 and the stamp image 545 for only a few seconds so that the log is not viewed. In addition, in each user avatar, the facial expression, posture, gesture, etc. (applause, cheer, greeting, gazing posture, frustrated gesture, etc.) are controlled according to the expression selected by the user.

[0240] The topic window 542 is a virtual object that displays the content viewed in the conversation room. There can be multiple topic windows 542. Each user can slide multiple topic windows 542a to 542c to select the content to pick up. Inside the conversation room, the picked-up topic window 542 (for example, the window at the center) starts playing back. Note that the picking-up right (channel right) can be given to all users entering the room.

[0241] Suppose the categories of the content displayed in the topic window 542 are, for example, news related to IP content (here, a specific football team) (the top 5 page views (PV) within the last 3 days, etc.), reaction videos among the motion images related to IP content (videos with a large number of views, videos with user excitement, etc.), highlight videos of the latest game, battle history, etc. In addition, in the topic window 542, private photos and videos can also be displayed for users entering the room, or the local screen of the user can be shared. In addition, the virtual space management server 200 or the conversation table server 307 can determine the category of the content displayed in the topic window 542 according to the location where the conversation table 600 is installed or the current time.

[0242] Figure 21 is a diagram showing another example of the conversation room screen according to an embodiment of the present disclosure. As in Figure 21 In the conversation room screen 560, only the picked-up topic window 561 can be displayed in a large size, and a text chat window 562 can be displayed. The display on-off switching of the text chat window 562 can be performed by a click operation on the display switching button 564. In addition, the on-off switching of the microphone in the conversation room can be performed by a click operation on the voice switching button 565.

[0243] Although examples of the conversation room screen have been described above, the conversation room screen according to the present disclosure is not limited to the examples shown in the drawings.

[0244] <5-3. Event Coordination>

[0245] Next, control of a performance (including a virtual event) in a virtual space will be described according to the development of an event occurring in the real space, where the performance is executed in cooperation with the event coordination server 308. A distribution video of the event occurring in the real space can be set in the virtual space. In addition, a user can view the distribution video of the event occurring in the real space outside the virtual space, such as on a television or in a window different from the window displaying the virtual space. Further, the performance in the virtual space is determined according to the development of the event occurring in the real space and the reactions of a plurality of users.

[0246] Figure 22 FIG. is a diagram showing a mechanism of event coordination according to an embodiment of the present disclosure. In Figure 22 the example shown, a soccer game being executed in real time in the real space is assumed to be a predetermined event.

[0247] First, the event coordination server 308 of the backend system 30 acquires real-time game data at an appropriate time and analyzes the game development. Further, the event coordination server 308 identifies whether the game will develop before the game, during the game, or after the game as the game development. For example, the event coordination server 308 can acquire, as game data, the text of the game development from a service that live-streams the game development as text, and automatically extract specific words. Examples of services that distribute the game development as text live include services that distribute the commentary (audio) of a commentator who relays the game as text. Examples of specific words to be automatically extracted include "free kick", "foul", "missed shot", "shot stopped", "corner kick", "opponent's free kick", "goal by *** (opponent team)", "start of the first half", "player substitution", "start of the second half", "yellow card", "offside", "goal by **** (own team)", and "end of the game". The text of the game development can be distributed independently on the official website of each team.

[0248] The analysis of the game development is not limited to the above example, and for example, the event coordination server 308 can perform image recognition on the relay video of the game in real time and determine the play.

[0249] Next, the virtual space management server 200a designates a corresponding scheduled performance based on the analysis result of the progress of the game obtained from the event coordination server 308 at an appropriate time and the reactions of multiple users, and instructs each client terminal 10 to immediately execute the performance in the virtual space. In an embodiment of the present disclosure, as an example, a performance according to the progress of the game executed in the real space can be executed in real time in the stadium 430. Therefore, in this system, users gathered in the stadium 430 existing in the virtual space can enjoy watching the game in the virtual space by experiencing the performance in the stadium 430 with other users according to the progress of the game and the reactions of multiple users. Note that the virtual space management server 200a can obtain the distribution video of the game occurring in the real space from the backend system 30 (for example, the content distribution server 306), send the distribution video to each client terminal 10, and instruct each terminal to arrange the distribution video in the virtual space (specifically, in the stadium 430). In addition, each client terminal 10 can obtain the distribution video of the game designated by the virtual space management server 200a from the content distribution server 306. The user can watch the performance in the stadium 430 from the user's viewpoint in the virtual space through the client application activated on the client terminal 10 such as a smart phone or a tablet terminal, while watching the live broadcast of the game on an external device such as a television device or a PC. The client terminal 10 can display the live broadcast of the game in one window displayed on the display unit 150, and can display the video of the virtual space (the state of the performance in the stadium 430) in another window displayed on the display unit 150.

[0250] In addition, the virtual space management server 200a obtains the reactions of users from each client terminal 10, estimates the progress of the game based on the content, and determines the corresponding performance. Examples of the reactions of users include user operation information (stamps, the number and type of executed expressions, etc.). In addition, the reactions of users can be information such as the number, content, frequency, and audio volume of voice chat transmissions. In addition, the reactions of users can be information such as the number, content, and frequency of text chat transmissions. The virtual space management server 200a can also estimate the progress of the game based on the transmission number of voice chat or text chat, etc. Note that the estimation of the progress of the game based on the reactions of users can be executed by the event coordination server 308.

[0251] Examples of the performances corresponding to the progress of the game include the following performances: in this performance, a DJ appears on the stage set up in the venue to excite the venue with music before the game, a video is displayed during practice or the team moves on the large display set up in the venue before the game, and a relay video showing the scene appearing in the game venue is displayed. In addition, a starting member announcement performance can be executed using the large display set up in the venue.

[0252] In addition, its examples may include a kick-off performance at the start of the game, as well as foul performances by players of one's own team or the opposing team during the game, missed scoring performances (performances that encourage players through the avatars of the team roles that appear), scoring performances (happy performances through the avatars of the team roles enjoying themselves while music is played, fireworks are set off, and confetti and balloons appear), a lottery event performance during halftime (currently abandoned), a profile display performance of star players (displayed on a large monitor set up on the field), yellow card performances by players of one's own team or the opposing team, and offside performances by players of one's own team or the opposing team. In addition, its examples may include an end-of-game performance at the end of the game and a victory performance (a performance where a DJ appears to energize the venue with music) or a win-over performance after the game.

[0253] In addition, when the user's reaction exceeds a certain number of times, the virtual space management server 200a may execute corresponding performances according to the progress of the game and the reactions of multiple users. For example, when many users (more than a certain number of users or more than a certain percentage of users) press the "GO" stamp, the virtual space management server 200a executes a performance for making non-player characters (NPCs) set in the field perform cheering expressions or a performance for playing back supporting music as a supporting performance. In addition, the virtual space management server 200a may set the number of times of the user's reaction as the excitement level and increase the size and number of balloons to be blown into the field according to the magnitude of the excitement level. In addition, the virtual space management server 200a may also increase the volume of the supporting music played in the field according to the user's excitement level. In addition, in the case where the user's excitement level continues, the virtual space management server 200a may extend the length of the supporting performance from a predetermined time. In addition, the virtual space management server 200a may determine the performance based on the reaction with the largest amount among the user's reactions.

[0254] The virtual space management server 200a may obtain the reaction of a user who operates the user avatar existing in the virtual space managed by the virtual space management server 200a as the user's reaction. The user's reaction is, for example, user operation information (stamps, the number and type of executed expressions, etc.). In addition, the user's reaction may be information such as the number, content, frequency, and audio volume of voice chat transmissions. In addition, the user's reaction may be information such as the number, content, and frequency of text chat transmissions.

[0255] In addition, the virtual space management server 200a can also obtain the reaction of a user who operates a user avatar existing in another virtual space managed by another virtual space management server as the reaction of the user. The other virtual space management server performs a performance based on the development of a game targeted at the same game as the game targeted by the virtual space management server 200a. Such a user reaction can also be user operation information, voice chat information, or text chat information. Note that in the case where the virtual space management server 200a sends a distribution video of a game to the client terminal 10a, the virtual space management server 200a can also obtain the reaction of a user who operates a user avatar existing in another virtual space managed by another virtual space management server that sends the same distribution video.

[0256] In addition, the reaction of the user is not limited to the reaction of the user who operates the user avatar, and can be the reaction of the audience watching the game live in the real space, or the reaction of the viewer watching the game on TV or the like. For example, such a reaction can be a facial expression, a movement, a cheer, etc. In addition, such a reaction is sensed by various sensors such as a camera device, an acceleration sensor, and a microphone. The virtual space management server 200a can appropriately obtain information about the reaction of the audience or the viewer in real time.

[0257] In addition, the virtual space management server 200a can determine the expression of the avatar existing in the virtual space as the performance to be determined based on the development of the game and the reactions of multiple users. For example, when the virtual space management server 200a recognizes the ongoing game as the game development and also recognizes that many users are performing supports such as selecting a "GO" stamp to encourage the game, an expression in which multiple avatars existing in the virtual space wave towels to support the team, etc. is reflected. At this time, the virtual space management server 200a can reflect this expression on the avatars in the non-operating state in the virtual space, can reflect this expression on the NPCs, or can reflect this expression on all avatars. In addition, the virtual space management server 200a can reflect the expression on all avatars and all NPCs in the non-operating state, or can reflect this expression on some of the avatars in the non-operating state and some of the NPCs.

[0258] (Specific examples of event coordination)

[0259] The game development generally includes before the game, during the game (first half), halftime, during the game (second half), and after the game. Hereinafter, specific examples of each performance in the general process of game development will be described in sequence.

[0260] Before the game

[0261] Forty-five minutes before the game, each user avatar is notified of the start of the live stadium (a performance event executed according to the real-time development of the game in the real space of stadium 430. In an embodiment of the present disclosure, it is also referred to as the "live stadium"). The user avatars in this area move to stadium 430.

[0262] For example, a user enters stadium 430, joins another user avatar (another user registered as a friend), and waits for the start of the game while chatting with friends in the group chat.

[0263] The DJ appears on the stage set up in the venue of stadium 430 and performs a performance to inspire the venue with music. In addition, a large display can be set above the stage. On the large display, a training video of the team before the game or a video of the arrival status of the team bus is displayed. Figure 23 It is a diagram showing an example of a display screen in the live stadium according to an embodiment of the present disclosure. As Figure 23 shown, on the display screen 710, starting member information, a video of the team's status before the game, etc. are displayed on the large display 711 set in the venue (field) of the stadium in the virtual space.

[0264] In addition, in the venue, performances for announcing the starting members and performances for making the avatars of the starting member players appear on the stage can be executed. The users gathered in the venue become even more excited. Next, when the players enter the game venue in the real space, the relay of the players entering the game venue is displayed on the large display 711, and the excitement of the users gathered in the venue reaches its peak.

[0265] During the game

[0266] During the game, the large display 711 displays performances corresponding to the game situation, or the reactions of the users gathered in the venue, or performances that prompt the users to react. Figure 24 It is a diagram showing an example of a display screen in the live stadium according to an embodiment of the present disclosure. In Figure 24 the shown display screen 720, the display windows 721 and 722 of the large display 711 and the text chat window 723 are displayed on the front. Such display switching can be appropriately executed through user operations. The user can use the text chat window 723 to have a text chat with other users in the area. In addition, the user can also perform a voice chat with the members in the group generated by other users registered as friends. These text chats and voice chats can be completed using the backend system 30.

[0267] In the display window 721, an avatar video of a team character that appears on a stage in a venue set in a virtual space and motivates the venue is displayed. In addition, in the display window 722, a support stamp identical to the support stamp selected by most users in the venue is displayed, or a support stamp prompting the user to make a selection is displayed. In such a performance situation, the user can have a unified feeling and the venue can be motivated. The virtual space processing unit 121 can appropriately play back music or audio according to a performance instruction from the virtual space management server 200. Support stamps include, for example, NICE, BOO, GOAL, etc.

[0268] Figure 25 FIG. is an example of a display screen in a live stadium according to an embodiment of the present disclosure. Figure 25 The display screen 730 shown in shows an example of a performance of a missed score. For example, in the case where the opposing team scores a goal, a performance in which the team character assumes a dejected pose is executed. In addition, when the user clicks the support stamp button 450 of "OMG (Oh My God)", the support stamp of "OMG" is displayed above the head of the user avatar U. In the case where many users have selected the support stamp of "OMG", a performance in which the large display 731 displays the support stamp of "OMG!" is executed.

[0269] In addition, the user can make the user avatar U perform the same cheering expression as the team character. Figure 26 FIG. is an example of a display screen in a live stadium according to an embodiment of the present disclosure. As Figure 26 shown, when the support button 742 of SYNC (SNYC) shown on the display screen 740 is selected, the virtual space processing unit 121 makes the user avatar U perform the same cheering expression as the team character 741. Therefore, the users in the venue can perform the cheering expression together with the team character and other users.

[0270] For example, during halftime, a lottery event performance is executed. Figure 27 FIG. is an example of a display screen in a live stadium according to an embodiment of the present disclosure. As Figure 27 shown in the display screen 750 of, for example, when a large number of boxes fall from the sky and the user operates the user avatar U and picks up the boxes, a gift such as a signed ball or a uniform is hit by the lottery.

[0271] When the home team scores a goal during the game, a scoring performance is executed. For example, in the scoring performance, music is played, fireworks are set off, confetti and balloons are dropped, and the team character dances. Figure 28 FIG. is an example of a display screen in a live stadium according to an embodiment of the present disclosure. As Figure 28As shown in the display 760, on the venue, performances such as setting off fireworks, dropping confetti and balloons are carried out. In addition, on the large display 761, information about the player who scored a goal, information about the players of the opposing team, the score, support stamps, videos of team roles, etc. are displayed. In addition, a huge avatar 762 of the player who scored a goal can also perform on the stage. In this way, when the team scores a goal, a performance to inspire the venue is carried out.

[0272] Note that NPCs can be set in the audience seats and on the venue around the field, and the virtual space processing unit 121 can appropriately perform performance expressions. For example, when a teammate player is on the offensive, shooting, or taking a free kick, the virtual space processing unit 121 makes the NPC perform a cheering expression. In addition, when a teammate player scores a goal, the virtual space processing unit 121 makes the NPC perform a happy expression. In addition, when an opposing player fouls, the virtual space processing unit 121 makes the NPC perform a booing expression. In addition, when an opposing player scores a goal, the virtual space processing unit 121 makes the NPC perform a disappointed expression.

[0273] In addition, in order to generate a sense of excitement in the virtual space, the virtual space management server 200 can instruct each client terminal 10 to make the user avatar in the non-operating state automatically perform performances such as expressions or stamps according to the overall performance.

[0274] After the game

[0275] At the end of the game, an end performance is carried out. In the case where the home team wins the game, the venue can be further inspired by a winning performance (for example, the DJ appears again), or in the case where the team loses the game, a performance respecting the players can be carried out. In addition, an interview video of the manager and players after the game can also be displayed on the large display on the venue. In addition, the team roles can also appear on the stage to perform winning expressions or play the team's cheering song.

[0276] After that, the user can move to a location where fans can interact with each other, such as the bar 440 described Figure 10 and Figure 12 and use the conversation table 600 to talk about today's game. In the conversation table 600, today's game video can be shared.

[0277] The specific performances according to the development of the game are described above. Note that the performances according to the present disclosure are not limited to the above specific examples, and various performances can be considered. In addition, here, performing a show according to the development of the game is taken as an example, but the present disclosure is not limited thereto, and for example, a show can be performed according to the development of a music live concert, a lecture, a recital, etc. performed in the real space.

[0278] <5-4 Game Replay UI>

[0279] Next, a UI for replaying a game in a virtual space using tracking data of a game executed in the real space will be described. As described above with reference to Figure 11 the client terminal 10 sets each player object and the ball in the stadium 430 (see Figure 10 ) in the virtual space based on the game content data received from the content distribution server 306, moves each player object according to the skeletal data of each player tracked during the game, and moves the virtual ball based on the tracking data of the ball to replay the game.

[0280] (Regarding the camera device position)

[0281] When displaying the replayed game on the display unit 150, the client terminal 10 may display a video starting from any camera device position among a large number of camera device positions. The camera device position may be selected by the user or may be automatically selected by the client terminal 10.

[0282] Figure 29 is a diagram showing an example of a camera device position according to an embodiment of the present disclosure. Figure 29 For example, camera device position 810, camera device position 811, camera device position 812, and camera device position 813 are shown.

[0283] At the camera device position 810: basic, the virtual camera C is set on the extension line connecting the goal and the ball. The camera is as follows. At the camera device position 811: bird's-eye view, the virtual camera C is set at a position behind the ball in the sky where a wide view can be obtained. These camera device positions and camera orientations can be operated by the user.

[0284] Each of the above camera device positions is a position for viewing the players from above, but the present disclosure is not limited thereto, and the client terminal 10 may set the position (viewpoint) of a specific player as the camera device position and provide a subjective video of that player. For example, as shown in the lower part of Figure 29 at the camera device position 812: GK, the viewpoint of the goalkeeper is set as the camera device position, and the scene seen by the goalkeeper is provided. In addition, at the camera device position 813: shooter, the viewpoint of the player who shoots at the end of each scene (also referred to as a "chapter") is set as the camera device position, and the scene seen by that player is provided.

[0285] Figure 30 is a diagram showing the game replay UI in the case of the camera device position 810: basic as shown in Figure 29 As shown in Figure 30As shown, the game replay UI 820 displays the video acquired by the virtual camera device C set on the extension line connecting the goal and the ball. Note that this will be described later. Figure 30 An overview of the screen of the game replay UI 820 shown.

[0286] Figure 31 Shows Figure 29 A diagram of the game replay UI in the case of an aerial view of the camera device position 811 shown. As Figure 31 As shown, in the game replay UI 830, the video acquired by the virtual camera device C set at a position with a wide view angle behind the ball in the sky is displayed. Note that Figure 31 The screen overview of the game replay UI 830 shown is common with Figure 30 the game replay UI 820 shown.

[0287] (Screen overview of the game replay UI) The screen overview of the game replay UI 820 will be described with reference to Figure 30 As shown, the game replay UI 820 displays, for example, an exit icon 821, a notification icon 822, an angle-changeable label 823, a camera device position switching icon 824, an avatar mode switching icon 825, a highlighting label 826, game progress information display 827, and a display switching icon 828. Note that the arrangement and shape of each display are not limited to Figure 30 the example shown. Figure 30 shown.

[0288] The exit icon 821 is an operation button for ending the game replay mode by controlling the display of player objects based on tracking data (specifically, skeletal data) of each player, etc., according to an embodiment of the present disclosure. The game replay mode according to an embodiment of the present disclosure may be a highlight replay, in which a replay is performed using the tracking data of one or more specific scenes in the tracking data of a game. Such a game replay mode of highlight replay performed in the stadium 430 in the virtual space is also referred to as a "highlight stadium" in this specification.

[0289] The notification icon 822 indicates the presence or absence of a notification to the user. For example, when a display indicating the number of notifications is shown on the notification icon 822 and the notification icon 822 is clicked, a notification list pops up.

[0290] The angle-changeable label 823 is a display indicating whether the user can arbitrarily operate the camera device angle. For example, in Figure 29 the camera device position 810: basic and the camera device position 810: basic shown, the camera device angle (i.e., the line-of-sight direction) is set to be operable by the user. On the other hand, in Figure 29In the case of the camera device position 812 shown: GK and the camera device position 813: shooter, the camera device angle is set to be inoperable by the user. In the case where it is operable by the user, the user can change the camera device angle in the upward, downward, leftward, and rightward directions by dragging or tapping any location on the screen with a finger or the like.

[0291] The camera device position switching icon 824 is an operation button for switching between a large number of camera device positions described Figure 29 For example, the user can switch the video of the game reproduction UI 820 to the video of the camera device position corresponding to the clicked icon by clicking any one of the basic icon, the bird's-eye view icon, the GK icon, and the shooter icon included in the camera device position switching icon 824.

[0292] The avatar mode switching icon 825 is an operation button for changing the viewing mode of the game reproduction to the avatar mode. The viewing modes according to the embodiments of the present disclosure include, for example, the view mode and the avatar mode. When the avatar mode switching icon 825 is not selected, the avatar mode switching icon is displayed by default in the viewing mode. As Figure 28 and Figure 29 shown, in the view mode, the user avatar is not displayed. In the avatar mode, the user avatar is displayed in the stadium, allowing the user to manipulate the user avatar to view the game from a free perspective. A display example of the avatar mode will be described later with reference to Figure 34 description.

[0293] The highlighting label 826 indicates that the function currently used by the user in the stadium 430 (see Figure 10 ) in the virtual space is to highlight the stadium (highlight the game reproduction mode of the reproduction). Note that in the stadium 430 in the virtual space according to the embodiments of the present disclosure, the function of the performance event "Live Stadium" executed according to the development of the real-time game in the real space described with reference to Figures 20 to 26 can also be used. In this case, for example, a label indicating "LIVE VIEW" is displayed on the screen.

[0294] The game progress information display 827 is a display that indicates the score, opponent, elapsed time, etc. of the game being reproduced.

[0295] The display switching icon 828 is an operation button for switching the display of the playback indicator. Each time the display switching icon 828 is clicked, the display of the playback indicator switches between on and off. The display example of the playback indicator will be described with reference to Figure 32 description.

[0296] Figure 32FIG. is a diagram showing a display example of a playback indicator according to an embodiment of the present disclosure. In Figure 32 In the game reproduction UI 830 shown in, a playback indicator 840 is displayed at the lower part of the screen in response to a click on the display switching icon 828. Note that the display position of the playback indicator 840 is not particularly limited. The playback indicator 840 includes a seek bar 841, a five-second return button 842, a previous chapter jump button 843, a playback and stop button 844, a slow playback button 845, a next chapter jump button 846, a five-second send button 847, and a chapter list display button 848.

[0297] The seek bar 841 indicates the playback position of the reproduced game. In addition, the game reproduced in the embodiment of the present disclosure includes an aggregation of one or more specific scenes (hereinafter also referred to as chapters). Therefore, as Figure 32 shown, the seek bar 841 may have breaks for each chapter.

[0298] When the chapter list display button 848 is clicked, a chapter list is displayed. Figure 33 FIG. is a diagram showing a display example of a chapter list according to an embodiment of the present disclosure. In Figure 33 In the game reproduction UI 850 shown, a chapter list 852 is displayed. The chapter list 852 includes a list of one or more specific scenes (chapters) in the game reproduced in the embodiment of the present disclosure. By selecting one of the chapters, the user can jump the playback time of the game video displayed on the game reproduction UI 850 to the playback start time of the selected chapter.

[0299] (Avatar mode)

[0300] Next, the avatar mode executed when the avatar mode switching icon 825 is clicked will be described with reference to Figure 34 FIG. is a diagram showing a display example of a game reproduction UI in the case of the avatar mode according to an embodiment of the present disclosure. Figure 34 In the avatar mode, as

[0301] shown, a user avatar U is displayed on the game reproduction UI 860. The user can operate the controller 521 or the jump icon 522 to move the user avatar U within the field. At this time, the position of the imaging device (i.e., the user's viewpoint) is the position including the user avatar U in the perspective view and follows the movement of the user avatar U. In addition, in the avatar mode, since the user can arbitrarily operate the imaging device angle, the display mode of the angle change label 823 changes as Figure 34 shown. Figure 34 FIG.

[0302] Note that collision detection between the user avatar U and virtual objects such as player objects and balls in the venue is not performed. In the avatar mode, the client terminal 10 may not be able to perform scenario controls such as pausing or fast-forwarding.

[0303] (Highlight replay)

[0304] Subsequently, a highlight replay of one or more specific scenarios (chapters) of a game played back by the client terminal 10 will be described with reference to Figure 35 The highlight replay of a game according to an embodiment of the present disclosure will be described. Figure 35 FIG. is a diagram for explaining the highlight replay of a game according to an embodiment of the present disclosure.

[0305] The client terminal 10 first obtains in-point and out-point information for each chapter in a game. For example, the in-point and out-point information is included in the game content data distributed from the content distribution server 306. The in-point and out-point information for each scenario can be automatically set in advance according to the game situation in the game data server 50.

[0306] Next, the client terminal 10 searches for the in-point of scenario 1 among the tracking data of players and balls included in the game content data, and starts playback (reproduction of the game by moving player objects). After performing playback until the out-point of scenario 1, it searches for the in-point of the next scenario (scenario 2) and starts playing back the next scenario. Then, the client terminal 10 repeats the above steps until all the scenarios set in a game ( Figure 35 scenarios 1 to 4 in the example shown) are played back.

[0307] <<6. Supplementary>>

[0308] Embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technology is not limited to such examples. Obviously, those with ordinary knowledge in the technical field of the present disclosure can design various variation examples or modification examples within the scope of the technical idea described in the claims, and it will be naturally understood that these variation examples or modification examples also belong to the technical scope of the present disclosure.

[0309] A computer program can also be created to cause hardware such as a CPU, ROM, and RAM built into the client terminal 10 or the virtual space management server 200 to exhibit the functions of the client terminal 10 or the virtual space management server 200. In addition, a computer-readable storage medium storing the computer program is provided.

[0310] In addition, the effects described in this specification are not restrictive. That is, in addition to or instead of the above effects, the technology according to an aspect of the present disclosure may also exhibit other effects that are obvious to those skilled in the art according to the description of this specification.

[0311] Note that this technology can be configured as follows.

[0312] (1) An information processing apparatus, comprising:

[0313] A circuit system configured to:

[0314] Based on an avatar operated by a user entering a first area that is a communication point for performing communication between users, provide the user with information about a choice of entering a communication room, where the communication room is related to the communication point and includes virtual objects, and the virtual objects include display objects that display information about an event occurring in a real space or a virtual space; and

[0315] Based on the avatar entering a second area located outside the first area, provide information about the communication,

[0316] where the event is a topic of conversation in the communication room.

[0317] (2) The information processing apparatus according to (1),

[0318] where the display object includes information about the event occurring in the real space or the virtual space.

[0319] (3) The information processing apparatus according to any one of (1) or (2), where the display object includes information about a team supported by a specific user group in the event.

[0320] (4) The information processing apparatus according to any one of (1) to (3),

[0321] where the display object includes a specific scene of the event.

[0322] (5) The information processing apparatus according to any one of (1) to (4),

[0323] where the specific scene of the event includes a highlighted video of the event.

[0324] (6) The information processing apparatus according to any one of (1) to (5),

[0325] where the circuit system is further configured to:

[0326] Receive a selection from the user of the information about the choice of entering the communication room to enter the communication room; and

[0327] Based on the user's selection to enter the communication room, provide the user with connection information corresponding to the communication room.

[0328] (7) The information processing apparatus according to any one of (1) to (6),

[0329] wherein the selection by the user regarding entering the communication room includes entering the communication room as a speaker and entering the communication room as an audience.

[0330] (8) The information processing apparatus according to any one of (1) to (7),

[0331] wherein the information regarding the communication includes audio information in the communication room, and

[0332] wherein the circuit system is further configured to increase the volume of the audio information based on a decrease in the distance between the avatar and the center of the virtual object.

[0333] (9) The information processing apparatus according to any one of (1) to (8),

[0334] wherein the circuit system is further configured to generate a communication room including the content viewed by the user in the communication room.

[0335] (10) The information processing apparatus according to any one of (1) to (9),

[0336] wherein the content includes news related to the team participating in the event.

[0337] (11) The information processing apparatus according to any one of (1) to (10),

[0338] wherein the content includes a video having a number of viewings greater than a predetermined number.

[0339] (12) The information processing apparatus according to any one of (1) to (11),

[0340] wherein the content includes a video having a highlight of the event.

[0341] (13) The information processing apparatus according to any one of (1) to (12),

[0342] wherein the content includes a chat text window.

[0343] (14) The information processing apparatus according to any one of (1) to (13),

[0344] wherein the content includes a picked topic window.

[0345] (15) The information processing apparatus according to any one of (1) to (14),

[0346] wherein the circuit system is further configured to perform two-way communication with the user based on the user entering the communication room.

[0347] (16) The information processing apparatus according to any one of (1) to (15),

[0348] wherein the circuit system is further configured to:

[0349] generate a communication room including an emoticon icon for displaying an emoticon menu screen; and

[0350] initiate display of an emoticon in the communication room based on a selection made by the user from the emoticon menu screen.

[0351] (17) The information processing apparatus according to any one of (1) to (16),

[0352] wherein the circuit system is further configured to:

[0353] generate a communication room including a stamp icon for displaying a stamp menu screen; and

[0354] initiate display of a stamp in the communication room based on a selection made by the user from the stamp menu screen.

[0355] (18) An information processing method, comprising:

[0356] providing information about a choice of entering a communication room to a user based on an avatar operated by the user entering a first area that is a communication point for performing communication between users, wherein the communication room is related to the communication point and includes virtual objects, and the virtual objects include display objects that display information about an event occurring in a real space or a virtual space; and

[0357] providing information about the communication based on the avatar entering a second area located outside the first area,

[0358] wherein the event is a topic of conversation in the communication room.

[0359] (19) An information processing apparatus, comprising:

[0360] a circuit system configured to:

[0361] In a communication room related to a communication point for performing communication between users, a virtual object is provided, the virtual object including a display object that displays information about an event occurring in a real space or a virtual space.

[0362] Wherein, based on an avatar operated by the user entering a first area that is the communication point for performing communication between users, information about a choice of entering the communication room is provided to the user.

[0363] Wherein, based on the avatar entering a second area located outside the first area, information about the communication is provided, and

[0364] Wherein the event is a topic of conversation in the communication room.

[0365] (20) An information processing method, comprising:

[0366] In a communication room related to a communication point for performing communication between users, a virtual object is provided, the virtual object including a display object that displays information about an event occurring in a real space or a virtual space.

[0367] Wherein, based on an avatar operated by the user entering a first area that is the communication point for performing communication between users, information about a choice of entering the communication room is provided to the user.

[0368] Wherein, based on the avatar entering a second area located outside the first area, information about the communication is provided, and

[0369] Wherein the event is a topic of conversation in the communication room.

[0370] (B-1)

[0371] An information processing system, comprising a control unit that performs the following operations:

[0372] Control to set a virtual object in a virtual space, the virtual object including a display object that displays information about an event occurring in a real space or a virtual space, the event being a topic of conversation in a communication room.

[0373] In a case where the position of an avatar operated by a user in the virtual space is included in a first area that is a communication point associated with the communication room for performing communication between users, control to send information about a choice of entering the communication room to a terminal used by the user, and

[0374] When the position of the avatar in the virtual space is in a second region outside the first region, control is performed to send information about the communication to the terminal.

[0375] (B-2)

[0376] The information processing system according to item (B-1), wherein the display object includes a display object that displays a specific scene of an event occurring in the real space.

[0377] (B-3)

[0378] The information processing system according to item (B-2), wherein the display object includes a display object that displays a highlighted video of a game being played in the real space.

[0379] (B-4)

[0380] The information processing system according to item (B-1), wherein the display object includes a display object that displays information about the team supported by a specific user group in the event.

[0381] (B-5)

[0382] The information processing system according to item (B-4), wherein

[0383] the display object includes a display object related to the team supported by a specific user group in the event, and

[0384] the information about the entered selection is sent to the terminal used by a user belonging to the specific user group.

[0385] (B-6)

[0386] The information processing system according to any one of items (B-1) to (B-5), wherein, when a request for the user to enter the communication room is received from the terminal based on user input of information about the entered selection, the control unit sends connection information for obtaining information by using one or more different communication means used in the communication room to the terminal used by the user.

[0387] (B-7)

[0388] The information processing device according to item (B-6), wherein the information obtained by the communication means includes information about audio, characters, or images sent from the terminal of each user in the communication room.

[0389] (B-8)

[0390] The information processing system according to any one of items (B-1) to (B-7), wherein the information regarding the selection of entering the communication room includes information for connecting to the server that generates the communication room.

[0391] (B-9)

[0392] The information processing system according to item (B-8), wherein the server performs control to send information regarding the users in the communication room to the terminals of each user.

[0393] (B-10)

[0394] The information processing system according to any one of items (B-1) to (B-9), wherein the control unit performs control to send information regarding the distributed content viewed in the communication room to the terminals of the users.

[0395] (B-11)

[0396] The information processing system according to item (B-10), wherein the information regarding the distributed content includes information specifying the distributed content and information for connecting to the content distribution server to which the distributed content is distributed.

[0397] (B-12)

[0398] The information processing system according to any one of items (B-1) to (B-11), wherein the control unit receives information regarding the display of the user avatar displayed on the communication room screen displayed on the terminal of the user, and sends the information to the terminals of other users in the communication room in real time.

[0399] (B-13)

[0400] The information processing system according to item (B-12), wherein the information regarding the display of the user avatar includes expression information or stamp information for moving the user avatar, and the stamp information is an image displayed near the user avatar and indicating an emotional expression.

[0401] (B-14)

[0402] The information processing system according to any one of items (B-1) to (B-13), in a case where the user avatar corresponding to the user moves within a predetermined distance from the communication point, the control unit recognizes the intention of entering the communication room, and performs control to send connection information for entering the communication room to the terminal of the user.

[0403] (B-15)

[0404] The information processing system according to any one of items (B-1) to (B-14), wherein when the user enters the communication room, the control unit performs control to instruct the user's terminal to transition to the communication room screen.

[0405] (B-16)

[0406] The information processing system according to item (B-15), wherein the communication room screen displays the user avatars of the users in the communication room.

[0407] (B-17)

[0408] The information processing system according to item (B-16), wherein characters or stamp images input by the user corresponding to the user avatar are displayed near the user avatar.

[0409] (B-18)

[0410] The information processing system according to any one of items (B-15) to (B-17), wherein the communication room screen displays content.

[0411] (B-19)

[0412] An information processing method, comprising:

[0413] The following operations are performed by a processor:

[0414] Setting a virtual object in a virtual space, the virtual object including a display object that displays information about an event occurring in a real space or a virtual space, the event being the subject of a conversation in a communication room,

[0415] When the position of the avatar operated by the user in the virtual space is included in a first area that is a communication point associated with the communication room for performing communication between users, sending information about the selection to enter the communication room to the terminal used by the user, and

[0416] When the position of the avatar in the virtual space is included in a second area outside the first area, sending information about the communication to the terminal.

[0417] (B-20)

[0418] A program for causing a computer to function as a control unit, the control unit performing the following operations:

[0419] Control to set a virtual object in a virtual space, the virtual object including a display object that displays information about an event occurring in a real space or a virtual space, the event being the subject of a conversation in a communication room.

[0420] When the position of an avatar operated by a user in the virtual space is included in a first area that is a communication point associated with the communication room for communication between performing users, control to send information about a selection to enter the communication room to a terminal used by the user, and

[0421] When the position of the avatar in the virtual space is included in a second area outside the first area, control to send information about the communication to the terminal.

[0422] (B-21)

[0423] An information processing system including a control unit, the control unit:

[0424] Set a distribution video of an event occurring in the real space in a virtual space where a user can view the distribution video of the event, and

[0425] Determine an event coordination performance experienced by the multiple users in the virtual space according to the development of the event occurring in the real space and the reactions of the multiple users.

[0426] (B-22)

[0427] The information processing system according to item (B-21), wherein the performance is determined according to the amount of reaction of the multiple users.

[0428] (B-23)

[0429] The information processing system according to any one of items (B-21) to (B-22), wherein the performance is determined based on the reaction having the largest amount of reaction among the reactions of the multiple users.

[0430] (B-24)

[0431] The information processing system according to item (B-21), wherein the performance is an expression of an avatar, and the expression is reflected in one or more avatars existing in the virtual space.

[0432] (B-25)

[0433] The information processing system according to item (B-24), wherein the expression of the avatar is reflected in avatars in a non-operating state existing in the virtual space.

[0434] (B-26)

[0435] The information processing system according to item (B-24), wherein the emotion of the avatar is reflected in the NPCs existing in the virtual space.

[0436] (B-27)

[0437] The information processing system according to any one of items (B-21) to (B-26), wherein the reactions of the plurality of users include at least one of the following: reaction information of a user operating an avatar existing in a virtual space, reaction information of a user operating avatars existing in a plurality of virtual spaces, or reaction information of a user experiencing an event in the real space.

[0438] List of reference numerals

[0439] 20 Virtual space system

[0440] 200 (200a, 200b, 200c,...) Virtual space management server

[0441] 210 Communication unit

[0442] 220 Control unit

[0443] 221 Virtual space management unit

[0444] 222 Content management unit

[0445] 223 User management unit

[0446] 230 Storage unit

[0447] 10 Client terminal

[0448] 110 Communication unit

[0449] 120 Control unit

[0450] 121 Virtual space processing unit

[0451] 122 Display control unit

[0452] 130 Operation input unit

[0453] 140 Sensor

[0454] 150 Display unit

[0455] 160 Audio output unit

[0456] 170 Storage unit

[0457] 30 Back-end system

[0458] 301 User Information Management Server

[0459] 302 Hosting Server

[0460] 303 Matching Server

[0461] 304 Text Chat Server

[0462] 305 Voice Chat Server

[0463] 306 Content Distribution Server

[0464] 307 Conversation Table Server

[0465] 308 Event Coordination Server

[0466] 309 Transaction Management Server

[0467] 310 Data Analysis Server

Claims

1. An information processing apparatus, comprising: a circuit system configured to: provide information about a selection to enter a communication room to the user based on an avatar operated by the user entering a first area that is a communication point for performing communication between users, wherein the communication room is related to the communication point and includes virtual objects, and the virtual objects include display objects that display information about an event occurring in the real space or the virtual space; and provide information about the communication based on the avatar entering a second area located outside the first area, wherein the event is a topic of conversation in the communication room.

2. The information processing apparatus according to claim 1, wherein the display object includes information about the event occurring in the real space or the virtual space.

3. The information processing apparatus according to claim 2, wherein the display object includes information about a team supported by a specific user group in the event.

4. The information processing apparatus according to claim 2, wherein the display object includes a specific scene of the event.

5. The information processing apparatus according to claim 4, wherein the specific scene of the event includes a highlighted video of the event.

6. The information processing apparatus according to claim 1, wherein the circuit system is further configured to: receive a selection of the information about the selection to enter the communication room from the user to enter the communication room; and provide connection information corresponding to the communication room to the user based on the selection of the user to enter the communication room.

7. The information processing apparatus according to claim 1, wherein the selection of the user about the selection to enter the communication room includes entering the communication room as a speaker and entering the communication room as an audience.

8. The information processing apparatus according to claim 1, wherein the information about the communication includes audio information in the communication room, and wherein the circuit system is further configured to increase the volume of the audio information based on a decrease in the distance between the avatar and the center of the virtual object.

9. The information processing apparatus according to claim 1, wherein the circuit system is further configured to generate a communication room including content viewed by the user in the communication room.

10. The information processing apparatus according to claim 9, wherein the content includes news related to the team participating in the event.

11. The information processing apparatus according to claim 9, wherein the content includes a video having a number of views greater than a predetermined number.

12. The information processing apparatus according to claim 9, wherein the content includes a highlighted video of the event.

13. The information processing apparatus according to claim 9, wherein the content includes a chat text window.

14. The information processing apparatus according to claim 9, wherein the content includes a picked topic window.

15. The information processing apparatus according to claim 1, wherein The circuit system is further configured to perform two-way communication with the user based on the user entering the communication room.

16. The information processing apparatus according to claim 1, wherein, the circuit system is further configured to: generate a communication room including an emoticon icon for displaying an emoticon menu screen; and initiate display of an emoticon in the communication room based on a selection made by the user from the emoticon menu screen.

17. The information processing apparatus according to claim 1, wherein, the circuit system is further configured to: generate a communication room including a stamp icon for displaying a stamp menu screen; and initiate display of a stamp in the communication room based on a selection made by the user from the stamp menu screen.

18. An information processing method, comprising: providing information about a selection of entering a communication room to the user based on an avatar operated by the user entering a first area that is a communication point for performing communication between users, wherein the communication room is related to the communication point and includes a virtual object, and the virtual object includes a display object that displays information about an event occurring in the real space or virtual space; and providing information about the communication based on the avatar entering a second area located outside the first area, wherein the event is a topic of conversation in the communication room.

19. An information processing apparatus, comprising: a circuit system configured to: provide a virtual object in a communication room related to a communication point for performing communication between users, the virtual object including a display object that displays information about an event occurring in the real space or virtual space, wherein information about a selection of entering a communication room is provided to the user based on an avatar operated by the user entering a first area that is the communication point for performing communication between users, wherein information about the communication is provided based on the avatar entering a second area located outside the first area, and wherein the event is a topic of conversation in the communication room.

20. An information processing method, comprising: providing a virtual object in a communication room related to a communication point for performing communication between users, the virtual object including a display object that displays information about an event occurring in the real space or virtual space, wherein information about a selection of entering a communication room is provided to the user based on an avatar operated by the user entering a first area that is the communication point for performing communication between users, wherein information about the communication is provided based on the avatar entering a second area located outside the first area, and wherein the event is a topic of conversation in the communication room.

Images