System, method, program, user terminal, and server for displaying object of user in virtual space

Abstract

Provided is a system capable of controlling the appearance in accordance with the relationship between users in a virtual space such as a meta universe. A system according to one embodiment of the present invention is used for displaying an object of a user in a virtual space, and is provided with: a distance score storage unit for storing a distance score indicating the relationship between one user and another user; and a control unit that determines a model of an object of the display target user on the basis of the distance score, which is stored in the distance score storage unit and represents the relationship between the display target user and the user of the user terminal. And controls to display the object of the display object user on the user terminal on the basis of the determined object model.

Description

Technical Field

[0001] The present invention relates to a system, method, program, user terminal and server for displaying a user's object in a virtual space. Background Art

[0002] In recent years, the Metaverse has attracted attention. The Metaverse is a virtual space on the Internet in which users can move around or communicate with other users using virtual images that are their own avatars. For example, Patent Document 1 discloses a system that can customize the clothing worn by virtual images in a virtual space.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application No. 2019-536131

[0006] Non-patent literature

[0007] Non-patent literature 1: Sihem Amer Yahia, Michael Benedikt, Laks VSLakshmanan, and Julia Stoyanovich. 2008. Efficient network aware search in collaborative tagging sites. Proc. VLDB Endow. 1, 1 (August 2008), 710-721. DOI: https: / / doi.org / 10.14778 / 1453856.1453934

[0008] Non-patent document 2: Norases Vesdapunt and Hector Garcia-Molina.2016.Updatingan Existing Social Graph Snapshot via a Limited API.In Proceedings of the25th ACM International on Conference on Information and Knowledge Management(CIKM'16).Association for Computing Machinery,New York,NY,USA,1693-1702.DOI:https: / / doi.org / 10.1145 / 2983323.2983703

[0009] Non - Patent Document 3: Han Hee Song, Tae Won Cho, Vacha Dave, Yin Zhang, and Lili Qiu. 2009. Scalable proximity estimation and link prediction in online social networks. In Proceedings of the 9th ACM SIGCOMM conference on Internet measurement (IMC'09). Association for Computing Machinery, New York, NY, USA, 322 - 335. DOI: https: / / doi.org / 10.1145 / 1644893.1644932 Summary of the Invention

[0010] Problems to be Solved by the Invention

[0011] The inventors of the present invention predict that in the future, when virtual spaces such as the metaverse function as a society, it is required to realize the control of the appearance corresponding to the social relationship between oneself and the other party. The present invention is completed to solve such problems, and the purpose is to provide a system or the like that can control the appearance corresponding to the relationship between users in a virtual space such as the metaverse.

[0012] Solutions to the Problems

[0013] 〔1〕The system according to an embodiment of the present invention is used to display an object of a user in a virtual space, and the system includes:

[0014] A distance score storage unit that stores a distance score indicating the relationship of one user with respect to other users; and

[0015] A control unit that determines a model of the object of the display target user based on the distance score indicating the relationship of the display target user with respect to the user of the user terminal stored in the distance score storage unit, and performs control to display the object of the display target user in the user terminal based on the determined model of the object.

[0016] 〔2〕In an embodiment of the present invention, in the system described in 〔1〕,

[0017] When the display object user is a user with a personal account, the object of the display object user is a character. The control unit determines a model of the character corresponding to a distance score indicating the relationship between the display object user and the user of the user terminal, and controls to display the character of the display object user based on the determined model of the character.

[0018] 〔3〕In one embodiment of the present invention, in the system described in 〔1〕 or 〔2〕,

[0019] When the display object user is a user with a non-personal account, the object of the display object user is an environment. The control unit determines a model of the environment corresponding to a distance score indicating the relationship between the display object user and the user of the user terminal, and controls to display the environment of the display object user based on the determined model of the environment.

[0020] 〔4〕In one embodiment of the present invention, in the system described in any one of 〔1〕 to 〔3〕,

[0021] The system includes an object model storage unit that stores models of objects at multiple stages corresponding to a distance score indicating the relationship between each user and other users.

[0022] The control unit selects one of the object models stored in the object model storage unit as the model of the object of the display object user based on the distance score indicating the relationship between the display object user and the user of the user terminal stored in the distance score storage unit, and controls to display the object of the display object user in the user terminal based on the selected object model.

[0023] 〔5〕In one embodiment of the present invention, in the system described in 〔4〕,

[0024] The object model storage unit stores models of characters at multiple stages corresponding to a distance score indicating the relationship between each user with a personal account and other users.

[0025] The model of the character corresponding to the smallest distance score is a model of a character for family members or a model of a character for relatives and friends.

[0026] 〔6〕In one embodiment of the present invention, in the system described in any one of 〔1〕 to 〔5〕,

[0027] The system includes a distance score calculation unit that calculates a distance score indicating the relationship between one user and other users.

[0028] The distance score calculation unit calculates a distance score representing the relationship of the one user with respect to the other user based on the weight set by the one user for the other user.

[0029] 〔7〕In one embodiment of the present invention, in the system described in 〔6〕,

[0030] The distance score calculation unit also calculates a distance score representing the relationship of the one user with respect to the other user based on the number of links between the nodes representing the respective users of the one user and the other user on the social graph.

[0031] 〔8〕The method of one embodiment of the present invention is for displaying an object of a user in a virtual space, and the method includes the following steps:

[0032] Determining a model of the object of the display target user based on a distance score representing the relationship of the display target user with respect to the user of the user terminal; and

[0033] Displaying the object of the display target user in the user terminal based on the determined model of the object.

[0034] 〔9〕In one embodiment of the present invention, in the method described in 〔8〕,

[0035] The method includes the following steps: calculating a distance score representing the relationship of the one user with respect to the other user,

[0036] In the determining step, determining a model of the object of the display target user based on the distance score representing the relationship of the display target user with respect to the user of the user terminal calculated in the calculating step.

[0037] 〔10〕The program of one embodiment of the present invention causes a computer to execute each step of the method described in 〔8〕 or 〔9〕.

[0038] 〔11〕The user terminal of one embodiment of the present invention is for displaying an object of a user in a virtual space, wherein

[0039] Storing a model of the object of the other user corresponding to a distance score representing the relationship of the other user with respect to the user of the user terminal,

[0040] Determining a model of the object of the display target user based on the score representing the relationship of the display target user with respect to the user of the user terminal stored in the distance score storage unit, and displaying the object of the display target user based on the determined model of the object.

[0041] 〔12〕A server according to an embodiment of the present invention is used to display an object of a user in a virtual space, wherein,

[0042] stores a distance score representing the relationship of one user to other users,

[0043] determines a model of the object of the display target user based on the distance score representing the relationship of the display target user to the user of the user terminal stored in the distance score storage unit, and controls to display the object of the display target user in the user terminal based on the determined model of the object.

[0044] Effects of the Invention

[0045] According to the present invention, it is possible to control the appearance corresponding to the relationship between users in a virtual space. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is an overall structure diagram of a system according to an embodiment of the present invention.

[0047] Figure 2 is a block diagram showing the hardware structure of a server according to an embodiment of the present invention.

[0048] Figure 3 is a block diagram showing the hardware structure of a user terminal according to an embodiment of the present invention.

[0049] Figure 4 is a functional block diagram of a system according to an embodiment of the present invention.

[0050] Figure 5 is a diagram showing an example of models of virtual avatars of a user stored in an object model storage unit at multiple stages.

[0051] Figure 6 is a diagram showing an example of a flowchart of the processing of the system of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] Next, embodiments of the present invention will be described with reference to the accompanying drawings. The system of the embodiment of the present invention can provide a user with a virtual space (virtual world) such as the metaverse. For example, system 1 can enable a user to act in a virtual space by means of an avatar that is the user's own clone. In the embodiment of the present invention, the metaverse refers to a virtual space on the Internet, services in that virtual space, and can also refer to metaverse services and metaverse platforms. In the embodiment of the present invention, the metaverse that means a virtual space on the Internet is simply expressed as a virtual space, but the embodiment of the present invention can also be applied to a virtual space having the same functions as the metaverse. The user's avatar, that is, the avatar representing the user, is an example of a role representing the user in the virtual space. In this specification, SNS (Social Network Service) can include services or websites in general that can build or assist in building a social network. In this specification, the application program is not limited to an application program installed in a mobile terminal such as a smartphone, and can also refer to application programs in general and programs in general. In this specification, expressions such as user A and user B represent any user, and are sometimes used when indicating that user A and user B are different users.

[0053] Figure 1 is the overall structure diagram of system 1 of an embodiment of the present invention. As Figure 1 shown, system 1 includes a server 10 and a plurality of user terminals 30. The server 10 and the user terminals 30 are connected to a network 2 such as the Internet and can communicate with each other. Regarding system 1 of the embodiment of the present invention, a known client-server type system is assumed for the description, but it is not limited thereto.

[0054] The server 10 is configured to include one or more devices and is a server or server system that can provide the metaverse to users. In the embodiment of the present invention, the server 10 has an SNS server (Web server, application server) that can provide a known social network service (SNS) in a virtual space (metaverse), or has the function of the SNS server. The server 10 can also be implemented by a virtual machine or a cloud system. The server 10 accepts access from the user terminal 30 and provides a metaverse corresponding to the user via the network 2. For example, when the user terminal 30 connects to the server 10 via a prescribed application program or a standardly installed web browser and completes identification and authentication using a user ID and password, etc., the user can utilize the metaverse as the user of that user ID. In one example, the server 10 has a server operated by a company that provides the metaverse (metaverse platform), or has the function of that server. In the following, for the sake of convenience of description, it is assumed that the server 10 is implemented by one device for the description.

[0055] Figure 2FIG. 0 is a block diagram showing the hardware configuration of a server 10 according to an embodiment of the present invention. The server 10 includes a processor 11, an input device 12, a display device 13, a storage device 14, and a communication device 15. These constituent devices are connected via a bus 16. In addition, an interface is inserted between the bus 16 and each constituent device as needed. The server 10 can include the same configuration as a general server, PC, or the like.

[0056] The processor 11 controls the overall operation of the server 10. For example, the processor 11 is a CPU. The processor 11 executes various processes by reading and executing programs and data stored in the storage device 14. The processor 11 may be composed of multiple processors.

[0057] The input device 12 is a user interface that accepts user input to the server 10, such as a touch panel, a touchpad, a keyboard, a mouse, or a button. The display device 13 is a display that displays an application program screen or the like for the user of the server 10 under the control of the processor 11.

[0058] The storage device 14 includes a main storage device and an auxiliary storage device. The main storage device is, for example, a semiconductor memory such as a RAM. The RAM is a volatile storage medium capable of high-speed reading and writing of information and is used as a storage area and a working area when the processor 11 processes information. The main storage device may also include a ROM as a read-only non-volatile storage medium. The auxiliary storage device stores various programs and data used by the processor 11 when executing each program. The auxiliary storage device may be any non-volatile storage device or non-volatile memory as long as it can store information, or it may be a detachable storage device.

[0059] The communication device 15 exchanges data with other computers such as user terminals or servers via a network, and is, for example, a wireless LAN module. The communication device 15 can be set as other wireless communication devices or modules such as a Bluetooth (registered trademark) module, or can be set as a wired communication device or module such as an Ethernet (registered trademark) module or a USB interface.

[0060] Figure 3 FIG. 16 is a block diagram showing the hardware configuration of a user terminal 30 according to an embodiment of the present invention. The user terminal 30 includes a processor 31, an input device 32, a display device 33, a storage device 34, and a communication device 35. These constituent devices are connected via a bus 36. In addition, an interface is inserted between the bus 36 and each constituent device as needed. In the embodiment of the present invention, the user terminal 30 can be set as a head-mounted display terminal (HMD terminal). In the following description of the hardware configuration, the user terminal 30 set as an HMD terminal will be described.

[0061] The processor 31 controls the overall operation of the user terminal 30. For example, the processor 31 includes a CPU, which executes various processes by reading and executing programs and data stored in the storage device 34. For example, the processor 31 includes a GPU for performing rendering processing, which generates a virtual space image for presenting to the user wearing the HMD. The processor 31 may also be composed of multiple processors.

[0062] The input device 32 is a user interface that accepts user input to the user terminal 30. In one example, the input device 32 is a controller independent of the main body worn by the user on the head. In this case, the main body and the controller are configured to be able to communicate. In one example, the input device 32 is a sensor such as a camera device that senses the movement of the user's hand or the like.

[0063] The display device 33 is a display that displays an application program screen or the like for the user under the control of the processor 31. The display device 33 is a non-penetrating display such as a liquid crystal display or an organic EL display, and displays a virtual space image for the user wearing the HMD. For example, the user terminal 30 can be set as a general fully immersive HMD or a video see-through HMD. For example, when the user terminal 30 is a video see-through HMD, the user terminal 30 is equipped with a camera device installed for photographing the user's visual field area when the user wears the user terminal 30.

[0064] The storage device 34 includes a main storage device and an auxiliary storage device. The main storage device is, for example, a semiconductor memory such as a RAM. The RAM is a volatile storage medium capable of high-speed reading and writing of information, and is used as a storage area and a working area when the processor 31 processes information. The main storage device may also include a ROM as a read-only non-volatile storage medium. The auxiliary storage device stores various programs and data used by the processor 31 when executing each program. As long as the auxiliary storage device can store information, it can be any non-volatile storage device or non-volatile memory, or a detachable storage device.

[0065] The communication device 35 exchanges data with other computers such as user terminals or servers via a network, and is, for example, a wireless LAN module. The communication device 15 can be set as other wireless communication devices or modules such as a Bluetooth (registered trademark) module, or can be set as a wired communication device or module such as an Ethernet (registered trademark) module or a USB interface.

[0066] Figure 4It is a functional block diagram of system 1 according to an embodiment of the present invention. System 1 includes a control unit 51, a distance score storage unit 54, a social graph DB (social graph database) 55, and an object model storage unit 56. The control unit 51 includes a distance score calculation unit 52. These functions of system 1 can be implemented by the server 10 and the user terminal 30 performing the same processing as the server and the client in a known client-server type system. These functions are implemented, for example, in the following ways: (1) The processor 11 of the server 10 executes a program and stores data in the storage device 14 as needed; or (2) The processor 31 of the user terminal 30 executes a program and stores data in the storage device 34 as needed; or (3) The processor 11 executes a program and stores data in the storage device 14 as needed, and the processor 31 executes a program and stores data in the storage device 34 as needed; or (4) On the basis of (1) to (3) above, data is transmitted and received between the server 10 and the user terminal 30 as needed. In this way, various functions are implemented by reading in programs, so a part or all of a functional unit (such as a software module) can also be possessed by other functional units, and the control unit 51 can also have other functional units other than those described above. In addition, at least one of the distance score storage unit 54, the social graph DB 55, and the object model storage unit 56 can also be implemented only by the storage device 34. In a variant, each functional unit of system 1 can also be implemented by hardware, such as by constructing an electronic circuit for implementing a part or all of it.

[0067] The control unit 51 has a function for providing a metaverse. In one example, the user terminal 30 is an HMD terminal installed with a terminal application Q. When the application Q is launched on the user terminal 30, the user terminal 30 communicates with the server 10 that is executing the server application P. Data required for providing services is transmitted and received between the server 10 and the user terminal 30. In one example, during the period when the application Q is launched (executed) in the user terminal 30, the server 10 communicates with the user terminal 30 regularly or intermittently, and performs processing such as moving an avatar according to the operation input in the user terminal 30. In one example, the server 10 executing the application P has a server-side function for providing a metaverse, and the user terminal 30 executing the application Q has a client-side function for providing a metaverse.

[0068] In an embodiment of the present invention, the server 10 stores the user ID and user information in association with each other in the storage device 14. In one example, the user information includes information related to a real-world user such as a user name and an email address, and information related to a user in a virtual space such as a user's avatar. In one example, the server 10 receives a user ID and a password from the user terminal 30, authenticates the user, and provides the user terminal 30 with the metaverse using the data stored in association with the user ID of the authenticated user. Once the control unit 51 authenticates the user ID, until the end of the metaverse or until a logout operation is received, the metaverse is provided based on the user information associated with the user ID, and the data is stored in association with the user ID. Once the control unit 51 authenticates the user ID, until the end of the metaverse or until a logout operation is received, the data is stored in association with the user ID. Therefore, in the following description, the situation of storing in association with the user ID may sometimes be simply expressed as storing.

[0069] Generally, the subject operating the avatar is the user, and the user recognized by the system 1 (control unit 51) is the user ID or account. In this specification, a "user of a personal account" refers to a natural person user, and a "user of a non-personal account" refers to a non-natural person user (for example, a corporate user). For example, a user of a non-personal account is a corporation that holds (creates) an official account on SNS such as Twitter and LINE, and is a user of an account existing in a virtual space. An account is a right to use the metaverse (metaverse service), and can be said to be a summary of the user ID and the user information associated with the user ID.

[0070] The control unit 51 can make the user of the personal account act in the virtual space with the help of the virtual image that is the user's avatar. In the metaverse (in the virtual space), for the user of the personal account, the virtual image represents the user himself. Therefore, for example, the field of view of the user's virtual image is sometimes represented as the field of view of the user, or the virtual image of the user in the virtual space (the virtual image representing the user) is represented as the user in the virtual space. For example, the control unit 51, for the user of the personal account, if the user A (the virtual image of the user A) of a personal account is located in the field of view of another user B in the virtual space, the display device 33 of the user terminal 30 of the user B displays (renders) a virtual space (metaverse) including the virtual image of the user A of a size corresponding to the distance in the virtual space from the virtual image of the user B to the virtual image of the user A. In addition, in the embodiment of the present invention, sometimes the display (rendering) of the virtual space including the virtual image is simply represented as displaying the virtual image. For the user of the non-personal account, the control unit 51 can make it provide services to other users in the virtual space by using the environment such as the place. In the embodiment of the present invention, a user whose character (eg, avatar) or environment (eg, location) is viewed by other users may be represented as a display target user.

[0071] The social graph DB 55 stores data of a social graph accumulated on the SNS. In an embodiment of the present invention, the server 10 is provided with a social graph DB 55. The social graph may include a social graph in a general sense or a data structure thereof. In addition, the social graph may include a data structure that defines the correlation and intimacy of multiple humans and services. The social graph is a network structure represented as a graph G = (V, E), where V represents a vertex set and E represents an edge set. For example, in an SNS such as Facebook where friends become friends only when they accept each other as friends, the graph G is a non-weighted undirected graph, while in an SNS such as Twitter where users can follow each other in a unidirectional manner, the graph G is a weighted directed graph.

[0072] The distance score calculation unit 52 calculates a distance score representing the relationship of a user to other users. In an embodiment of the present invention, the server 10 is provided with a distance score calculation unit 52. In an embodiment of the present invention, the distance score is a distance on a social graph, and the distance score calculation unit 52 calculates the distance from one user to other users on the graph G. However, the distance score can also be set to a value corresponding to the distance on the social graph. The distance score calculation unit 52 calculates the distance between users on the graph G by applying a known distance function (for example, any of the distance functions shown in non-patent documents 1 to 3).

[0073] When calculating the distance (distance score) between user A and user B, the distance score calculation unit 52 calculates the distance from user A to user B and the distance from user B to user A. Here, the distance from user A to user B and the distance from user B to user A are not necessarily equal. This is because, for example, even if one user A has the intention to unidirectionally shorten the distance to another user B, the other user B may have no intention of shortening the distance to the one user A.

[0074] In one example, the distance score calculation unit 52 calculates a distance score representing the relationship of the one user to the other user based on the weight set by the one user for the other user. In one example, the distance score calculation unit 52 calculates a distance score representing the relationship of the one user to the other user based on the number of links between the nodes representing the respective users of the one user and the other user on the social graph.

[0075] In one example, the distance score calculation unit 52 is defined as a function that numerically calculates the distance from user A to user B as shown in the following formula (1).

[0076] graph_distance(UserA,UserB)->number (1)

[0078] The distance score storage unit 54 stores the distance score representing the relationship of one user (display object user) to the other user (the user of the user terminal 30 that is to display the avatar of the display object user), that is, the distance score from the display object user to the user of the user terminal 30. In the embodiment of the present invention, the server 10 includes the distance score storage unit 54. In one example, the distance score storage unit 54 stores a table that associates the identifier of one user (for example, user ID), the identifier of the other user (for example, user ID), and the distance score as shown in the following formula (2).

[0079] scache=[<user 1 ,user 2 ,score 12 >,<user 1 ,user 3 ,score 13 >,..,<user m ,user n ,score mn >] (2)

[0081] The object model storage unit 56 stores, for users of individual accounts, models of virtual avatars at multiple stages corresponding to distance scores representing the relationship of each user with respect to other users, in association with, for example, user IDs. In one example, the model of the virtual avatar corresponding to the smallest distance score is the model of the virtual avatar for family members or the model of the virtual avatar for close friends. In an embodiment of the present invention, the server 10 includes the object model storage unit 56.

[0082] Figure 5 FIG. is an example of models of virtual avatars at multiple stages of a user stored in the object model storage unit 56. Figure 5 The virtual avatar on the left side is a model of a virtual avatar corresponding to a distance score of N (N being a specified value) or less, showing a relatively casual model. On the other hand, Figure 5 the virtual avatar on the right side is a model of a virtual avatar corresponding to a distance score greater than N, showing a relatively formal model. As Figure 5 shown, the multiple virtual avatars of a user associated with one user ID are virtual avatars that are different in clothing, etc., but can be recognized as the same person. In Figure 5 , for ease of explanation, models of two virtual avatars are shown, but the number of models of virtual avatars, clothing, etc. are not limited thereto.

[0083] The control unit 51 accepts, from users of individual accounts, the input of virtual avatars (data of virtual avatars) via the user terminal 30. For example, the control unit 51 receives data of virtual avatars from users, or accepts the production input of virtual avatars from users by known methods. The control unit 51 stores the virtual avatar for which the input has been accepted from the user as the model of the virtual avatar of the user in the object model storage unit 56. In an embodiment of the present invention, the control unit 51 accepts, from users of individual accounts, the input of multiple virtual avatars corresponding to the intimacy (distance score). The number of virtual avatars accepted for input by the control unit 51 and the magnitude of the distance score corresponding to the virtual avatar can be arbitrarily set.

[0084] In one example, the control unit 51 receives, for each user of a personal account, inputs of avatars of a family model corresponding to a distance score DS of 1 or less (0 ≤ DS ≤ 1), a relative and friend model corresponding to a distance score DS of 3 or less (1 < DS ≤ 3), a friend model corresponding to a distance score DS of 4 or less (3 < DS ≤ 4), and other models corresponding to other distance scores (DS > 4). The object model storage unit 56 stores, for each user, data of the avatars corresponding to these distance scores as models of the avatars. In one example, the object model storage unit 56 is implemented as a database, and the retrieval process in this database is defined as a process of retrieving a model with a distance score equal to or less than that of a certain user, as shown in the following formula (3).

[0085] find_model(user,score)->model (3)

[0087] Based on the distance score stored in the distance score storage unit 54, which represents the relationship between the display object user and the user of the user terminal 30, that is, based on the distance score from the display object user to the user of the user terminal 30, the control unit 51 selects (determines) one of the avatar models stored in the object model storage unit 56 as the avatar model of the display object user. The control unit 51 controls to display (render) the avatar of the display object user in the user terminal 30 based on the selected avatar model.

[0088] When the avatar of another personal account user B exists in the field of view of the personal account user A in the virtual space (the field of view of the avatar of user A), the control unit 51 selects the avatar model corresponding to the distance from user B to user A from the avatar models associated with the user ID of user B, and uses the selected model to display the avatar of user B in the user terminal 30 of user A. For example, when the object model storage unit 56 stores avatar models corresponding to three distance scores, namely, a distance score of 1 or less, a distance score of 3 or less, and other distance scores, for the avatar of user B, and the distance score from user B to user A is 2.5, the control unit 51 selects the avatar model corresponding to the distance score of 3 or less.

[0089] In an embodiment of the present invention, when it is necessary to display the avatar of another user B, such as when the avatar of user B of another personal account in the virtual space may enter the field of view of user A of the personal account, if the model of the avatar is not stored, the user terminal 30 requests the model of the avatar from the server 10 and receives the model of the avatar from the server 10. The user terminal 30 stores the received model of the avatar of the other user B in the storage device 34 of the user terminal 30 in association with the user ID of the user B. When the avatar of user B enters the field of view of user A, the user terminal 30 uses the model of the avatar of user B to display (render) the virtual space including the avatar of user B.

[0090] In one example, the function of the user terminal 30 of user A to request the model of the avatar of another user B from the server 10 can be defined as a function for retrieving the model of the avatar of another user (display object user) as viewed from the user A, as shown in the following formula (4).

[0091] get_model(User1,User2)->model (4)

[0093] In one example, the user terminal 30 of the user stores a table that associates the identifiers (such as user IDs) of other users with the models of avatars, as shown in the following formula (5). Using this table, the user terminal 30 stores the models of the avatars of other users requested from and received from the server 10.

[0094] mcache=[<user 1 ,model 1 >,<user 2 ,model 2 >,..,<user n ,model n >} (5)

[0096] In the above-described embodiment of the present invention, the server 10 includes at least a part of the control unit 51, and includes a distance score calculation unit 52, a distance score storage unit 54, a social graph DB 55, and an object model storage unit 56. In addition, in the above-described embodiment of the present invention, the user terminal 30 includes at least a part of the control unit 51, and has a function of requesting the model of the avatar from the server 10 and a function of storing the models of the avatars of other users received from the server 10. The user terminal 30 does not include the distance score calculation unit 52, the distance score storage unit 54, the social graph DB 55, and the object model storage unit 56.

[0097] Figure 6 This is an example of a flowchart showing the processing of the system 1 of the present invention. In addition, this flowchart shows the processing related to the display (rendering) of an avatar, but the processing related to the display (rendering) of the environment, etc. is the same.

[0098] In step S1, when it is necessary to display the avatar of another user B, such as when another user B may enter the field of view of user A in the virtual space, the user terminal 30 of user A determines whether there is a model of the avatar of user B as the display target user stored. If there is a model of the avatar of the display target user B stored, this flowchart proceeds to step S7. If there is no model of the avatar of the display target user B stored, this flowchart proceeds to step S2.

[0099] In step S2, the user terminal 30 sends a request for the model of the avatar of the display target user B to the server 10.

[0100] In step S3, the server 10 determines whether the distance score storage unit 54 stores the distance score from the display target user B to user A of the user terminal 30. If there is such a distance score stored, this flowchart proceeds to step S5. If there is no such distance score stored, this flowchart proceeds to step S4.

[0101] In step S4, the distance score calculation unit 52 calculates the distance score from the display target user B to user A of the user terminal 30 stored in the social graph DB 55, that is, the distance score indicating the relationship of the display target user B with respect to user A.

[0102] In step S5, the control unit 51 acquires (determines) the model of the avatar of the display target user B stored in the object model storage unit 56 that corresponds to the distance score from the display target user B to user A of the user terminal 30. In step S6, the control unit 51 sends the acquired (determined) model of the avatar to the user terminal 30 that received the request in step S2.

[0103] In step S7, the user terminal 30 uses the model of the avatar of the display target user B to display the virtual space including the avatar of user B.

[0104] Next, the main effects of the system 1 of the embodiment of the present invention will be described. In the embodiment of the present invention, the distance score storage unit 54 stores distance scores indicating the relationship of one user with respect to other users, and the control unit 51 determines (selects) the model of the virtual avatar of the display target user based on the distance score indicating the relationship of the display target user with respect to the user of the user terminal 30 stored in the distance score storage unit 54, and controls to display the virtual avatar of the display target user on the user terminal 30 based on the determined model of the virtual avatar.

[0105] With such a configuration, in the present embodiment, it is possible to change one's own appearance (e.g., the model of one's own virtual avatar) according to the other party in the metaverse, that is, it is possible to perform appearance control corresponding to the relationship with the user. In addition, it is possible to implement the inherent function of the metaverse, "dynamically select one's own appearance according to the other party who sees oneself", which cannot be achieved in the physical space, and this function is a function that humans are generally interested in, such as "human appearance is the first social message".

[0106] In addition, the above configuration of the present embodiment realizes the following "appearance control corresponding to social relations": when rendering the display device 33 corresponding to the field of view of a certain specific user, in addition to measuring the physical distance in the virtual space between other users existing in the visible range of the user and oneself, the distance relationship on the social network between the other user and oneself, that is, "intimacy", is also measured, and the display model of the user is obtained and rendered according to this distance. Such a rendering process that integrates the distance on the social network and the distance relationship in the metaverse 3D space can only be achieved by the configuration of the present embodiment.

[0107] For example, in the past, SNS controlled the display / non-display of information through two mechanisms, the social graph and the timeline, thereby realizing sociality. On the other hand, the metaverse system strongly relies on the simulation of the physical space. Therefore, when rendering information in the three-dimensional space, if only the display / non-display is simply controlled, a natural three-dimensional space cannot be achieved. In order to realize sociality in the metaverse era, it is not enough to simply control the display / non-display (visibility) of information, and the appearance (appearance) of information also needs to be controlled. By adopting the configuration of the present embodiment, a general model for controlling the overall appearance (appearance) in the metaverse can be provided.

[0108] In addition, in the present embodiment, the object model storage unit 56 stores models of avatars at multiple stages corresponding to distance scores representing the relationship of each user with respect to other users. With such a configuration, as the setting of the user's own avatar, the user can, for example, set a "public use" model, a "friend use" model, a model that can be made public to "friends of friends", an "exclude friends other than" model, an "include some friends" model, a model that is "only visible to oneself", a "custom (group, etc.)" model, etc., and can automatically change one's own appearance as seen by others according to the relationship on the SNS. Thus, for example, different displays can be made for each other as follows: when accidentally and simultaneously meeting a friend and a supervisor in a shopping center in the metaverse, when seen by the supervisor, it is rendered as a passerby, and when seen by the friend, it is rendered as a punk rock-style avatar.

[0109] Unless otherwise specified, the above effects are the same in other embodiments and variations.

[0110] As described above, the embodiments of the present invention can be implemented as the system 1, the server 10, or the user terminal 30. The embodiments of the present invention can also be implemented as a method, a program, or a computer-readable storage medium storing the program for implementing the functions of the system 1, the server 10, or the user terminal 30 and the information processing shown in the flowchart described above. In addition, the embodiments of the present invention can also be implemented as a server capable of supplying a program for implementing the functions of the embodiments of the present invention and the information processing shown in the flowchart described above to a computer.

[0111] In one or more embodiments of the present invention, the control unit 51 can receive input of the environment in the virtual space from a user of a non-personal account. For example, the environment in the virtual space is a building, a landscape, or a designed space (such as a street) in the virtual space, and is managed and utilized by a user of a non-personal account. In one example, the user of the non-personal account is a local government such as a prefecture, and this user can manage the streets in the virtual space. In one example, the user of the non-personal account is a user of an official account such as a company, and this user can manage the space (such as a shopping center facility) in the virtual space. Similarly to the virtual avatar, the number of environments for which the control unit 51 receives input and the magnitude of the distance score corresponding to the environment can be arbitrarily set. When the environment of a user A of a non-personal account is within the field of view of a user B of a personal account in the virtual space, the control unit 51 can cause a virtual space (metaverse) including a scene created based on the environment of user A having a magnitude corresponding to the distance in the virtual space from user B to user A to be displayed (rendered) on the display device 33 of the user terminal 30 of user B. The distance score calculation unit 52 can calculate the distance score between a user of a personal account and a user of a non-personal account, and can also calculate the distance score between users of non-personal accounts. For example, the distance score calculation unit 52 can calculate the distance from a user of a non-personal account to a user of a personal account and the distance from a user of a personal account to a user of a non-personal account. For example, the distance score calculation unit 52 can calculate the distance from a user of a personal account to a user of a non-personal account based on whether the user of the personal account follows the user of the non-personal account, the number of forwards, likes, etc. Therefore, the distance score storage unit 54 can store the distance score related to the user of the non-personal account. For example, the distance score storage unit 54 can store the distance score indicating the relationship of a user A of a non-personal account to a user B of another personal account.

[0112] In one or more embodiments of the present invention, for example, an object drawn in a virtual space including a virtual avatar, an environment, etc. can be represented as an object. The object model storage unit 56 stores the model of the object, and for a user of a non-personal account, can store models of multiple stages of the environment corresponding to the distance score indicating the relationship of the user to other users for each user. When there is an environment of another user B of a non-personal account within the field of view of a user A of a personal account in the virtual space, the control unit 51 can use the model of the environment corresponding to the distance from user B to user A in the model of the environment stored in association with the user ID of user B to display the environment of user B on the user terminal 30 of user A. By adopting such a structure, for example, it is possible to realize "Akihabara that looks normal to people other than otaku", "Kabukicho that looks like an ordinary shopping street to children", "Ikebukuro that looks normal to people other than fujoshi", etc.

[0113] In one or more embodiments of the present invention, for users of non-personal accounts, items in the virtual space such as products and goods in the virtual space can be set to replace the environment in the virtual space. For example, in this case, the control unit 51 can receive an input of a product in the virtual space from the user, and the object model storage unit 56 can store models of products at multiple stages corresponding to a distance score representing the relationship of the user to other users for each user. For example, in this case, when a product of another user B with a non-personal account is present in the field of view of user A with a personal account in the virtual space, the control unit 51 can use the model of the product corresponding to the distance from user B to user A in the model of the product stored in association with the user ID of user B to display the product of user B in the user terminal 30 of user A.

[0114] In one or more embodiments of the present invention, when the avatar of another user B with a personal account may enter the field of view of user A with a personal account in the virtual space and thus the avatar of the other user B needs to be displayed, if the avatars of user A and user B are within the environment of user C with a non-personal account, the user terminal 30 can use the model of the environment of user C corresponding to the distance from user C to user A and the model of the avatar of user B corresponding to the distance from user B to user A to display the avatar of user B in the environment of user C in the user terminal 30 of user A. In one example, the user with a non-personal account is a user of an official account such as a company, and this user can manage services and products in the virtual space.

[0115] In a modified embodiment of the present invention, when the avatar of another user B with a personal account may enter the field of view of user A with a personal account in the virtual space and thus the avatar of the other user B needs to be displayed, if the avatars of user A and user B are within the environment of user C with a non-personal account, the user terminal 30 can also use the model of the environment of user C corresponding to the distance from user B to user C and the model of the avatar of user B corresponding to the distance from user B to user C to display the avatar of user B in the environment of user C in the user terminal 30 of user A. By adopting such a structure, as a relationship on the SNS, not only the relationship between a specific individual and oneself is used, but also the relationship between "services" and oneself is used, thereby enabling automatic model selection corresponding to the TPO in the metaverse space. For example, it is possible to seamlessly switch between services in the metaverse, such as using a model of daily clothing when participating in a general service such as shopping and using a fantasy-style model when participating in a game service.

[0116] In one or more embodiments of the present invention, for a user of a non-personal account, the control unit 51 can, in the same manner as for a user of a personal account, enable the user to act in the virtual space by means of an avatar that is its own clone. In this case, the control unit 51 can also be configured not to receive input of the environment in the virtual space from the user.

[0117] In one or more embodiments of the present invention, the server 10 can also cooperate with a known SNS server to provide the metaverse. In this case, the server 10 may not have an SNS server or its function. For example, in this case, as long as the server 10 can identify the user ID, it may not store the user ID in association with the user information.

[0118] In one or more embodiments of the present invention, when the system 1 is configured to be able to communicate with an external device having a distance score calculation unit 52, it may not have the distance score calculation unit 52. In one or more embodiments of the present invention, when the system 1 is configured to be able to communicate with an external device having a social graph DB 55, it may not have the social graph DB 55. In one or more embodiments of the present invention, when the system 1 is configured to be able to communicate with an external device having an object model storage unit 56, it may not have the object model storage unit 56.

[0119] In one or more embodiments of the present invention, when the SNS provided by or cooperating with the system 1 is an undirected SNS, the table stored in the distance score storage unit 54 can also be set to one distance score between users.

[0120] In one or more embodiments of the present invention, it can also be configured that the control unit 51 receives only the input of one avatar from the user via the user terminal 30, and the object model storage unit 56 stores only the model of one avatar for one user ID. Even when the system 1 does not have the object model storage unit 56, the control unit 51 can determine the model of the virtual image of the display object user based on the distance score indicating the relationship between the display object user and the user of the user terminal 30 stored in the distance score storage unit 54, and perform control to display the virtual image of the display object user on the user terminal 30 based on the determined model of the virtual image. For example, in this case, the control unit 51 can be configured to store correction data such as clothing corresponding to the distance score, and automatically change the model of the virtual image based on the correction data according to the distance score. The above embodiments can also be applied equally to the environment, items, etc.

[0121] In one or more embodiments of the present invention, the system 1 or the server 10 can be set as a system or server for assisting an existing metaverse system, and has the function of the distance score storage unit 54 and the function of the control unit 51 for assisting in the following processing: determining (selecting) the model of the object of the display object user based on the distance score representing the relationship between the display object user and the user of the user terminal 30 stored in the distance score storage unit 54, and displaying the object of the display object user in the user terminal 30 based on the determined object model.

[0122] In one or more embodiments of the present invention, the display device 33 can be set as a transmissive display. In this case, the user terminal 30 is an optical transmissive HMD. The user terminal 30 set as an optical transmissive HMD enables the user wearing the HMD to visually confirm the real space through the transmissive display, and overlays and draws virtual objects in the transmissive display, thereby being able to provide an extended reality image or a mixed reality image. Additionally, in one or more embodiments of the present invention, one or more user terminals 30 can also be set as wearable terminals such as smart glasses having the same functions as any type of HMD terminal.

[0123] In one or more embodiments of the present invention, one or more user terminals 30 can be set as general personal computers. For example, the user terminal 30 can also be set as a terminal such as a smartphone or a tablet computer. In this case, the user terminal 30 includes a processor 31, an input device 32, a display device 33, a storage device 34, and a communication device 35 corresponding to the processor 11, the input device 12, the display device 13, the storage device 14, and the communication device 15 respectively. When the user terminal 30 is a personal computer, it can also be configured to connect a head-mounted display terminal (HMD terminal) to the computer, use the controller of the HMD terminal as the input device 32, and use the display of the HMD terminal as the display device 33. In addition, it is assumed that a user with a non-personal account does not move in the virtual space with an avatar, so it is assumed that this user uses a personal computer rather than an HMD terminal as the user terminal 30.

[0124] In one or more embodiments of the present invention, when user A of a personal account or a non-personal account is in a state of not being registered on the SNS or not registering personal information on the SNS, and the avatar of user A exists in the virtual space, the control unit 51 can display the avatars of other users in a form that cannot identify the avatar or the user of the avatar. In one example, when user A is in a state of not being registered on the SNS or not registering personal information on the SNS, and the avatar of user A exists in the virtual space, if the user terminal 30 of user A requests the model of the avatar of other user B from the server 10 using Equation (4), the server 10 will return the model model that makes the avatar of user B unidentifiable. unknown As the model of the avatar of user B. For example, the server 10 determines whether the distance score can be calculated in step S4. If the distance score cannot be calculated, the server 10 obtains the model model that makes the avatar of the user who accepted the request in step S5 unidentifiable from the object model storage unit 56. unknown , and in step S6, the obtained model model is sent to the user terminal 30 that accepted the request in step S2. unknown . In this case, the user terminal 30 of user A stores a table shown in Equation (6) below, for example.

[0125] mcache = [<user 1 , model unknown >, <user 2 , model unknown >,.., <user n , model unknown >] (6)

[0127] In one or more embodiments of the present invention, when the avatar of user A of a personal account or a non-personal account does not have information on the SNS or exists in the virtual space in an unregistered state, the control unit 51 can also display the environments and items of other users in a form that cannot identify the environments and items.

[0128] In one or more embodiments of the present invention, the user terminal 30 may not have the function of storing the models of the avatars of other users received from the server 10. In this case, when the user terminal 30 needs to display the model of the avatar of the target user, the user terminal 30 can use the model of the avatar requested from and received from the server 10 to display the avatar.

[0129] In one or more embodiments of the present invention, the user terminal 30 may also be configured as a thin client terminal for input and output. In this case, the user terminal 30 does not have the function of requesting the model of the avatar from the server 10 and the function of storing the models of avatars of other users received from the server 10, and the server 10 has the functions of the system 1 except for the input and output functions in the user terminal 30.

[0130] In one or more embodiments of the present invention, Figure 6 The flowchart shown may also be configured such that when the user terminal 30 of user A needs to display the avatar or environment of another user B, even if it is determined in step S1 that the model of the display target user B's avatar or environment is stored, step S2 is entered after a predetermined period. By adopting such a structure, it is possible to prevent the model stored in the object model cache 56 from being updated so that the model is different from the model stored in the user terminal 30.

[0131] As described above, in one or more embodiments of the present invention, the system 1 can also be composed of five modules: an object model cache, a graphic distance cache, a graphic distance function, a social graph DB, and a 3D object model DB. By modularizing in this way, it is possible to apply the system by changing a part of the modules.

[0132] The object model cache is a module that temporarily caches the models of objects (such as avatars, etc.) corresponding to the distance on the social graph in the user terminal 30, and is a function provided in each user terminal 30. The object model cache has a function of caching data in a table as shown in formula (5), for example. The graphic distance cache is a module having a function corresponding to the distance score storage unit 54, and is a function provided in the server 10. The graphic distance function is a module having a function corresponding to the distance score calculation unit 52. The social graph DB is a module having a function corresponding to the social graph DB 55. The 3D object model DB is a module having a function corresponding to the object model storage unit 56.

[0133] When the object model cache does not store the model of the avatar of user A when it is necessary to display the avatar of user A, it requests the model of the avatar from server 10. When receiving the request, the graphic distance cache obtains the model corresponding to the distance score representing the relationship between user A and user B of the user terminal 30 from the 3D object model DB, and sends the obtained model to the user terminal 30 that sent the request. When the graphic distance cache does not store the distance score representing the relationship between user A and user B of the user terminal 30, it requests the graphic distance function to calculate the distance score.

[0134] In the processes or actions described above, as long as there is no contradiction in the processes or actions such as using data that should not be available yet in a certain step, the processes or actions can be freely changed. In addition, each of the embodiments described above is an illustration for explaining the present invention, and the present invention is not limited to these embodiments. The present invention can be implemented in various ways as long as it does not deviate from its gist.

[0135] Description of Reference Numerals

[0136] 1: System; 2: Network; 10: Server; 11: Processor; 12: Input Device; 13: Display Device; 14: Storage Device; 15: Communication Device; 16: Bus; 30: User Terminal; 31: Processor; 32: Input Device; 33: Display Device; 34: Storage Device; 35: Communication Device; 36: Bus; 51: Control Unit; 52: Distance Score Calculation Unit; 54: Distance Score Storage Unit; 55: Social Graph DB; 56: Object Model Storage Unit.

Claims

1. A system for displaying a user's object in a virtual space, the system comprising: A distance score storage unit that stores a distance score representing the relationship of one user to other users; and A control unit that determines a model of the object of the display object user based on the distance score representing the relationship of the display object user to the user of the user terminal stored in the distance score storage unit, and controls to display the object of the display object user in the user terminal based on the determined model of the object.

2. The system according to claim 1, wherein, when the display object user is a user with a personal account, the object of the display object user is a character, and the control unit determines a model of the character corresponding to the distance score representing the relationship of the display object user to the user of the user terminal, and controls to display the character of the display object user based on the determined model of the character.

3. The system according to claim 1, wherein, when the display object user is a user with a non-personal account, the object of the display object user is an environment, and the control unit determines a model of the environment corresponding to the distance score representing the relationship of the display object user to the user of the user terminal, and controls to display the environment of the display object user based on the determined model of the environment.

4. The system according to any one of claims 1 to 3, wherein, the system includes an object model storage unit that stores models of objects at multiple stages corresponding to the distance score representing the relationship of each user to other users, the control unit selects one of the object models stored in the object model storage unit as the model of the object of the display object user based on the distance score representing the relationship of the display object user to the user of the user terminal stored in the distance score storage unit, and controls to display the object of the display object user in the user terminal based on the selected model of the object.

5. The system according to claim 4, wherein, the object model storage unit stores models of characters at multiple stages corresponding to the distance score representing the relationship of each user with a personal account to other users, the model of the character corresponding to the smallest distance score is a model of a character for family members or a model of a character for relatives and friends.

6. The system according to claim 1, wherein, the system includes a distance score calculation unit that calculates a distance score representing the relationship of one user to other users, the distance score calculation unit calculates a distance score representing the relationship of the one user to the other user based on the weight set by the one user for the other user.

7. The system according to claim 6, wherein, the distance score calculation unit further calculates a distance score representing the relationship of the one user to the other user based on the number of links between the nodes representing the users of the one user and the other user on the social graph.

8. A method for displaying a user's object in a virtual space, the method comprising the steps of: determining a model of the object of the display object user based on a distance score representing the relationality of the display object user relative to the user of the user terminal; and displaying the object of the display object user in the user terminal based on the determined model of the object.

9. The method according to claim 8, wherein the method comprises the steps of: calculating a distance score representing the relationality of one user relative to another user, and in the determining step, determining a model of the object of the display object user based on the distance score representing the relationality of the display object user relative to the user of the user terminal calculated in the calculating step.

10. A program for causing one or more computers to execute each step of the method according to claim 8 or 9.

11. A user terminal for displaying a user's object in a virtual space, wherein storing a model of the object of the other user corresponding to a distance score representing the relationality of the other user relative to the user of the user terminal, determining a model of the object of the display object user based on the score representing the relationality of the display object user relative to the user of the user terminal stored in the distance score storage unit, and displaying the object of the display object user based on the determined model of the object.

12. A server for displaying a user's object in a virtual space in a user terminal, wherein storing a distance score representing the relationality of one user relative to another user, determining a model of the object of the display object user based on the distance score representing the relationality of the display object user relative to the user of the user terminal stored in the distance score storage unit, and controlling to display the object of the display object user in the user terminal based on the determined model of the object.

Images