Information processing device, data synchronization program, data synchronization method, data synchronization system, terminal device, execution program, and information processing system

Abstract

[Problem] To provide an information processing device, a data synchronization program, a data synchronization method, a data synchronization system, a terminal device, an execution program, and an information processing system which have a configuration in which a plurality of user terminals participate and which suppress increases in processing load and communication load in a situation in which 3D data of the plurality of user terminals is synchronized. [Solution] A server 1 is connected by a network to a plurality of clients participating in a virtual space, and a synchronization target object in the virtual space is synchronized between clients in a synchronization group. The server 1 includes: a data reception means 104 for receiving a difference in the synchronization target object from a client having ownership of the object; a data update means 107 for using the difference to update replicas 120a, 120b, 120c... which replicate the synchronization target object; and a data transmission means 105 for transmitting the difference to clients other than the client having the ownership among the plurality of clients.　

Description

Information Processing Apparatus, Data Synchronization Program, Data Synchronization Method, Data Synchronization System, Terminal Device, Execution Program, and Information Processing System

[0001] The present invention relates to an information processing apparatus, a data synchronization program, a data synchronization method, a data synchronization system, a terminal device, an execution program, and an information processing system.

[0002] As a conventional technique, an information processing apparatus for synchronizing a user object of a user terminal and a simulation target of a simulator has been proposed (see, for example, Patent Document 1).

[0003] The information processing apparatus disclosed in Patent Document 1 is connected to an individual simulator that simulates the behavior of a predetermined simulation target and a user terminal that determines the behavior of a user object according to an instruction input from a user. The information processing apparatus holds copies of the user object and the simulation target. The user terminal acquires information on the simulation target from the individual simulator via the information processing apparatus at each synchronization timing determined based on a predetermined timing determination rule, and predicts the behavior of the simulation target based on the information on the simulation target acquired last at a time point outside the synchronization timing, and presents the result of the prediction to the user. In such a configuration, the processing load and the communication load are reduced.

[0004] Japanese Patent Application Laid-Open No. 2007-185446

[0005] However, the information processing device described in Patent Document 1 holds copies of user objects and simulated targets, and transmits copies of simulated targets included in the user object's field of view to the user terminal, thereby reducing processing and communication burdens. However, because the information processing device performs collision detection of simulated targets and notifies the corresponding simulator to perform the interaction, there is a processing burden on the information processing device for this detection. Furthermore, since it does not transmit copies of user objects and other simulated targets to the simulator, it has the problem that it cannot be directly applied to situations where data from multiple user terminals is synchronized in a configuration in which multiple user terminals participate. This problem becomes particularly apparent in metaverses and massively multiplayer online games (MMOs), especially in systems that require the synchronization of 3D data, where a very large number of user terminals participate, and where users can freely join and leave at any time, causing user terminals to change at any time.

[0006] Therefore, the object of the present invention is to provide an information processing device, a data synchronization program, a data synchronization method, a data synchronization system, a terminal device, an execution program, and an information processing system that suppress the increase in processing load and communication load in a configuration in which multiple user terminals participate and 3D data from multiple user terminals are synchronized.

[0007] One aspect of the present invention provides the following information processing device, data synchronization program, data synchronization method, data synchronization system, terminal device, execution program, and information processing system to achieve the above objective.

[0008] [1] An information processing device connected by a network to a plurality of clients that synchronize basic data representing a three-dimensional object, the information processing device comprising: receiving means for receiving from the plurality of clients any difference of the basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data; and updating means for updating the duplicate basic data which is a copy of the synchronized object based on the received difference, the action, or the conversion data. [2] The information processing device according to [1], further comprising transmitting means for transmitting any of the difference, the action, or the conversion data to the plurality of clients. [3] The information processing device according to [2], wherein the transmitting means transmits the encoded difference, the action, or the conversion data without decoding it. [4] The information processing device according to [2], further comprising codec means for decoding the received difference, the action, or the conversion data, wherein if there is no problem in the result of checking the result of updating the duplicate basic data with any of the difference, the action, or the conversion data, the transmitting means transmits any of the difference, the action, or the conversion data to the plurality of clients. [5] A data synchronization program executed on an information processing device connected via a network to a plurality of clients that synchronize basic data representing a three-dimensional object, wherein the information processing device functions as a receiving means for receiving either the difference in basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data from the plurality of clients, and as an updating means for updating duplicate basic data that replicates the synchronization target using any of the received difference, action, or conversion data. [6] A data synchronization method executed on an information processing device connected via a network to a plurality of clients that synchronize basic data representing a three-dimensional object, wherein the data synchronization method comprises the steps of receiving either the difference in basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data from the plurality of clients, and updating duplicate basic data that replicates the synchronization target using any of the received difference, action, or conversion data.[7] A terminal device connected via a network through a relay server to a plurality of clients that synchronize basic data representing a three-dimensional object, and having receiving means for receiving either the difference of the basic data to be synchronized, an action on the basic data, or conversion data that transforms the basic data from the plurality of clients, and updating means for updating the duplicated basic data which is a copy of the synchronized object based on the received difference, the action, or the conversion data. [8] The terminal device according to [7], further comprising execution means for updating the basic data of the synchronized object according to the execution content, codec means for encoding either the difference, the action, or the conversion data according to the changes made by the execution means, and transmission means for transmitting either the difference, the action, or the conversion data to the plurality of clients. [9] The terminal device according to [8], wherein the execution means renders the basic data according to the execution content.

[10] An executable program executed on a terminal device connected via a network through a relay server to a plurality of clients that synchronize basic data representing a three-dimensional object, wherein the terminal device functions as a receiving means for receiving either the difference in the basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data from the plurality of clients, and as an updating means for updating the duplicated basic data which is a copy of the synchronized object based on the received difference, the action, or the conversion data.

[11] An information processing system comprising a network connecting a plurality of clients and an information processing device that synchronize basic data representing a three-dimensional object with each other, wherein the information processing device has receiving means for receiving from the plurality of clients any of the following: the difference in basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data, and updating means for updating the duplicated basic data which is a copy of the synchronized object based on the received difference, the action, or the conversion data, and the plurality of clients each have receiving means for receiving from the plurality of clients other than themselves any of the difference in basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data, and updating means for updating the duplicated basic data which is a copy of the synchronized object based on the received difference, the action, or the conversion data.

[0009] According to the present invention, in a configuration in which multiple user terminals participate and 3D data from multiple user terminals is synchronized, it is possible to suppress increases in processing load and communication load.

[0010] Figure 1 is a schematic diagram showing an example of the configuration of a synchronization system according to an embodiment. Figure 2 is a schematic diagram showing the relationship between terrain information (area) and room (world). Figure 3 is a block diagram showing an example of the server configuration according to an embodiment. Figure 4 is a diagram showing the configuration of terrain information (area management information). Figure 5 is a diagram showing the configuration of room management information. Figure 6 is a diagram showing the configuration of participant management information for a certain room. Figure 7 is a diagram showing the configuration of replica management information for a certain area. Figure 8 is a diagram showing the configuration of replica management information for a certain room. Figure 9 is a flowchart to explain an example of synchronization operation. Figure 10 is a flowchart to explain an example of synchronization operation based on ownership. Figure 11 is a flowchart to explain an example of synchronization operation when joining midway. Figure 12 is a flowchart to explain an example of synchronization operation when leaving midway. Figure 13 is a flowchart to explain another example of synchronization operation when leaving midway. Figure 14 is a flowchart to explain an example of synchronization operation when an area owner joins midway. Figure 15 is a flowchart to explain another example of synchronization operation when an area owner joins midway. Figure 16 is a diagram showing the relationship between the synchronization operation of area information synchronization objects managed for each area and non-area information synchronization objects managed for each room. Figure 17 is a schematic diagram showing an example of the configuration of a terminal device. Figure 18 is a schematic diagram to explain an example of the synchronization operation of a 3D object. Figure 19 is a schematic diagram to explain the codec, transmission / reception, and rendering operation of a 3D object.

[0011] [Embodiment] In the synchronization system of this embodiment, multiple participants engage in activities in a common virtual space, such as a metaverse or a massively multiplayer online game (MMOG). This virtual space includes the ground, buildings, rooms and their associated objects (furniture, clocks, doors, etc.), as well as services provided in the ground, buildings, and rooms, and programs executed in those services. These are managed as terrain information, which will be described later. This terrain information of the virtual space corresponds to the content information of the experiences that can be had in the virtual space (hereinafter sometimes referred to as "game experiences"). In this embodiment, this terrain information is also centrally managed on the server and can be obtained from the server when joining the virtual space. Compared to cases where content is downloaded or distributed to participant terminals in advance, it becomes possible to experience more content in a more up-to-date state. Furthermore, since the latest content information is managed on the server, participants can also contribute to the development and modification of content.

[0012] In metaverses and massively multiplayer online games, participants' activities interact with each other as time progresses in the virtual space. Multiple virtual characters controlled by multiple participants interact, fight, cooperate, work together, or collect items as time passes. Through these activities, characters increase or decrease the items they possess and experience points related to combat and actions. Interaction between characters allows players to share experiences in the virtual space, creating a sense of immersion and unity. Furthermore, the ground, buildings, rooms, and associated objects included in the terrain information also change based on character actions (for example, ground deformation, building expansions and demolition, furniture movement, clock operation, door opening and closing, etc.).

[0013] In such a virtual space, client terminals display synchronized changes in virtual characters, items, and terrain information over time, allowing participants to experience the same virtual space. In other words, it is necessary to synchronize virtual characters, items, and other objects and terrain information that exist in the metaverse or game between each client.

[0014] In particular, in the metaverse, it is often possible to define rooms (worlds, areas) that participants can freely create, modify, and customize, such as a participant's home world or a world created by a participant. These can be personal rooms expressing a favorite hobby's worldview (anime, space travel, cats, minigames, dungeons, etc.) or rooms for companies intended for online sales, showrooms, or exhibitions. Note that rooms, worlds, and areas are often synonymous with virtual space, but in this embodiment, areas and rooms correspond to certain regions where experiences can be shared in all or part of the virtual space. Hereafter, virtual space may mean all of the virtual space provided by this embodiment (a collection of certain regions divided as places for sharing experiences, or the entire virtual space provided by a virtual space application or virtual space service using this embodiment), or it may mean a part of the virtual space that is a certain region divided as a place for sharing experiences. When it is necessary to distinguish between them, the former may be called one or more virtual spaces, and the latter may be called an area or room, as will be described later. Due to various restrictions and circumstances, there may be multiple virtual spaces (corresponding to rooms) where the same content (corresponding to areas) can be experienced. Even if participants enter the same virtual space simultaneously, they may be assigned to different virtual spaces (rooms, which represent a parallel world state for the participants), and despite being able to experience the same area's content, they may not encounter each other within those virtual spaces (rooms). In this embodiment, the content of the virtual space itself is sometimes referred to as a uniquely existing area, while the virtual spaces that participants actually experience upon joining an area are sometimes referred to as rooms, of which there are zero or more. When there is only one virtual experience space (room) that can be joined in a single virtual space (area), then area and room are almost synonymous.

[0015] In other words, in network applications such as the metaverse, online games as massively multiplayer online games, and other conferencing systems, technologies are used to synchronize content information such as documents and videos between participants' devices. However, as the number of participants and the number of content items to be synchronized (synchronized objects; hereafter, all objects whose changes over time are synchronized, including virtual characters, items, and terrain information, will be referred to as "synchronized objects") increase, the amount of data required for synchronization increases, leading to increased synchronization delays (lag or latency), making the network application unusable.

[0016] Furthermore, the synchronized objects described above can be distinguished into two types: area information and non-area information. Area information is the terrain information described above, and is attribute information of the area necessary to reproduce the content (area) that constitutes the virtual space regardless of the terrain. It includes synchronized information and may change through character actions (e.g., deformation of the ground, addition or demolition of buildings, movement of furniture, operation of clocks, opening and closing of doors, etc.). Non-area information is synchronized information other than area information and mainly consists of participant attribute information. This includes characters, owned items, combat-related values, and experience points for actions. The boundary between area information and non-area information, and the specification of synchronized objects, can be changed by the game design. For example, area information manages the content of the area, and the results of collaborative work are applied and maintained. On the other hand, non-area information manages the status of users and their possessions within the room, and maintains the progress of ongoing tasks, etc. (Hereafter, all information that users uniquely manage and operate in the virtual space, such as the status of users and their possessions, including avatars, clothing, weapons, and other equipment and belongings, may be collectively referred to as user information. By sharing user information with each other, participants in the virtual space (room) can recognize and share the activities of each other's avatars.) While terrain information and user information are representative examples of area information and non-area information, they are not limited to these. However, the following explanation will mainly focus on terrain information and user information, but these explanations can be interpreted as explanations of area information and non-area information.

[0017] In the metaverse, MMORPGs, and especially FPS (First-Person Shooter) games, synchronization delays are fatal and render the service unusable. Therefore, efforts are being made to reduce the number of objects to be synchronized and the number of participants.

[0018] Reducing the number of objects to be synchronized requires ingenuity in the metaverse and game design. To reduce the number of participants, the shared virtual space can be divided, or multiple identical virtual spaces (rooms, worlds) can be created with participant limits. However, with this method, participants in different rooms (participants in parallel worlds) cannot share the game experience. Therefore, to fully share the game experience, it is desirable for all participants to join the same room.

[0019] Increasing the number of shared objects and participants that can be accommodated in a single room greatly improves the freedom of game design and the players' gaming experience. Below, we will describe how to increase the number of shared objects and participants that can be accommodated in a single room (world) and how to configure a synchronization system to suppress synchronization delays. Furthermore, we will describe a synchronization system that simplifies the methods of identifying, selecting, and sharing rooms to join. Finally, we will describe a synchronization system that facilitates the customization (creation, editing, deletion, etc.) of ground, buildings, rooms, and their associated objects included in terrain information.

[0020] (Configuration of the synchronization system) Figure 1 is a schematic diagram showing an example of the configuration of the synchronization system according to the embodiment.

[0021] This synchronization system consists of a server 1 acting as an information processing device and terminal devices 2a, 2b, and 2c acting as clients, all connected to each other via a network to enable communication. The devices and network may be cloud services, personal computers, game consoles, LANs, the internet, or virtual machines and virtual networks.

[0022] Server 1 is a server-type information processing device that operates in response to requests from terminal devices 2a, 2b, and 2c operated by the operator. It contains electronic components such as a CPU (Central Processing Unit), HDD (Hard Disk Drive), flash memory, volatile memory, and a LAN board (wireless / wired) that have functions for processing information. Server 1 communicates with terminal devices 2a, 2b, and 2c, and synchronizes the data handled on terminal devices 2a, 2b, and 2c by sending and receiving data. Examples of such servers include metaverse servers and game servers. Server 1 may be configured as multiple clusters, or configured to perform distributed processing. Alternatively, it may be configured as a virtual machine in a cloud environment.

[0023] Terminal devices 2a, 2b, and 2c are terminal-type information processing devices that operate based on a program and are equipped with electronic components such as a CPU, HDD, or flash memory that have functions for processing information. Terminal devices 2a, 2b, and 2c have programs such as a metaverse or MMOG pre-installed, and operate based on this program, sequentially outputting data to server 1 as a result of their operation, and receiving data from server 1 as a result of the operation of other terminals, thereby synchronizing objects among terminal devices 2a, 2b, and 2c at a high frequency. Although terminal device 2 is depicted as three devices, it may be a single device, two devices, or composed of four or more devices, and preferably provides a configuration that allows communication even when thousands of terminal devices are connected. The programs such as the metaverse or MMOG may be independent applications or programs that run on a web browser.

[0024] A network is a communication network capable of high-speed communication, such as the internet, intranets, LANs (Local Area Networks), and wired or wireless communication networks like the fifth-generation mobile communication system (5G).

[0025] For example, Server 1 and terminal devices 2a, 2b, and 2c operate to synchronize information (synchronization target objects) held as objects within each device in response to activities on the metaverse and the progress of the game. They maintain copies of the objects within each device as replicas (duplicate objects), and assist in the synchronization operation by sending and receiving replicas (relaying) to each device to copy the replicas (as tertiary copies) while managing the ownership of these replicas. Although Server 1 does not actually perform the synchronization operation itself, but rather acts as a relay, the relay for synchronization is sometimes referred to more broadly as "synchronization," and this operation as "synchronization operation" or "synchronization process." More specifically, the contents of the replica on Server 1 are updated only when the object on the terminal device of the user who owns it is updated, and the object (tertiary copy) on the terminal device of other users who do not own it is updated through the synchronization operation. Objects (tertiary copies) on the terminal devices of other users who do not own them are not updated. Hereafter, the configuration and operation details of when Server 1 acts as a relay server to synchronize objects between terminal devices 2a, 2b, and 2c will be specifically described in the embodiments.

[0026] Furthermore, the terms "object," "class," "field," and "instance" used in this embodiment are used with the same meaning as the terms used in, for example, Java®, C++, C#, Python, JavaScript®, Ruby, etc. However, hereafter, classes and instantiated classes (instances) may be collectively referred to as "objects." A "class" is a collection of processes for executing a program, and within a class, there are methods that execute the processes and fields that are used by the methods when executing the processes. An object has zero or more fields, and each field has one or more field names and field values.

[0027] Figure 2 is a schematic diagram showing the relationship between areas (terrain information 111) and rooms (worlds).

[0028] The terrain information 111a to 111c, as area information, defines the terrain and buildings (and their accessories) that serve as the setting for the metaverse or game, such as a hobby world or business setting freely created by an individual or company. When Server 1 accepts a user's request to participate, it places the user's avatar 4a to 4c into rooms 112a to 112c corresponding to the desired terrain information 111a to 111c. (Hereafter, participating in a room to experience an area, whether whole or part, may simply be described as "participating in an area." If there is only one room corresponding to an area, "participating in an area" and "participating in a room" are synonymous. However, if there are multiple rooms corresponding to an area, it may be more precise to use "participating in a room" rather than "participating in an area" when participating in a specific room. Typically, participating in a virtual space involves participating in both an "area" and a "room.") Server 1 may always keep all area management information, all room management information, and their respective replica management information 114 in memory. However, in practice, it is practical to keep only the active management information (area management information, room management information, and replica management information) for which participants exist. In other words, when all avatars 4a to 4c leave an area or room (when there are no participants), Server 1 saves the settings of the corresponding management information and deletes it from memory. In this operation, when a request to join an area is received, the area management information, room management information, and their respective replica management information will join the corresponding management information if Server 1 has the corresponding management information (area management information, room management information, and replica management information) that has already been generated. If some or all of the generated management information is missing, the area management information, room management information, and replica management information will join the respective management information that Server 1 has generated (restored).Furthermore, in order for each piece of management information (area management information, room management information, and replica management information) to permanently accumulate the results of updates to the state of the objects it manages, regardless of the existence of the area or room (more precisely, the presence or absence of area participants and room participants, respectively), it may be appropriate to update the respective content periodically at regular intervals to synchronize it with the changes in the replica management information, in order to synchronize the objects corresponding to the content held by the data management program 110, area management information 111, and room management information 112. For example, if a room 112c is created for one area (terrain information 111c), and the number of participants in that room 112c reaches the upper limit due to the performance of server 1, another room 112x may be created for the area (terrain information 111c). Even in that case, by synchronizing the terrain information 111c of rooms 112c and 112x, avatars participating in rooms 112c and 112x can share the experience of changes in terrain information, even if they cannot see each other. For example, even if they are unaware of each other's presence, they can share the results of changes in areas (terrain information 111) such as construction work and excavation work, which can be divided among rooms, so avatars participating in rooms 112c and 112x can work together. Room participants experience both the synchronization of area management information and the synchronization of non-area information (synchronization of information other than area management information within the room, such as avatars and objects) (see Figure 16 below). Terrain information 111a to 111c are managed by identifiers called area IDs, and rooms 112a to 112c and 112x are managed by identifiers called room IDs.

[0029] To participate in a room to experience an area (terrain information 111a to 111c), each area (terrain information 111a to 111c) is assigned a unique identifier, such as a URL (Uniform Resource Locator; in the context of this embodiment, this may be read as URI (Uniform Resource Identifier)). Entering this URL into the web browser of terminal devices 2a, 2b, 2c, etc., activates a program for the metaverse or game, and processing is initiated to allow participation in the room corresponding to the URL. If a room corresponding to the specified area (terrain information) does not yet exist on server 1 (is not active), or if all rooms corresponding to the specified area (terrain information) are full, the user will participate in a room newly created by server 1. If a room corresponding to the specified area (terrain information) exists on server 1 and there is still room capacity available, the user will join one of the available rooms midway through the session. Furthermore, to join a session by directly specifying existing rooms 112a-112c and 112x, such as by being invited by an existing participant, it may be possible to specify a unique URL for each room 112a-112c and 112x. In that case, if the room the participant tried to join no longer exists for any reason, such as the inviting participant leaving, the system may not generate an error but instead create a new room in the corresponding area and allow the participant to join.

[0030] Furthermore, each area (terrain information 111a to 111c) has an area ID and an owner defined, and the owner (user) can edit the area's contents, such as terrain information 111a to 111c. When editing terrain information 111a to 111c, a unique URL is used for that area (terrain information 111a to 111c), and by accessing the terrain editing page in the web browser of terminal devices 2a, 2b, 2c, etc., and entering the URL on that page, the corresponding terrain information is loaded and editing becomes possible. Authentication when joining a room and when starting terrain editing of an area will use various existing authentication methods, such as terminal device authentication or user ID / password authentication. Therefore, ownership is not necessarily uniquely associated with a terminal device, and it is possible to participate and edit as an owner from another terminal as appropriate.

[0031] (Server Configuration) Figure 3 is a block diagram showing an example of the configuration of Server 1 according to the embodiment.

[0032] Server 1 comprises a control unit 10 consisting of a CPU (Central Processing Unit) and the like, which controls each part and executes various programs; a storage unit 11 consisting of a storage medium such as flash memory that stores information; a memory 12 consisting of a volatile recording medium that temporarily stores information; and a communication unit 13 that communicates with the outside world via a network.

[0033] The control unit 10 functions as a terrain information management means 100, terrain information editing means 101, room management means 102, participant management means 103, data receiving means 104, codec means 108, data transmission means 105, replica management means 106, data update means 107, etc., by executing a data management program 110 as a data synchronization program.

[0034] The terrain information management means 100 manages terrain information 111, which is the terrain and buildings (and their accessories) that serve as the setting for the metaverse or game. Specifically, it loads terrain information 111 for rooms, manages area IDs, issues URLs as an example of identifiers corresponding to terrain information 111, and updates terrain information 111. The identifier is not limited to URLs, and it is preferable that it be a general-purpose identifier that can be used in an open environment rather than being limited to a specific MMOG or metaverse application. It may also be an object ID, an object UUID (Universally Unique Identifier), or a part of such identifiers, or a string containing such identifiers, or a byte sequence that is the result of operations such as compression or encoding of such identifiers or strings.

[0035] The terrain information editing means 101 edits terrain information 111, particularly terrain data, in response to requests from terminal devices 2a to 2c. Specifically, terrain data includes terrain, buildings, and their associated structures. The terrain information editing means 101 modifies the shape and arrangement of terrain and buildings, and modifies the operation of terrain associated structures. It also edits programs (scripts) that describe operations associated with terrain data (for example, opening and closing doors). Although not shown in the diagram, non-terrain information, such as avatars, monsters, equipment like vehicles, and chat bulletin boards, may also be edited in a similar manner to the terrain information editing means 101.

[0036] The room management means 102 creates, pauses, resumes, deletes, etc., a common virtual space called a room, which is usually assigned a room ID, and manages room management information 112 according to these operations.

[0037] The participant management means 103 records the user ID of each participant joining the room, the socket ID of the participant's terminal device, etc., and stores it in the storage unit 11 as participant management information 113 for that room. In a system that provides only a single virtual space, there is no need to distinguish between rooms, so a room ID may not be necessary.

[0038] The data receiving means 104 receives zero or any number of objects (including terrain data) from terminal devices 2a, 2b, and 2c via the communication unit 13 as data to be synchronized, duplicates them, and stores replicas 120a, 120b, and 120c as duplicated objects in the memory 12. The data to be synchronized are terrain information 111 loaded into the room and objects such as room participants and items. Synchronization objects related to area management information are managed on an area basis (replica management information 114A), and synchronization objects related to non-area information are managed on a room basis (replica management information 114B).

[0039] The codec means 108 decodes the data if the object received by the data receiving means is encoded in a format such as LZ4 compression if it is a voxel (PNG image format if it is a height map, or FBX zip compression or glb + draco compression if it is polygon data), and the decoded data is stored in memory 12 by the data receiving means 104 as replicas 120a, replica 120b, and replica 120c. Memory 12 holds both the encoded and decoded versions of replicas 120a, replica 120b, and replica 120c, and both versions may be referred to as replicas 120a, replica 120b, and replica 120c.

[0040] The data transmission means 105 transmits replicas 120a, 120b, and 120c via the communication unit 13 to terminals designated as destinations based on the transmission rules of participant management information 113 and replica management information 114A or 114B. The data transmission means 105 transmits replicas 120a, 120b, and 120c to the designated terminal devices 2a, 2b, and 2c in their undecoded state.

[0041] When replicas 120a, 120b, and 120c are generated, the replica management means 106 records the replica IDs of replicas 120a, 120b, and 120c, the user ID of the owner who owns the replica of the replica with the replica ID, a transmission rule indicating the user to synchronize, and a policy such as whether or not the replica can be transferred. For replicas of area information and replicas of non-area information, this information is stored in the storage unit 11 as replica management information 114A on an area basis and replica management information 114B on a room basis, respectively.

[0042] When an object corresponding to replicas 120a, 120b, or 120c is updated, the data update means 107 receives the difference from the user's terminal device that owns the object based on the replica management information 114A or 114B via the data receiving means 104 and updates the corresponding replica. Receiving the difference is preferable from the viewpoint of reducing the amount of information on the network, but the updated object itself may also be received. Furthermore, the data transmission means 105 sends the difference to the terminal designated as the destination based on the participant management information 113 and the replica management information 114A or 114B. Sending the difference is preferable from the viewpoint of reducing the amount of information on the network, but the updated object itself may also be sent. In addition, if the data receiving means 104 receives an action that causes a difference instead of a difference (for example, a force that changes the shape of an object), the data transmission means 105 may send the action. Terminal devices 2a, 2b, and 2c that receive the action shall change their tertiary according to the action. Furthermore, replica management information 114A is used to manage objects to be synchronized with terrain information 111, and replica management information 114B is used to manage objects to be synchronized with non-area information such as avatars and other characters, and items.

[0043] The storage unit 11 stores a data management program 110 that causes the control unit 10 to operate as the above-described respective means 100 - 107, topographic information 111, room management information 112, participant management information 113, replica management information 114A and 114B, etc. The storage unit uses a relational database, a file system, or the like. For speeding up, an in-memory database such as Redis may be used, or used in combination.

[0044] The memory 12 temporarily stores replicas 120a, b, c... and other information.

[0045] FIG. 17 is a schematic diagram showing a configuration example of the terminal devices 2a, 2b, 2c. The terminal devices 2a, 2b, 2c are collectively referred to as the terminal device 2 for explanation.

[0046] The terminal device 2 includes a control unit 20 composed of a CPU or the like that controls each part and executes various programs, a storage unit 21 composed of a storage medium such as a flash memory that stores information, a memory 22 composed of a volatile recording medium that temporarily stores information, a communication unit 23 that communicates with the outside via a network, an operation unit 24 composed of a keyboard, a mouse, a touch pad, etc., that receives inputs from users 3a, 3b, 3c, and a display unit 25 that displays the processing results of objects such as a metaverse and a game.

[0047] By executing a metaverse execution program 210 described later, the control unit 20 functions as communication establishment means 200, data reception means 201, codec means 202, data transmission means 203, metaverse execution means 204, etc.

[0048] The communication establishment means 200 designates a user ID and provides a socket ID of the terminal device 2 to establish communication with the server 1.

[0049] The data receiving means 201 receives zero or any number of objects (including terrain data) from the server 1 as data to be synchronized via the communication unit 23, duplicates them, and stores the duplicated objects tertiaries 220a, 220b, and 220c in the memory 22. If the user ID is that of the user who owns the data, the original object is received from the server 1 as data to be synchronized and stored in the memory 22 as object 221a... Furthermore, if tertiaries 220a, 220b, and 220c are changed, the data receiving means 201 receives the update difference from the server 1 and updates tertiaries 220a, 220b, and 220c. Alternatively, the data receiving means 201 receives an action that causes a difference (for example, a force that changes the shape of an object) from the server 1 instead of the update difference, and updates tertiaries 220a, 220b, and 220c by reflecting the action.

[0050] The codec means 202 decodes the data if the objects received by the data receiving means are encoded, and the decoded data is stored in the memory 22 by the data receiving means 201 as tertiaries 220a, 220b, 220c..., objects 221a.... The memory 22 holds both encoded and decoded versions of the tertiaries 220a, 220b, 220c..., objects 221a..., and both versions may be referred to as tertiaries 220a, 220b, 220c..., objects 221a....

[0051] The data transmission means 203 transmits the update difference to the server 1 when the object 221a... is updated by the metaverse execution means 204, which will be described later. Alternatively, instead of transmitting the update difference, the data transmission means 203 transmits an action that causes the difference (for example, a force that changes the shape of the object, or conversion data that causes the receiving device to modify the shape of the object). Whether to transmit the difference or the action is determined by considering the respective advantages and disadvantages described later and by the implementation details of the metaverse, game, etc. to be executed. Note that it may be fixed to transmit either the difference or the action, or it may be dynamically switched between the two.

[0052] The metaverse execution means 204 updates the tertiaries 220a, 220b, 220c..., objects 221a... based on the content of the metaverse and game execution according to the content of the input to the operation unit 24, and displays the tertiaries 220a, 220b, 220c..., objects 221a... on the display unit 25. The metaverse execution means 204 also performs other processing necessary for the progress of the metaverse.

[0053] The memory unit 21 stores a metaverse execution program 210, etc., which causes the control unit 20 to operate as the means 200-204 described above.

[0054] The advantages and disadvantages of the data transmission means 203 transmitting differences and transmitting operations are as follows.

[0055] When sending differences between objects, if the amount or frequency of differences is large, the amount of network data sent and received over the network increases, which can increase costs and affect performance. On the other hand, when sending action or transformation data, it is sufficient to send small amounts of data for each action, so the amount of network data required is small.

[0056] However, in the case of actions and converted data, if the state of the receiving side before the action is the same as that of the sending side, the result will be the same. However, if the states differ between terminals, the result may be completely different. In particular, when there are multiple terminals sending the action, the order in which the actions are processed may vary, and the results may differ completely depending on the processing timing, so it is necessary to design the system to guarantee the order of actions, or to design the system to send and receive data that can reproduce the same result even if the processing timing is different.

[0057] A similar phenomenon can occur when sending differential data, but the problem is relatively easy to fix. This is because, for example, if the differential data is the type that overwrites a portion of the data, the same state can be achieved by simply processing it so that the most recent differential takes precedence. If the differential data is data that increases or decreases, although the state may differ temporarily, adding up all the differentials will bring it to the same state.

[0058] Furthermore, data that is originally in the form of arrays such as voxels, images, or videos is often sent as differential data, while the creation and destruction of new objects, and notifications, are often sent as actions. From a programmer's perspective, if you simply want to update data stored in a variable, you send it as differential data. However, if you want to change not only the data but also other data in conjunction, or execute other processes, and you want to process it in a way similar to calling a function, then sending and receiving as actions is used.

[0059] Furthermore, in the case of games, sending an action is more suitable for preventing cheating. For example, if you want to synchronize that a player has dealt 10 damage to an enemy, sending the difference would simply send the value of the enemy's HP reduced by 10. However, if the value is changed to 100 damage through cheating, the process may still go through. While limitations on damage values ​​can prevent or detect cheating, there are cases where it is not possible to determine whether the data of 100 damage is correct or incorrect, such as when there is an attack method that is very slow to activate but deals a lot of damage.

[0060] On the other hand, when sending a message as an action, it sends a message saying "I attacked enemy A with the weapon I currently have equipped," and the server calculates the damage value from the attack power of the weapon the player is currently equipped with and the enemy's defense power, and then processes the server to reduce the enemy's HP. This makes it more difficult to tamper with the damage value and makes the system more resistant to cheating.

[0061] When sending data from the server to the client, anti-cheat measures are unnecessary, so data can be sent differentially. However, it is more common to implement a system where messages received by the client as actions are relayed directly to other clients. In this case, the message indicating that damage has been dealt to an enemy is relayed to other players and can be used as a trigger for damage animations and other effects. However, since the HP value after reducing the enemy's HP should be the same, the server may send a separate message with the correct HP value to match the correct value. While this configuration is strong against cheats, it often requires dedicated implementation for each action, which tends to increase implementation effort. On the other hand, sending differential data simply requires applying the data difference, making it easier to reuse implementations and resulting in simpler implementation with less effort.

[0062] Figure 4 shows the structure of management information for the entire area (all terrain information 111).

[0063] The terrain information 111 is information about terrain, buildings, and their appendages that are loaded into a virtual space called a room. When a virtual space (area) is created, one piece of terrain information (corresponding to one row in Figure 4) is created. The terrain information 111 includes an area ID for identifying the terrain information, terrain data indicating the shape and arrangement of terrain and buildings, the operation of terrain appendages, a URL for specifying an area to participate in in order to experience the virtual space represented by the area, or for accessing the terrain information editing page described later, the update date and time which indicates when the terrain information was updated, the creator indicating the participant ID that created the room, the owner and editor indicating the participant ID that has ownership or editing rights to the room, a destination rule for sending a replica of the terrain information for synchronization, and a new room creation rule that specifies the rules for creating a new room using terrain information 111 for which a room has already been created. Note that there may be multiple creators, owners, and editors. Also, if the owner of the terrain information 111 does not have editing rights, temporary editing rights may be granted to that owner. The rules for creating a new room may be conditional, such as "more than 2,000 participants," or they may be based on external requests, such as "respond to participant requests." The area ID and URL are generated when the terrain information editing means 101 creates the terrain information (one line). The terrain data is updated according to the creation and editing by the terrain information editing means 101, and also according to the results of activities by participants within the area that affect the terrain information. For example, depending on the various operation rights (create, delete, edit, view, transfer of ownership) for the terrain information, one or more participant IDs can be specified and managed, such as creator (create only), owner (can create, delete, edit, view, and transfer of ownership), editor (can edit and view only), and viewer (can view only).Furthermore, the URL may be issued, for example, at the owner's request, either as the URL itself or as an equivalent URL with modifications such as shortening. The owner can then disclose this URL to other participants or to people completely unfamiliar with the virtual space service (potential new registrants and participants who are not registered or used by the virtual space service), thereby inviting them to join the virtual space using only the URL, whether they are users of the virtual space service or potential users who have never used it before. However, for unregistered users of the virtual space service, registration or downloading of the virtual space service may be required before joining the virtual space.

[0064] Figure 5 shows the configuration of the room management information 112.

[0065] The room management information 112 includes a room ID for identifying the room, which is a virtual space; an area ID indicating the terrain information 111 deployed in the room; a URL for joining the room; and a creation date and time, which is the date and time the room was created.

[0066] Figure 6 is a diagram showing the configuration of participant management information 113.

[0067] The participant management information 113 is created for each virtual space room and includes a user ID to identify the participant, the participant's username, a socket ID for a socket (TCP socket or UDP socket) containing information necessary for sending and receiving, such as the IP address and port number of the terminal device operated by the participant and server 1, and a participation date and time indicating the date and time of joining the room. Furthermore, it may also describe the participant's role within the room, the groups they belong to, the rules for selecting synchronized objects to send and receive when shared objects are updated (close distance, within line of sight), the format of reception, or its preference (default).

[0068] Figure 7 shows the configuration of replica management information 114A for synchronized objects related to terrain information managed on an area-by-area basis.

[0069] The terrain information replica management information 114A is created for each area, which is a virtual space (template), and includes a list of active room IDs for the area, a replica ID to identify the replica, an owner indicating the participant ID that owns the replica, a transmission rule for when to send the replica for synchronization, and a transfer indicating whether the ownership of the replica is transferable. Active participants in the area can be obtained by referring to the list of room IDs and the participant management information 113 for each room, but the terrain information replica management information 114A may also redundantly maintain a list of active participants (which needs to be updated in synchronization when the room's participant management information 113 is updated). Furthermore, if transfer is possible, rules for the transfer method, the data format for sending and receiving when the object is updated, or their preferences may be described. The transmission rules may also be defined by the performance of the client terminal, the network status of the client terminal, the role of the user who owns the data, and the group to which the user belongs. Ideally, the object owner for terrain information should be the owner of the area, but it is not realistic to assume that the owner is always participating in the rooms of that area. In typical usage scenarios, the owner of the synchronized object for terrain information is either a special user (such as a dummy client) for all replicas, or the oldest participant in the room corresponding to the area. When using a dummy client, a dummy client may be prepared on server 1 or another server, and this dummy client may be designated as the owner of the terrain information 111. Here, "dummy client" refers to, for example, a device running on server 1 that runs part or all of the client software and participates in a synchronization group to synchronize with clients in other synchronization groups. Here, the oldest participant is the first participant to join the area, or, if the oldest participant (such as the first participant) leaves, the next oldest participant. When determining the owner, other methods may be used, such as selecting the participant with the smallest participant ID.Most topographic information (e.g., ground shape information G1, G2 and building information B1) should be transferred and maintained, but there may be some that do not need to be transferred, such as special work marks (e.g., M1).

[0070] If a dummy client becomes the owner (of the synchronized object), the dummy client is activated when any room in that area becomes active, retrieves the area information from area management information 111, and creates and holds a primary for the synchronized object in that area. If the oldest participant becomes the owner (of the synchronized object), when the participant joins a room in the area, they retrieve the area information from area management information 111, and if there are other participants in the area (one of whom is the oldest participant), they retrieve a replica of the synchronized object in that area from server 1, create and hold a tertiary. If there are no participants in any room in the area (including cases where no rooms exist), i.e., if the participant is the first participant in a room in the area, the participant becomes the oldest participant in that area, retrieves the area information from area management information, and creates and holds a primary for the synchronized object in that area.

[0071] Figure 8 shows the configuration of replica management information 114B for non-area information in a certain room.

[0072] The non-area information replica management information 114B is created for each room, which is a virtual space, and includes a replica ID to identify the replica, an owner indicating the participant ID that owns the replica, a transmission rule for when to send the replica for synchronization, and a transfer indicating whether the ownership of the replica is transferable. Furthermore, if it is transferable, the rules for the transfer method, the data format for sending and receiving when the object is updated, or its preference may also be described. The transmission rule may be defined by the performance of the client terminal, the network status of the client terminal, the role of the user who owns the object, and the group to which it belongs ("Group α" in Figure 8), or by criteria such as being close to the object or being within the line of sight ("Within 50 Distance" in Figure 8). Alternatively, it may be defined by the performance of the client terminal and the network status of the client terminal. Participants in the room can be obtained by referring to the participant management information 113, but the non-area information replica management information 114B may also redundantly maintain a list of participants (this also needs to be updated in synchronization when the room's participant management information 113 is updated).

[0073] (Operation of the Information Processing Device) Next, the operation of this embodiment will be explained by dividing it into (1) basic operation, (2) terrain information editing operation, (3) synchronization operation, (4) joining / leaving operation, and (5) 3D data processing operation.

[0074] (1) Basic Operation Multiple users join a virtual space (room) into which a certain terrain information 111 (area) has been loaded, in order to synchronize multiple objects and share an experience (metaverse, gameplay) in the same virtual space (room, world). Note that in "(3) Synchronization Operation", for simplicity in explaining the synchronization operation of objects, the explanation will focus on the example of a metaverse that provides only a single virtual space. When the same terrain information 111 (area) is loaded into multiple virtual spaces (rooms), the synchronization operation of the terrain information 111 is determined based on the metaverse, game design and policy. As a concrete example of the synchronization operation of terrain information 111, non-area information can be synchronized for each room, and then rooms belonging to the same area can be grouped together and area information can be synchronized for each area (see Figure 16). In addition, while each participant terminal in a room retains ownership of its own avatar, avatar accessories, or associated synchronized objects, the synchronization group is the same for all synchronized objects on the participant terminals in the room. Therefore, as shown in Figure 114B, it is efficient to manage multiple objects in a single synchronization group on a room-by-room basis. Of course, it is also possible to manage them as multiple synchronization groups, or to manage them individually for each synchronized object. The same applies to terrain information within an area; as shown in Figure 114A, multiple terrain objects can be managed in a single area group on an area-by-area basis.

[0075] Users 3a, 3b, and 3c connect to the server to participate in their desired virtual space (specified area) in the metaverse by operating terminal devices 2a, 2b, and 2c. To participate, they use a unique URL provided for each area ID of the terrain information 111, directly joining a room corresponding to the terrain information 111 (area) where the desired room or event venue is located. To launch an application from a URL, particularly to activate a specific state of the application (a specific location or position within the virtual space, such as the participant's avatar sitting on a chair in the room), methods such as URL schemes, deep links, and Universal Links can be used. Therefore, after launching the application, there is no need to repeatedly perform various searches, selections, and movements within the virtual space to finally reach the desired room. Furthermore, if a participant is already in a room and wants to invite other participants to join the virtual experience together, they can simply share the URL corresponding to the room (or area, if applicable) they are in via email or message, allowing other participants to easily join the same room. Alternatively, in a metaverse such as a trade show, participating companies can communicate the URL corresponding to their exhibition area to customers through methods such as posters, brochures, business cards, email, and social media (Twitter, Facebook, etc.), allowing customers to directly participate in the invited company's room without having to make detours to other companies' rooms.

[0076] In other words, terminal devices 2a, 2b, and 2c launch a metaverse application via a browser into which a URL is entered on the terminal. This application downloads terrain data (including non-synchronization objects and programs and scripts for object operation) for the terrain information 111 (area) from the server, and then requests server 1 to join the room, initiating the synchronization process of area information and non-area information. The synchronization target information (parameters such as location) for the terrain data download is described in the data receiving means 104. Specifically, the first participant in a room belonging to an area keeps a primary copy on their own and creates a replica on the server, and subsequent participants obtain the server's replica, create a tertiary copy on their own, and initialize the parameters (synchronization target objects) necessary for synchronization.

[0077] The area (terrain information 111) corresponds to the content of the virtual space. In typical MMORPGs and other games, the client program needs to be updated when content is added or modified. However, by configuring the system to download terrain information each time, as in this embodiment, updates to the client program when content is added or modified become unnecessary. In particular, in metaverses that provide a vast amount of content (areas), it is inefficient, or in some cases impossible, to include and store all the content in the client program. On the other hand, if content is downloaded each time, users can only join a room after the content has been downloaded, which can lead to waiting times to join a room. Instead of downloading each time, it is possible to cache downloaded content and reuse the cached content if there are no content updates, or to anticipate and download content that is likely to be used next.

[0078] When the room management means 102 of server 1 receives a request from terminal devices 2a, 2b, and 2c, it refers to the room management information 112 to check if a room is available for the corresponding area ID. If a room is not available, or if a room already exists but the participant management means 103 determines that the rules for creating a new room of terrain information 111 are met (for example, if it is determined that the number of participants in the room has reached the limit), it prepares a new room and loads the terrain information 111 into the new room and provides it. If an area provides multiple rooms, the room management means 102 associates multiple room IDs with one area ID, the participant management means 103 manages the participants joining the room using participant management information 113 for each room ID, and the replica management means 106 manages the owners of the area information and replicas of the area information handled in the room using replica management information 114A for area information (area) or replica management information 114B for non-area information (room), respectively.

[0079] When the participant management means 103 of server 1 receives a request to join a room, it records the participant's user ID, username, socket ID, and date and time of joining in the participant management information 113 associated with the room ID in the storage unit 11.

[0080] The form of joining a room can vary depending on the design of the metaverse and terrain information 111. This embodiment assumes a form in which users specify a URL corresponding to the area or room they wish to join and directly join a room in a specific area (terrain information), or a specific room that already exists. However, this embodiment can be applied to other forms as well. If only one room is provided for the entire virtual space (single terrain information and single room), all participating users (e.g., those who log in) will join the default room. If multiple rooms are provided for the entire virtual space, participating users may enter a special room called a lobby (there may be multiple lobbies if there is a limit on the number of people) which acts as a waiting area, and then move to their desired room. Also, if multiple rooms are provided for a single area (terrain information) (parallel world), users may return to a room they previously participated in. When a room is newly created, the metaverse may start with one person, or it may start at an appropriate time after several participants have gathered. Also, if the metaverse has already started in a room, participating users will be treated as latecomers (see "(3) Late-Game Participation / Exit Operation" below). This embodiment does not particularly limit the method of joining a room. If multiple rooms are provided, they will be managed using room management information 112.

[0081] Terminal devices 2a, 2b, and 2c process terrain information 111 and multiple objects (characters, weapons, items, etc.) sequentially by generating, updating, and deleting them in conjunction with program execution during activity in the metaverse and games. Server 1 performs relay operations for the synchronization of terrain information 111 and objects (hereinafter referred to as "synchronization target objects") in terminal devices 2a, 2b, and 2c. Below, first, in "(2) Terrain Information Editing Operation," the editing operation of terrain information 111 will be explained, and then in "(3) Synchronization Operation," the synchronization operation will be explained, for example, in the case where, during activity in the metaverse, after all terrain information 111 and objects in terminal devices 2a, 2b, and 2c have been synchronized, a new synchronization target object is generated in terminal device 2a.

[0082] (2) Terrain Information Editing Operation First, user 3a operates the terminal device 2a to edit the terrain information 111, launches a web browser, connects to the site for terrain information editing, and logs in. In addition, in order to specify the area ID to be edited, as shown in the terrain information 111 in Figure 4, the URL associated with the area ID is entered into the site in the web browser. However, the user who can edit the terrain information 111 (the user with editing rights) is assumed to be the owner of the corresponding area ID in the replica management information 114A as shown in Figure 7, but editing rights may be set for a user other than the owner of the area ID.

[0083] When the terrain information editing means 101 of server 1 receives access from terminal device 2a via the above URL, it identifies the area ID from the string contained in the URL, compares it with the user ID of user 3a, confirms in replica management information 114A whether the user is the owner, and if the user is the owner, controls the display of a screen on terminal device 2a for editing terrain data corresponding to that area ID.

[0084] User 3a edits programs, etc., for providing the ground, buildings, rooms, and associated objects, as well as services, while checking the screen displayed on terminal device 2a. The edited content is transmitted from terminal device 2a to server 1, and if the ground, buildings, rooms, and associated objects are edited, the terrain data of terrain information 111 is updated by terrain information editing means 101.

[0085] Normally, editing of terrain data in terrain information 111 is performed when terrain information 111 is not loaded (i.e., the room does not exist). However, if the metaverse, game design, and policies allow it, editing of terrain data in terrain information 111 may also be performed when terrain information 111 is loaded (i.e., the room exists). In this case, the difference between the terrain data before editing and the terrain data after editing is treated as a difference in the synchronized object as described in "(3) Synchronization Operation". Similarly, if the terrain information is changed due to the progress of the metaverse or game, the difference caused by that change needs to be reflected in the terrain information 111 being edited, and is therefore treated as a difference in the synchronized object as described in "(3) Synchronization Operation". Note that editing of terrain information 111 refers to editing the primary copy as described in "(3) Synchronization Operation".

[0086] (3) Synchronization Operation Diagram 9 is a flowchart illustrating an example of non-area information synchronization. Non-area information synchronization is achieved by the terminal device sending changes to the object it owns (primary copy) to Server 1 while referring to the replica management information 114B of the non-area information, Server 1 updating the corresponding replica and sending update information to terminal devices that do not own the object corresponding to the destination rule, referring to the room participant list, and each terminal device that receives the update information updates the object it does not own (tertiary). Area information synchronization is achieved in much the same way. However, the scope of object synchronization (difference between room and area) and the method of selecting the object owner (the area owner is not always online) are different. The participant list subject to synchronization is not necessarily limited to participants in the same room, and if there are multiple rooms in the same area, it will be the participants of all rooms corresponding to the same area. Therefore, the owner of the replica of area information is not necessarily limited to participants in the same room. The original owner of the object should be the area owner, but they are not always online. If the area owner is not participating in the room for that area, another participant must be designated as the owner. Therefore, the owner may be a special user provided by the system (such as a dummy client) or the participant who has been participating in the room for that area the longest (the participant with the longest continuous participation time in the room corresponding to the area; if there are multiple rooms, this may also be a participant in another room). If the area owner returns to the room for that area, ownership may be transferred to the area owner using the same procedure as for early departure, as described below.

[0087] Figure 16 shows the relationship between the synchronization process of area information (replica management information 114A) and the synchronization process of non-area information (replica management information 114B) when an area has multiple rooms. The basic synchronization mechanism (primary copy, replica, tertiary) is the same for all of them.

[0088] In Figure 16, the relationship between areas and rooms is an inclusion relationship, where a client to be synchronized within an area belongs to multiple rooms, each consisting of multiple terminal devices (clients) that operate in synchronization. However, the synchronization operation in this embodiment is not limited to inclusion relationships, as a synchronization group is defined for each object. Therefore, more generally, a client can have multiple objects to be synchronized, and each object to be synchronized can have a different owner and synchronization group, allowing for a more flexible configuration. Specifically, synchronization groups can be defined for each object based on party (group), distance, purpose, etc., and these groups may be independent of each other or overlap.

[0089] For example, when synchronization is performed on a room-by-room basis, the synchronization group will consist of clients that synchronize user information among the synchronized objects. Similarly, when synchronization is performed on an area-by-area basis, the synchronization group will consist of clients that synchronize terrain information for reproducing the content that constitutes the virtual space among the synchronized objects.

[0090] First, terminal device 2a generates object A0 as a new object to be synchronized as the metaverse and game progress (S120a). Next, terminal device 2a generates a primary copy A0 of object A0 (S121a) and sends it to server 1. The reason for generating a primary copy is that object A0 changes over time, so a primary copy is generated as data representing the state of the object at a certain timing (a predetermined synchronization timing or the timing when a change occurs, etc.) and treated as synchronization information. Another reason for generating a primary copy is that in subsequent synchronizations, the information used in the previous synchronization may be necessary to check the differences due to changes over time. On the other hand, depending on the implementation, object A0 may be treated directly as synchronization information and sent to server 1 without creating a primary copy. Alternatively, especially in the case of a composite object composed of multiple objects, a combination of both may be used. In the following explanation, the primary copy and the object to be synchronized are not strictly distinguished.

[0091] The data receiving means 104 of server 1 receives a primary copy of the object as data to be synchronized from terminal device 2a via communication unit 13, duplicates it, and stores replica A0 as a secondary copy in memory 12 (S110).

[0092] When replica A0 is created, the replica management means 106 of server 1 updates the replica management information 114A or 114B and stores it in the storage unit 11. This means records policies such as the owner of replica A0 "U001" (the user of terminal device 2a), the destination rule being "other than the owner" (other than the user of terminal device 2a), and whether or not it can be transferred "OK". The policies may be fixed and defined for each type of object (for example, by default, the destination is all room participants other than the owner, and it cannot be transferred), or they may be specified when terminal device 2a creates the object, or the terminal device 2a that owns it may specify or change it as the game progresses, or the destination terminal device 2b may specify that it will not receive it temporarily for reasons such as being too far away from terminal device 2a in the virtual space.

[0093] Next, the data transmission means 105 of server 1 transmits replica A0 via the communication unit 14 to the user terminal devices 2b and 2c designated as destinations based on the replica management information 114A or 114B (referencing participant management information 113, etc., as needed).

[0094] Terminal devices 2b and 2c receive and duplicate replica A0 from server 1, and store tertiary A0 as a tertiary copy in memory (S120b, S120c). Terminal devices 2b and 2c advance the metaverse and game based on the objects synchronized by tertiary A0. If differential updates of the tertiary are performed in a later step, measures must be taken to avoid directly modifying the tertiary due to the progress of the metaverse and game. If the contents of the tertiary are to be changed in order to predict movement to reduce interaction and lag in response to the progress of the metaverse and game, a copy of the tertiary is used in advance for operations related to the progress of the metaverse and game, and then, when differential update information is sent from server 1 later, the copy of the tertiary used for metaverse and game progress is replaced with the tertiary with the difference applied and updated, and synchronized. If instantaneous replacement results in large differences or an unduely slow transition, the user experience may be improved by gradually and smoothly matching the field values ​​of the tertiary copy to the updated tertiary field values ​​over time.

[0095] Next, terminal device 2a further modifies the field values ​​of object A0 as the metaverse and game progress to create object A1 (S122a). Next, terminal device 2a updates the primary copy to correspond to object A1 (primary copy A1) and generates a difference A01 between it and primary copy A0 corresponding to object A0 (S123a), and sends the updated field information as difference A01 to server 1. Preferably, of all fields of the synchronized object, only fields where the difference since the last transmission of synchronization information is sufficiently large to exceed a predetermined threshold according to the metaverse and game content, and where updating is necessary to maintain the synchronization state, are sent as a list of field name and field value pairs, with only the number of fields requiring updating being sent. If there are zero fields, they are not sent, or zero updates are sent. In addition, if the field names are long, data serialization techniques that compress the data size by coding the field names with numbers may be used.

[0096] The data receiving means 104 of server 1 receives the differential A01 from terminal device 2a via communication unit 13 (S111), and the data updating means 107 updates replica A0 in memory 12 to replica A1 (S112).

[0097] Next, the data transmission means 105 of server 1 transmits the difference A01 of replica A0 to terminal devices 2b and 2c, which have been designated as destinations based on the replica management information 114A or 114B, via the communication unit 14.

[0098] Terminal devices 2b and 2c receive the difference A01 from server 1 (S121b, S121c), update the tertiary A0 in memory to tertiary A1 (S122b, S122c), and proceed with the metaverse and game based on the object synchronized by tertiary A1. Terminal devices 2a, 2b, and 2c each assume that the primary copy and tertiary copy stored locally represent the latest state of the object to be synchronized, and proceed with the metaverse and game independently while processing user operations and performing physical simulations. As a result, if a change occurs in each primary copy, they perform the processing corresponding to S422a and S423a and repeatedly send the difference to server 1.

[0099] The above description explains the case where terminal device 2a, which has the ownership user ID "U001", creates or modifies object A0. However, it can also delete the object. If it is deleted, it notifies server 1 of the deletion, and server 1 notifies the other terminal devices 2b and 2c of the deletion. Furthermore, the above description explains the case where terminal device 2a, which has the ownership user ID "U001", creates only one object A0. However, there is no limit to the number of objects that each of the terminal devices 2a, 2b, and 2c can create. Each device may create two or more objects, or it may create none at all.

[0100] Furthermore, terminal device 2a reduces communication volume by notifying server 1 if there are no changes to the object, but it may also be configured to notify even if there are no changes. Also, terminal device 2a reduces communication volume by sending differential information when there are changes to the object, but it may also be configured to notify the entire object, including fields that have not changed, if even a part of the field has changed, or periodically whether there are changes or not. Furthermore, these specifications may be switched statically or dynamically for each synchronized object to be notified.

[0101] Next, we will explain the synchronization rules when an object is updated.

[0102] Figure 10 is a flowchart illustrating an example of synchronization based on ownership.

[0103] Assuming that object A0 of terminal device 2a (S220a, S221a) is duplicated on server 1 as replica A0 (S210), and then duplicated from server 1 to terminal devices 2b and 2c as tertiary A0 (S220b, S220c), if tertiary A0 is subsequently updated on terminal device 2b as the metaverse and game progress, and changes to tertiary A0' (S221b), the ownership of object A0, which is the original of tertiary A0', is defined as belonging to the user of terminal device 2a. Therefore, the difference A00' (S222b) between tertiary A0' and tertiary A0 updated by terminal device 2b does not update replica A0 on server 1, object A0 on other terminal devices 2a, or tertiary A0 on terminal device 2c.

[0104] Thus, the prohibition of synchronization by tertiary A0' is achieved by any of the following: (a) Terminal device 2b not sending the difference A00' between tertiary A0 and tertiary A0' to server 1; (b) Server 1 not performing synchronization processing even if it receives the difference A00' (S211) (not performing update processing on the replica, nor sending update information processing) (ignoring, discarding); (c) Server 1 not updating its own replica A0 by applying the difference A00' to it and not sending it to terminal device 2a (and terminal device 2c); (d) Terminal device 2a not receiving the difference A00' (even if it receives it, it does not perform reception processing, but ignores, discards, or rejects it); or (e) Terminal device 2a not performing synchronization processing even if it receives the difference A00' (not performing update processing on either the primary copy or the tertiary). The prohibition of synchronization based on object updates on terminal devices other than those of the user who owns the object can be implemented by any of the methods A through E, or any combination of one or more of them. However, using method A results in the least amount of information being transmitted and received via network communication, and the amount of information transmitted and received via network communication increases in the order of B, C, D, and E, so the configurations are preferable in the order of A through E. Also, since server verification is usually performed to prevent cheating in games, the combination of A and B, or the combination of A, B, and E, is the preferable minimum configuration.

[0105] (4) Joining and leaving midway Next, we will explain the case when terminal device 2a joins midway. We assume that terminal device 2b has created object B0 and the user of terminal device 2b owns the object B0, and that terminal device 2c has created object C0 and the user of terminal device 2c owns the object C0.

[0106] Figure 11 is a flowchart illustrating an example of synchronization during a mid-game participation.

[0107] First, terminal device 2a joins the room. Server 1 adds the information of user U001 associated with terminal device 2a to participant management information 113. Also, terminal device 2a, upon joining midway, generates a new object A0, for example, a new character (S320a). Next, terminal device 2a generates a primary copy of object A0 (S321a) and sends it to server 1.

[0108] The data receiving means 104 of server 1 receives a primary copy of the object as data to be synchronized from terminal device 2a via communication unit 13, duplicates it, and stores replica A0 as a secondary copy in memory 12 (S310).

[0109] When replica A0 is created, the replica management means 106 of server 1 records the owner of replica A0 "U001" (the user of terminal device 2a), the destination rule "other than the user of terminal device 2a", and the transfer permission "OK", and updates the replica management information 114A or 114B and stores it in the storage unit 11.

[0110] Next, the data transmission means 105 of server 1 transmits replica A0 to terminal devices 2b and 2c, which have been designated as destinations based on the replica management information 114A or 114B, via the communication unit 14.

[0111] Terminal devices 2b and 2c receive and duplicate replica A0 from server 1, and store the tertiary copy of A0 in memory (S320b, S320c).

[0112] Furthermore, since the data transmission means 105 of server 1 pre-manages replicas of objects owned by users of other terminal devices 2b and 2c (S311), it transmits replicas B0 and C0 to terminal device 2a designated as the destination based on replica management information 114A or 114B via the communication unit 14. Note that this transmission may occur before the synchronization operation of object A0.

[0113] Terminal device 2a receives and duplicates replicas B0 and C0 from server 1, and stores tertiary copies of B0 and C0 in memory (S322a). Through this series of operations, terminal device 2a completes its participation by communicating only with server 1 without communicating with terminal devices 2b and 2c. In this case, there are two terminals that have already participated, but even if there are already 1,000 terminals that have participated, terminal device 2a can complete its participation by communicating only with server 1. The 1,000 terminals that have already participated do not need to notify terminal device 2a of the synchronization target objects they each own, and the load on the network and each terminal device caused by terminal device 2a joining midway is small.

[0114] Next, we will explain the case when terminal device 2a leaves the terminal. Here, we will explain the cases in which objects retain ownership and when ownership is transferred to terminal device 2a, specifically when the objects retain ownership and when the objects are deleted. Although not shown in the diagram, in the area information synchronization process, when the area owner joins (returns to) one of the area rooms, the same method is used when transferring ownership of each area information object to the area owner. In that case, the trigger is not the departure of terminal device 2a that holds ownership, but rather the area owner terminal joining the area room. Server 1 instructs terminal device 2a to discard the objects it has transferred ownership of / owns, and it performs actions similar to those shown in Figures 12 and 13, respectively.

[0115] Figure 12 is a flowchart illustrating an example of synchronous operation during early exit.

[0116] Terminal device 2a generates object A0 (S420a), updates it to object A1 (S422a), and exits (S424a) after tertiary A0 and A1 are synchronized to terminal devices 2b and 2c respectively (S422b, S422c). First, as shown in Figure 4, we will explain the case where the transfer item of replica ID "A" in replica management information 114A or 114B is "OK". Here, examples of objects for which the transfer item is "OK" include items used in the game, but they may also be characters (NPC: Non-Playing Character) or enemies.

[0117] Terminal device 2a requests server 1 to exit. Server 1's participant management means 103 deletes the user information associated with terminal device 2a from participant management information 113. Furthermore, Server 1's replica management means 106 checks the transfer item in replica management information 114A or 114B, and if the transfer item is "OK", it changes the next owner to the user of terminal device 2b and updates the contents of replica management information 114A or 114B (S413), and also notifies terminal device 2b that ownership has been changed.

[0118] When terminal device 2b receives notification from server 1 that ownership has been transferred to terminal device 2b, it completes the exit operation by changing the stored tertiary A1 to object A1 owned by itself (S423b). Here, the exit (S424a) is explicitly notified to server 1 by terminal device 2a, but there may be cases where the exit cannot be notified due to a sudden line loss, etc. Even in such cases, if server 1 detects that terminal device 2a has disconnected (so socket closed, etc.), it can complete the exit and ownership change process in the same way as when an exit notification has been explicitly received. If each terminal device strictly manages the ownership of tertiaries, although not shown in the diagram, server 1 will also notify all remaining terminal devices (in this case, terminal device 2c) participating in the synchronization operation other than terminal device 2b, which has been transferred ownership and will subsequently manage the tertiary as primary, that ownership of object A (tertiary A) has been transferred from terminal device 2a to terminal device 2b. If each terminal device only manages whether the synchronized object it manages is a primary object it owns or a tertiary object it does not own (i.e., it does not manage the owner of the tertiary object), then notification to terminal devices that do not own the object (2c) is not necessary. However, in that case, server 1 should only send update information to terminal devices that do not own the object when it receives change information that a terminal device that owns the object has changed the primary object. If server 1 receives change information that a terminal device that does not own the object has changed the tertiary object, it should not send the update information to the terminal device that does not own the object. The former allows terminal devices to strictly manage the owner of the tertiary object and makes it easier to deal with abnormal update information such as cheats, but the amount of communication due to early exits increases. The latter relies on server 1 to handle abnormal update information such as cheats, but the amount of communication due to early exits is extremely small and is effective when a huge number of terminals are participating in the synchronization operation. Depending on the type of object to be synchronized and the performance of the terminal devices, the former and the latter may be selected on a case-by-case basis.

[0119] If the transfer item is [OK], you may pre-determine which user to transfer it to, determine it upon departure, transfer it in order of joining date and time, assign it randomly, assign it to a user with a high-performance terminal, or assign it to a user with a good network environment (high bandwidth, low latency / RTT (round-trip time), and stable communication (stable and low RTT)).

[0120] Next, we will explain the case where the transfer item for replica ID "A" in replica management information 114A or 114B is "NG". Here, examples of objects whose transfer item is "NG" include characters played in the game (the user's own avatar) and enemies, but they may also be items used in the game.

[0121] Figure 13 is a flowchart illustrating another example of synchronous operation during early exit.

[0122] Terminal device 2a requests server 1 to exit (S524a). Server 1 checks the transfer items in replica management information 114A or 114B, and if the transfer items are "NG", it deletes the information related to replica A1 and updates the contents of participant management information 113 and replica management information 114A or 114B (deletion of user information associated with terminal device 2a, deletion of synchronized objects owned by the user associated with terminal device 2a) (S513), and notifies terminal devices 2b and 2c that replica A1 has been deleted.

[0123] When terminal devices 2b and 2c receive notification from server 1 that replica A1 has been deleted, they delete the stored tertiary A1 (S523b, S523c) to complete the exit operation.

[0124] Figures 14 and 15 are flowcharts illustrating an example of how ownership of the synchronized object of the area information can be restored when an area owner (user with editing rights for area information) 2c joins (returns to) any room in the area midway through (S620c, S720c).

[0125] In the case of an object to be transferred according to replica management information 114A, as shown in Figure 14, ownership of the object is transferred from terminal device 2a to terminal device 2c, the area owner, in a procedure almost identical to that in Figure 12. The primary of terminal device 2a becomes the tertiary (for example, server 1 notifies terminal device 2a of the loss of ownership due to the transfer of ownership) (S624a), and terminal device 2c holds the primary (S621c) (for example, server 1 sends a replica to terminal device 2c and further notifies it of the acquisition of ownership due to the transfer of ownership (S614), and as a result, terminal device 2a holds the primary as the tertiary, while terminal device 2c receives the replica and holds it as the primary). Although not shown, if necessary, such as for managing the owner of the tertiary at the terminal, terminal device 2b is also notified of the change of owner. If the object is not subject to transfer according to the replica management information 114A, the replica of the object is destroyed (S714) in a procedure almost identical to that in Figure 13, as shown in Figure 15, and the corresponding primary and tertiary for participants in all rooms of the area are also destroyed (S724a, S723b).

[0126] (5) 3D data processing operation diagram 18 is a schematic diagram illustrating an example of the synchronization operation of a 3D object. Note that explanations that are common to the synchronization operation described in Figure 9 may be omitted.

[0127] First, the metaverse execution means 204 of the terminal device 2a generates a new 3D object A0 as a synchronized object as the metaverse and game progress. Next, the data transmission means 203 of the terminal device 2a generates a primary copy A0 of the object A0 and sends it to the server 1.

[0128] The data receiving means 104 of server 1 receives a primary copy of an object as data to be synchronized from terminal device 2a via communication unit 13, duplicates it, and stores replica A0 as a secondary copy in memory 12.

[0129] Next, the data transmission means 105 of server 1 transmits replica A0 via the communication unit 14 to the user terminal devices 2b and 2c designated as destinations based on the replica management information 114A or 114B (referencing participant management information 113, etc., as needed).

[0130] The data receiving means 201 of terminal devices 2b and 2c receive and duplicate replica A0 from server 1, and stores the tertiary copy of A0 in memory. The metaverse execution means 204 of terminal devices 2b and 2c advances the metaverse and game based on the objects synchronized by the tertiary A0.

[0131] Next, the metaverse execution means 204 of the terminal device 2a changes the field values ​​of object A0 to create object A1 by, for example, applying action I to object A0 as the metaverse and game progress. Next, the data transmission means 203 of the terminal device 2a updates the primary copy (primary copy A1) in accordance with the object A1, and may also generate the action I itself that was applied to object A0, or conversion data corresponding to action I (data for converting replica A0 to replica A1 at the receiving device), and send it to server 1. Of course, it may also generate a difference A01 between object A0 and primary copy A0, and send the updated field information as difference A01 to server 1.

[0132] The data receiving means 104 of server 1 receives operation I or conversion data corresponding to operation I from terminal device 2a via communication unit 13, and the data updating means 107 updates replica A1 of memory 12 by reflecting operation I in replica A0 or by reflecting the conversion data.

[0133] Next, the data transmission means 105 of server 1 transmits operation I or conversion data corresponding to operation I to terminal devices 2b and 2c designated as destinations based on replica management information 114A or 114B, via the communication unit 14.

[0134] The data receiving means 201 of terminal devices 2b and 2c receive action I or conversion data corresponding to action I from server 1, update tertiary A1 in memory tertiary A0 by reflecting action I or by reflecting the conversion data, and advance the metaverse and game based on the synchronized objects of tertiary A1.

[0135] Furthermore, in the synchronization operation of the above objects, codecs may be performed between each device as described below.

[0136] Figure 19 is a schematic diagram illustrating the codec, transmission / reception, and rendering operations of a 3D object.

[0137] First, terminal device 2a generates a 3D object A0 of basic data as a new synchronization target object as the metaverse and game progress. The basic data is represented, for example, in voxels, but may also be a height map or polygon mesh. Next, the data transmission means 203 of terminal device 2a generates a primary copy A0 of the object A0, encodes it in a format such as LZ4 compression if it is a voxel (PNG image format if it is a height map, or FBX zip compression or glb + draco compression if it is polygon data), and sends it to server 1.

[0138] The data receiving means 104 of server 1 receives a primary copy of the object encoded as data to be synchronized from terminal device 2a via communication unit 13, duplicates it, and stores replica A0 as a secondary copy in memory 12.

[0139] Furthermore, the codec means 108 of server 1 decodes replica A0 to obtain the base data for replication (replica A0), such as voxels. When server 1's codec means 108 simply functions as a relay server, it does not need to decode replica A0, but when differences or operations I are to be reflected, it is necessary to decode it into the base data. In addition, server 1's codec means 108 checks whether the content of replica A0 as base data is the same as the original data format, whether it is infected with a virus, etc.

[0140] Next, the data transmission means 105 of server 1 transmits the undecoded replica A0 via the communication unit 14 to the user terminal devices 2b and 2c designated as destinations based on the replica management information 114A or 114B (referencing participant management information 113, etc., as needed). If the format of the basic data handled by server 1 and terminal devices 2b and 2c is the same, the decoded replica A0 may also be transmitted.

[0141] The data receiving means 201 of terminal devices 2b and 2c receive and duplicate replica A0 from server 1, and stores the tertiary copy of A0 in memory. The metaverse execution means 204 of terminal devices 2b and 2c advances the metaverse and game based on the decoded object (basic data) of the tertiary A0.

[0142] Furthermore, the metaverse execution means 204 of terminal devices 2b and 2c, when advancing the metaverse and game based on the object (basic data) decoded from the tertiary A0, converts the voxel data into polygon mesh data, etc., and performs rendering while reflecting effects such as lighting and materials.

[0143] Furthermore, this operation may be applied not only to copying the object itself between devices, but also to differential data, operation I, and transformation data, or only the codec may be omitted.

[0144] (Effects of the Embodiment) According to the embodiment described above, terminal device 2a does not receive objects to be synchronized from all other terminal devices 2b, 2c, but rather server 1 stores replicas of objects generated by all terminal devices, and terminal device 2a receives objects to be synchronized from server 1. Therefore, in a configuration in which multiple terminal devices participate and data from multiple terminal devices are synchronized with each other, an increase in communication load can be suppressed. In addition, since server 1 does not run the metaverse or game, the processing load can be reduced. As a result, object synchronization of a metaverse or game on a scale of 1000 users becomes possible.

[0145] Furthermore, joining mid-process only requires receiving a replica from the server and creating a tertiary object; there is no need to receive objects from other clients, thus suppressing increases in communication and processing load.

[0146] Furthermore, in cases where there are many terrain information areas (111), such as in a metaverse, the user can specify which terrain information area to join using a URL. This allows users to join a room (a specified area) to which terrain information area belongs without being aware of the room itself, and it also makes it easier to inform and share with other users. In particular, if the program, such as a metaverse or MMO, runs on a web browser, the URL can be embedded on a web page, allowing users to join from their web browser.

[0147] Furthermore, since the editing of the terrain information 111 is also performed via a web browser using a URL, there is no need to install an editing application on terminal devices 2a, 2b, and 2c, thus reducing the steps required to start editing.

[0148] Furthermore, since it only needs to refer to the information of the synchronized objects received from the management information, and there is no need to refer to the permission list until the corresponding user is found, the processing burden is reduced.

[0149] In the above embodiment, even with fewer than 1,000 users (for example, 100 or 10 people), the processing load is reduced to one-tenth or one-hundredth compared to the case of 1,000 users. Therefore, the processing load can be reduced compared to conventional methods, and consequently, the service using this invention can be provided to users at a low cost.

[0150] For example, in the case of an object like a ball in a soccer game, where ownership of the ball is defined to the player holding it, the ownership of the ball object changes during the game's progress. However, by managing ownership for each object, the management information only needs to be updated once, eliminating the need to check and update permission lists for multiple users, thus reducing processing and communication burdens.

[0151] A further challenge of this invention is to provide a server capable of performing synchronous operations in massively multiplayer online games, regardless of the game's design.

[0152] The present invention has a configuration that can adequately withstand designs where, for example, ownership of synchronized objects changes rapidly among multiple users, or where the number of owners of synchronized objects is large, and can provide an environment that can handle games with unknown designs that are currently unforeseen.

[0153] Furthermore, in 3D object synchronization, Server 1 stores replicas of objects generated by all terminal devices, and terminal devices 2a, 2b, and 2c receive the objects to be synchronized from Server 1. Subsequently, if an object is updated, instead of receiving the difference, they receive the action I that caused the update or the conversion data corresponding to action I, and synchronize by updating the tertiary in memory to reflect action I or the conversion data. This configuration, involving multiple terminal devices and synchronizing data from multiple terminal devices, can potentially further suppress the increase in communication load. In addition, Server 1 does not perform the reflection of action I or the reflection of conversion data for the sake of metaverse and game progression, thus reducing the processing load. As a result, object synchronization for a metaverse and game on a scale of 1000 users becomes possible.

[0154] Furthermore, when Server 1 operates as a relay server, it receives encoded data from terminal devices, particularly differential data (or operation or transformation data), transmits it to other terminal devices without decoding, and then decodes the received data and renders the basic data. This separates the rendering, converter (data conversion), and transmission / reception into layers, which speeds up the processing of 3D data, which tends to be computationally intensive. Additionally, by expanding the converter of the terminal devices, it becomes possible to handle various types of 3D data.

[0155] [Other Embodiments] The present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention.

[0156] In the above-described embodiment, the configuration is such that an object updated by the user who owns it updates the replica on Server 1, and then propagates the update information to the remaining clients in the same room. Therefore, in the metaverse and game progression, inconsistency resolution will prioritize the owner client. However, if the metaverse and game design allows each client to own their own avatar, equality can be achieved. Conflicts between owner clients should be resolved by the metaverse and game design. For example, the design could be such that objects do not have direct relationships with each other, or they could have relationships through another mechanism such as RPC (Remote Procedure Call), or through another object, so that the objects being synchronized do not have direct relationships with each other.

[0157] Furthermore, ownership of synchronization objects related to area information is defined for each area, so if there are multiple rooms in an area, it applies to and is shared by all rooms in the same area. In other words, when a client with ownership updates the state of an area's synchronized object, the area information is updated not only in the rooms that the client is participating in, but also in rooms belonging to the same area that the client is not participating in. Area information is not strictly an object exclusive to a specific client, so there may be cases where a client that does not own an area wants to make an update. In order for a client that does not own an area to make an update, it will need to request the update from the client that owns (primary) the synchronized object. One example of how to make such a request is to use a program (script) such as RPC. Operations that involve changing such objects that are not owned, such as opening and closing a building door, are operated as a program of the synchronized object (in this case, the door), and when that program is operated, the operation is executed similarly on all clients that are synchronously operating that object. The operation on the client without ownership will fail, but the operation on the client with ownership will succeed, and the state of the object will change (for example, a closed door will open) on the primary copy of the client that owns it. This change is reflected in the replica on Server 1 through a mechanism for synchronizing object state changes, and subsequently reflected in the tertiaries of the remaining clients participating in each room of the same area who do not own the object. This mechanism also allows clients to manipulate objects they do not own. While this is an example of synchronizing area information across rooms within an area, the same RPC or other program / script synchronization method is not limited to areas and can also be used for other people's objects within a room that you do not own.However, in this system, if multiple clients attempt to manipulate the same object simultaneously, numerous similar RPC requests will be generated. The execution order will vary depending on the client, taking into account communication delays and luck. Furthermore, only the RPC of one client on the owning terminal will be valid, giving the owning client a significant advantage. Between clients without ownership, the outcome depends on the distance between the client and the owning client, the client's CPU performance, and the network environment to which the client is connected. Therefore, it is not necessarily fair to use this for synchronous operations in time-critical operations such as FPS games, but it is sufficient for infrequent changes like those in social content. As mentioned above, the selection of the object to be synchronized, the scope of synchronization (entire area or within a room), and the method of synchronization (direct object synchronization or indirect synchronization via a program such as RPC) greatly affect the functionality and performance of the service provided, and should be decided according to the design of the service to be implemented (game design).

[0158] Furthermore, in the above-described embodiment, the method for synchronizing the metaverse and game progress among clients is omitted in detail because existing technologies can be applied. In particular, in virtual environments where a large number of people participate from all over the world, delays or losses may occur in the transmission of information about the objects to be synchronized. Therefore, depending on the type of virtual space (metaverse, game genre) and the type of objects to be synchronized, it is necessary to use various general techniques in addition to the method proposed in this embodiment for information transmission.

[0159] First, to handle network latency, each client often synchronizes the virtual space time within the virtual space it is participating in and attaches the virtual space time to the communication information related to the object being synchronized.

[0160] To handle network latency, a common synchronization method is used that creates a slight delay in the metaverse and game progression for each client participating in the virtual space. In other words, terminal devices 2a, 2b, and 2c each progress through the virtual space independently, but the time in each virtual space they are experiencing will be slightly out of sync. If terminal devices 2a and 2b could be viewed side by side, it would appear that terminal device 2a is slightly ahead of terminal device 2b in virtual space time (a certain event occurring on terminal device 2b will also occur on terminal device 2a). This is particularly noticeable when terminal devices 2a and 2b are geographically separated, and when there is a significant difference in the distance between terminal devices 2a and 2b and server 1 (for example, if server 1, terminal device 2a, and terminal device 2b are located in Tokyo, Osaka, and New York, respectively).

[0161] Furthermore, to compensate for network loss, communication protocols may retransmit data. This retransmission further delays the transmission of information about the synchronized objects.

[0162] Furthermore, in order to reduce the frequency of transmission of information about synchronized objects, limiting the transmission frequency is also widely practiced, in addition to the threshold processing mentioned above. When the progress of the virtual space is managed by frames (sampling the metaverse and game state at appropriate time intervals and synchronizing at the sampling frequency), the primary copy is updated by the frame rate (e.g., 20 times per second). This may be combined with update control by threshold. Frame management is the synchronization of each terminal device at each frame time common to the virtual space. More specifically, the primary copy is updated at each client at a specific common virtual space time within the virtual space (considering threshold and other conditions as needed), and at the time of the update, the frame time (virtual space time or frame serial number) of that frame is attached and sent to server 1 as update information (differential or full). On the other hand, each terminal device receives update information of the primary copy of other terminals along with the frame time at the time of each update. The frame time attached to the update information of other terminals received by the terminal device will match the frame time of the latest primary copy in the absence of delay. In situations with latency, the frame time attached to update information received by terminal devices will differ depending on the terminal device, due to various reasons such as the terminal device's location, network environment, or temporary changes in communication status. Generally, this frame time will be older than the primary copy's frame time. Simply running the metaverse or game based on primary and tertiary copies with different frame times can lead to significant synchronization discrepancies between terminals. In such cases, for example, the frame time difference received from each terminal can be measured in advance, and the metaverse or game can be run to match the frame time of the terminal that sends the slowest frame time on average. In this case, several frames of update information are buffered on the terminal device, and those with the same frame time are combined to strictly run the metaverse or game. However, if the lag exceeds 0.1 seconds (a delay of 2 frames or more in the case of 20 frames per second), the user experience will deteriorate, depending on the metaverse or game genre.I won't go into further detail, but various ingenious solutions and compromises are used to adapt the approach to the metaverse and game genre. On mobile devices, frame delays change dynamically and significantly, so different approaches are necessary.

[0163] Alternatively, due to the characteristics of the communication channel, virtual space time information within the virtual space may be attached. When using reliable communication protocols such as TCP / IP or RUDP (Reliable UDP), the virtual space time order of the received shared object update information will not be altered. On the other hand, if the entire synchronized object is sent each time, and a protocol such as UDP, which does not guarantee reliability, is used to anticipate communication failures and transmit the entire synchronized object as update information, then some update information may be lost, or the arrival of update information may not be in the order of the virtual space time attached to the primary copy, but rather out of order. In such cases, the terminal device uses the virtual space time attached to the update information to rearrange the received information, or if old update information is received, it discards unnecessary old information to synchronize the objects and allow the metaverse and game to progress.

[0164] Furthermore, the participant management information 113 may manage links to participant objects as a list in the order of participation. The server 1 may obtain the information necessary for corresponding communication from the participant information of each link and communicate with the terminal devices 2a, 2b, and 2c.

[0165] In the above embodiment, the functions of each means 100 to 104 of the control unit 10 were implemented by program, but all or part of each means may be implemented by hardware such as ASIC. Furthermore, the program used in the above embodiment can be stored and provided on a recording medium such as a CD-ROM. Also, the steps described in the above embodiment can be rearranged, deleted, or added without altering the gist of the above embodiment. Additionally, the functions of each means may be combined with other means as appropriate, or separated into multiple means.

[0166] [Another Embodiment 2] Groups of terminal devices that synchronize with each other can be flexibly specified. In the embodiment and the other embodiment, the synchronization operation of area information of an area and the synchronization operation of non-area information of a room have been described, but groups of terminal devices that synchronize with each other (hereinafter sometimes simply referred to as "synchronization groups") are not limited to areas (participant terminals of all rooms in an area) or rooms (participant terminals of a room). Here, a room synchronization group consists of one or more clients per room that synchronize user information among the synchronized objects (Figure 8, 114B shows the case of one), and an area synchronization group consists of one or more clients per area that synchronize the terrain information (area) for reproducing the content that constitutes the virtual space among the synchronized objects (Figure 7, 114A shows the case of one). Furthermore, each synchronization group may be dynamically created, modified, and deleted, as explained using the room as an example. Group management is managed and maintained as group management information, similar to room management information and terrain management information. Terminal devices can join and leave one or more groups as appropriate, according to the policy of each group. For example, if you choose a specific area to experience a virtual reality session, you will join the area's synchronization group and the room's synchronization group. When you leave the room, you will also leave both the area's and the room's synchronization groups.

[0167] As another embodiment (Embodiment 2), a more general-purpose method for configuring synchronization is shown. Here, the basis of the synchronization operation is a group of terminal devices for synchronization (synchronization group) consisting of the object to be synchronized and the terminal devices that participate in the synchronization operation of the object. Depending on the configuration method of the object to be synchronized and the configuration method of the terminal device group to be synchronized, it is possible to accommodate various forms of synchronization use and improve the efficiency of the synchronization operation.

[0168] (1) Basic Synchronization Operation Synchronization operation of an object is achieved when the owner terminal and the non-owner terminals that synchronize the target object connect and communicate with Server 1 for each synchronized object. In this case, the terminal devices that synchronize form a synchronization group. This terminal device group (synchronization group) is all or some of the terminal devices connected to Server 1 in order to participate in the virtual space.

[0169] The synchronization operation within a synchronization group for each synchronized object is as described in Figures 9-13. The terminal device of the participant who owns the object holds the primary of the object, Server 1 holds the replica of the object, and the terminal devices of the remaining participants in the synchronization group who do not own the object hold the tertiary of the object. Changes in the primary are sequentially applied to the replica and tertiary, so that the primary, replica and tertiary are synchronized to the same state. Terminal devices of participants who do not own the object must not directly change the tertiary, and even if they do change the tertiary, the replica, primary, and the tertiary of other terminals will not be changed. A terminal device of a participant who does not own the object and wants to change the tertiary should not directly change the tertiary, but instead request the terminal device of the participant who owns the object to change the primary. If the terminal device of the participant who owns the object updates the primary in response to the request, the update information will propagate to the replica and tertiary, and as a result, the tertiary of the terminal on the participant's own terminal will also be changed.

[0170] Synchronization is managed and operated by a replica management device for each object being synchronized. As shown in Figures 7 and 8, Server 1 maintains, for each replica being synchronized, the replica ID, the terminal device group for synchronization (synchronization group) (owner, participants without ownership, and destination restriction rules if necessary), and transfer information. Figures 7 and 8 represent cases where area participants and room participants constitute the terminal device group for synchronization, respectively. Participants without ownership are the participants in the participant list excluding the owner. Figures 7 and 8 show how multiple objects being synchronized that share the same terminal device in the synchronization group are managed together on an area or room basis, but the same synchronization operation can be achieved even if the objects are managed individually. However, it is advantageous to manage objects being synchronized that share the same terminal device in the synchronization group together as shown in Figures 7 and 8, as this reduces the number of active synchronization groups.

[0171] (2) Configuration of objects to be synchronized The objects to be synchronized may be objects that are recognized as objects in the virtual space, but it is desirable to reduce the load of the synchronization operation by reducing the number and content of the objects to be synchronized, such as by limiting it to the object parameters such as position and orientation (parts of the object's fields or their encoded versions) that are necessary to reproduce the synchronization state of the object, rather than the object itself, or by forming a combined object that groups together multiple objects that behave similarly.

[0172] In metaverses and MMORPGs, objects corresponding to avatars and their accessories (clothing and equipment), and the content of the experience (buildings, terrain), change in state in response to user actions and the passage of time. Virtual experiences are shared by other participants experiencing the same content by sharing (synchronizing) the changes in these objects. While avatars, clothing, building windows and doors are examples of objects to be synchronized, synchronizing each object individually would result in an enormous number of objects, increasing the burden of managing replicas (number, size, communication volume, etc.). One way to reduce the burden of the number of objects to be synchronized is to design the system so that only some of the parameters that make up each object are synchronized. In other words, even when synchronizing an avatar, it is sufficient to synchronize only parameter information such as position and orientation, rather than synchronizing the unchanging shape and details of the avatar. Furthermore, if multiple objects to be synchronized behave semantically in the same way, it is best to create a single synchronized object consisting only of the parameters to be synchronized from the parameters of each of the multiple objects (further encoding such as merging and compression if necessary), manage it, and synchronize its operation. For example, instead of managing the avatar itself and its accessories as separate synchronized objects, it is preferable to synchronize them as a single group of objects, as a special synchronized object that manages parameters. For user avatars and avatar accessories, the information should be treated as non-area information, and for information regarding the state and shape of buildings and the ground surface in an area, the information should be treated as area information. For each user (avatar) and each area, only the parameters that manage their respective states should be collected and managed as a single (or small number of) special synchronized object by replica management information (replica management information for the room 114b if the synchronization group is a room, and replica management information for the area 114a if the synchronization group is an area), and these should be the target of the synchronization operation of individual non-area information and area information.

[0173] Furthermore, when designing objects to be synchronized, careful consideration must be given to the method of synchronization, such as whether to synchronize the object's fields (values, parameters) (synchronize them as objects) or to synchronize them as program operation events (triggers). Values ​​and parameters are synchronized as objects as described above, but events are not synchronized as object values. Instead, trigger messages are sent to the synchronization group, including the terminal device itself, and these triggers are executed almost simultaneously and almost identically by the programs (scripts) of each terminal device (RPC, Remote Procedure Call). This ensures that each terminal shares almost identical experiences as a result of executing the client program. Here, "almost simultaneously" and "almost identically" means that the execution results may not always be perfectly identical due to differences in execution between terminals or subtle differences in the internal state of the terminals.

[0174] For example, complex animations such as avatar jumps are not triggered by sequentially synchronized operation of changes in position and posture parameters, but rather by event reception, such as the animation playback process for jumps, and are executed independently on each terminal device. If the initial virtual space environment before program execution is nearly the same on all terminal devices, nearly the same animation will be executed, effectively enabling shared experiences in the virtual space. If the program result affects the position or posture of an object, a mechanism for synchronized object operation is used in accordance with the program execution result. The program execution result on the terminal device that owns the object being changed becomes the change to the primary object, and the replica and tertiary objects are updated to reflect this result. Program execution on terminal devices that do not own the object does not directly change the tertiary object, but since the primary object is changed by the execution of a similar program on the terminal device that owns the object, that change is reflected in the replica and tertiary objects, and as a result the tertiary object is also changed.

[0175] For example, as an example of area information, if a terminal device that does not own a door executes a program to open and close a door, the program's result on that terminal device will fail and will not affect the door. However, the same program will also be executed on a terminal device that owns the door, and as a result of that program's execution, the primary corresponding to the door will be updated. This update to the primary will then propagate to the replica and tertiary, and the tertiary will also be changed on the terminal device that attempted to open or close the door, and as a result, the result of the door opening and closing operation will be reflected. Although not described in detail here, these programs (scripts) themselves are not subject to synchronization, but they are part of the non-area information and area information, respectively, and are loaded onto the client terminal as part of the non-area information and area information that is not subject to asynchronous operation when the corresponding user and terrain are represented in the virtual space.

[0176] Depending on the nature and functionality of the object to be implemented, it is effective to use either a synchronization method based on object parameter values ​​or a synchronization method based on the execution results of a program (RPC). This embodiment discloses the basic operation of the object synchronization method, and further discloses that program synchronization (RPC) utilizes existing RPC mechanisms (various other methods are known, such as a combination of simple messaging and message handlers), and also discloses the synchronization method of objects updated as a result of program execution by combining the basic synchronization operation with the RPC mechanism. In this embodiment, program (script) synchronization (RPC) itself will not be described in detail. However, when applying the synchronization operation of this embodiment together with the RPC synchronization method, if an object owned by a user without ownership changes as a result of a program launched by that user, it is often advisable for the owned object to verify the validity and appropriateness of the program execution. In addition to verifying operation rights, it is also possible to perform mutual exclusion when multiple terminals attempt to operate a single object simultaneously.

[0177] Similarly, when multiple clients own a single object to be synchronized within a synchronization group, a synchronization method based on program execution results is necessary to synchronize each client's primary ownership.

[0178] Area information and non-area information are content within the virtual space and can be created, edited, and deleted in ways separate from operations within the virtual space (game operations). These contents are assigned a unique ID, and it would be more convenient if one of the unique IDs could also be specified using a URL or similar method.

[0179] For example, to participate in a virtual space, users often need to register. During this process, they create and edit non-area information (content) within the virtual space, such as their name and avatar.

[0180] Furthermore, when adding new user experience content (areas) in a virtual space, you will need to create and edit the space that provides the user experience, including its name and area information, for the new virtual space.

[0181] These various content editing operations are performed using 3D editors and other tools. By being able to access area-specific and non-area-specific content via URLs, it becomes possible to directly edit the content using the 3D editor. This offers improved immediacy and convenience compared to the method of downloading or uploading content for each individual component object and then editing it in the 3D editor.

[0182] In a 3D editor, you can create intricate objects by combining basic units such as points and lines, and then combine and place them. Alternatively, you can place basic objects provided by the editor, such as ground, houses, and mountains, and customize their characteristics and attributes (such as whether the ground is grass or concrete, or specifying the height, size, and number of floors of a house). Or, you can place and customize objects such as desks, benches, and stars in template spaces such as living rooms, showrooms, parks, and outer space.

[0183] In metaverses and MMGOs, the ability to create and edit non-area and area information is often provided as part of the virtual space application's functionality, or data created with general-purpose 3D editing applications is uploaded and registered in the virtual space. However, in this embodiment, non-area and area information is managed on the server as a general rule. It is downloaded as needed when the program is executed on the terminal device. In configurations that perform synchronization operations (primary, replica, tertiary) of synchronized information, a mechanism is provided that allows content such as non-area and area information existing on the server to be directly specified, making it easy to share various types of content between users or between multiple terminals of the same user. In this embodiment, instead of internal application representations such as terrain IDs and character IDs, a more general definition such as a URL is used. For example, techniques such as URL schemes, deep links, and Universal Links are used, and by adjusting parameters according to the target content and access method, it is possible to directly edit the target content or access it in the desired mode (program), such as participating in the content, without taking detours. Furthermore, by making the content accessible via a general-purpose URL, its use is not limited to this service, but can be shared and reused in a more open environment.

[0184] For example, to join an area A100, you would use "https: / / sample.metaverse.jp / ?area_id=A100&action=join", or to join a specific room R101 within an area A100, you would use "https: / / sample.metaverse.jp / ?area_id=A100&room_id=R101" or "https: / / sample.metaverse.jp / ?area_id=A100&room_id=R101&action=join", specifying the management site sample.metaverse.jp, the area ID A100, and if necessary, the room ID R101, and the access mode join.

[0185] Furthermore, to edit the area information for a specific area, A101, you can specify the management site sample.metaverse.jp, the area ID A100, and the access mode edit, like this: "https: / / sample.metaverse.jp / ?area_id=A100&action=edit".

[0186] Furthermore, to edit non-area information owned by a user U501 (for example, a "home" area equivalent to a web homepage), the access mode should be set to edit, like this: "https: / / sample.metaverse.jp / ?user_id=U501&action=edit", specifying the management site sample.metaverse.jp, user ID U501, and access mode edit. Depending on the URL, a different application will be launched as needed, and after entering access management information (such as user ID and password) and any missing parameters as necessary, the user can directly access the specified content in the specified mode using the appropriate program (game program, metaverse program, WebGL, 3D editor) that has been launched.

[0187] (3) Method of configuring synchronization groups It is desirable that the terminal devices to be used for synchronized operation be limited to synchronized operation only among groups where the operation of the synchronization object is meaningful for sharing a virtual experience.

[0188] When the number of terminal devices to be synchronized becomes enormous, the processing and communication volume for synchronization on Server 1 and the terminal devices becomes enormous, leading to high CPU load and exceeding the communication bandwidth limits. While Server 1 can handle the increase in the number of devices by distributing and scaling the processing related to the replicas of the objects to be synchronized using methods such as the cloud, the communication bandwidth will reach its limits due to the increased communication volume on the terminal devices. In particular, terminal devices cannot always be expected to have sufficient performance in terms of CPU performance and communication speed. Especially when used in wireless environments such as mobile devices, communication speeds may not be stable, and for terminals with metered communication contracts, an increase in communication volume may not be welcome.

[0189] To provide a shared experience in a virtual space, it would be ideal if all objects within the virtual space could be shared by all participants. However, as the number of participants increases, this becomes economically difficult or technically impossible. Furthermore, it is impossible or meaningless for users to actually understand the behavior of all participants. To solve this problem, it is necessary to reduce the number of objects that need to be synchronized, or to reduce the scope (number of users) over which objects are synchronized. An example of the former is explained in the previous section on the configuration of synchronized objects (for example, grouping objects with similar behavior together and creating only one special synchronized object by extracting only the parameters (fields, etc.) that require synchronization from those objects).

[0190] To reduce the scope (number of users) of synchronized object operation, the number of terminals that synchronize each object is limited to the bare minimum necessary. In most cases, user terminals do not represent all of the objects that make up the virtual space, but only a small portion. Therefore, if synchronized operation is achieved for only a small portion of the represented objects, the virtual space experience perceived by the terminal user will be the same whether or not the other objects are internally synchronized.

[0191] One can define a terminal group for each object to perform synchronized operations, and synchronized operations should only be performed within that group. Furthermore, subgroups can be created within the synchronization group using the destination rules of the replica management information 114. In typical virtual spaces and MMOs, each virtual space is divided into multiple areas (areas where virtual space content can be experienced), and the user's terminal device participates in a room to experience the virtual space corresponding to that area. Terminal devices participating in the same room, as avatars, virtually experience the content provided by the area (buildings, attractions, etc.) together with the avatars of other terminal devices, according to the content of the area. In this environment, avatars and their attachments created by participants are synchronized objects owned by the participant who created them, have a primary on their own terminal device, and can inform other participants of their activity status by performing synchronization processing with the terminal devices of other participants participating in the room. For the sake of simplicity, the objects to be synchronized here are listed as avatars and their accessories. However, in actual implementation, it is more efficient to aggregate the information (non-area information) of multiple user-managed objects consisting of avatars and their accessories into the minimum number of parameters such as position and orientation, group the avatars and their accessories for each user, and implement them as a single special object for user information management (synchronized non-area information), then perform the synchronization process.

[0192] In order to share the activity status of other participants within a specific area (room) of the virtual space, each participant owns ownership of their own synchronized non-area, and each participant's terminal device holds the primary for their own synchronized non-area and the tertiary for the synchronized user management information of other participants.

[0193] Similarly, it is possible to share changes in the state (area information) of content (areas) in a specific region (area) within a virtual space. If multiple rooms coexist in an area, and a participant in one room changes the area information, the change can be reflected not only in the area information of other participants in the same room, but also in the area information of other rooms (parallel worlds) in the same area, even if they are in different rooms. To share changes across multiple rooms, the terminal group to be synchronized should consist of participant terminals for all rooms in that area.

[0194] For example, consider a case where the area creator is also a participant in one of the area's rooms. The area creator is the owner of the area's area information. Regarding the synchronization of the area's area information, instead of directly synchronizing individual terrain objects, the area information to be synchronized, which is part of the fields of multiple terrain objects to be synchronized, is grouped together as a special single object. If the area creator's terminal is a participant in one of the area's rooms, it will hold the primary of its own synchronized non-area information, the tertiary of the synchronized non-area information of other participants in the same room, and the primary of the area's synchronized area information. In this case, the area creator's terminal is the creator of the area, the owner of the area, and also the owner (primary holder) of the synchronized object related to the area in the synchronization operation. Participant terminals in the area's room who are not area participants will hold the primary of their own synchronized non-area information, the tertiary of the synchronized non-area information of other participants in the same room, and the tertiary of the area's synchronized area information, regardless of whether they are in the same room as the area participants or in a different room.

[0195] Depending on the type of virtual space, it may not always be practical to assume that the area creator is always involved. Here, we assume that the area creator is both the owner and the editor. However, for example, in a virtual space used for company promotion, the area creator might be a content creator outsourced by the company, the owner might be the company president, and the editors might be the president or other employees in the public relations department. Here, we will describe the case where the area creator is also the area editor, but as in the example of the company promotional virtual space mentioned above, it may be appropriate to interpret the area creator as the area editor in the following description, as needed.

[0196] Even if the area creator (editor) is not participating, the area information to be synchronized always requires an owner for the synchronized object for the synchronization process to work. In order to synchronize the area information to be synchronized, one of the terminal devices belonging to the synchronization group of the area information to be synchronized must be the owner of the synchronized object. There are no methods for determining the owner of the synchronized object. For example, it could be the terminal device that has been a member of the synchronization group the longest, or the terminal device with the smallest terminal ID. Furthermore, in order to reduce the number of synchronized objects, a method has been described to synchronize a single or small number of synchronized area information objects consisting only of the parameters to be synchronized from all the objects that make up the area information. However, in cases like area information, where there is no designated participant who actively operates the synchronized object, and the owner of the synchronized object can be anyone, it may be possible to generate multiple synchronized area information objects and have each be owned by a different terminal device in order to balance the load on the server. However, the distribution of ownership must be done carefully, as having different terminal devices own synchronized objects that have dependencies on each other may complicate control or even increase the load.

[0197] Figure 16 shows how, when there are multiple rooms 101 and 102 in area 100, the non-area and area information to be synchronized are synchronized as primary, replica, and tertiary on each target terminal. Users A, B, and C are participating in room 101 of area 100 with terminal devices A, B, and C respectively. Similarly, users X and Y are participating in room 102 of area 100 with terminal devices X and Y respectively. The non-areas (A0, B0, C0) targeted for synchronization by users A, B, and C are synchronized (A2, B2, C2) on user terminal devices A, B, and C, which are participants in room 101, via replicas of server 1 (A1, B1, C1). The non-areas (X0, Y0) targeted for synchronization by users X and Y are synchronized (X2, Y2) on user terminal devices X and Y, which are participants in room 102, via replicas of server 1 (X1, Y1). The area information (G0) targeted for synchronization by area 100 is synchronized (G2) on user terminal devices A, B, C, X, and Y, which are participants in rooms 101 and 102, which are rooms in area 100, via a replica of the server (G1).

[0198] The above describes an embodiment in which, when multiple rooms exist in a single area, area information and non-area information are synchronized by designating the terminal device group to be synchronized as area participating terminals and room participating terminals. However, synchronization groups are not limited to areas and rooms. Furthermore, the relationship between synchronization groups is not limited to one being encompassed by the other, as in areas and rooms. Synchronization groups can be created freely. Synchronization groups may be completely identical, partially included, partially common, or completely independent with no common parts. Each synchronization group manages participant terminals in the manner shown in Figure 4 or Figure 5. While examples of dynamically created and deleted rooms have been mentioned above, the terminal devices of synchronization groups are not necessarily fixed and immutable. Depending on the content of the virtual space experience provided, each synchronization group may have participants that change over time, and synchronization groups may be dynamically created and deleted, or terminals may join or leave the synchronization group midway. For example, in addition to areas and rooms, synchronization groups can include chat groups, collaborative parties, and affiliated organizations. Chat groups, collaborative parties, and affiliated organizations exist beyond the boundaries of rooms and areas, and each unit synchronizes and shares text, voice messages, and activity statuses (online, visiting space X, visiting cat world Y, battling monster X, visiting company M showroom, negotiating with company N, etc.) by overlaying them onto the shared synchronized experience within the room or area. For example, voice chat can be overlaid on the speaker for listening, and users can switch between speaking to the room or the chat group by toggling buttons or switches on the screen. Activity statuses can also be displayed by clicking the team status icon in the upper right corner of the screen, which will overlay a list of the statuses of affiliated organization members onto the display screen of the virtual world they are participating in.

[0199] Furthermore, another aspect of the present invention provides the following information processing device, data synchronization program, data synchronization method, data synchronization system, terminal device, execution program, and information processing system to achieve the above objectives.

[0200] [1] An information processing device connected by a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of a synchronized object from a client that owns the synchronized object; updating means for updating a duplicate object obtained by duplicating the synchronized object using the change information; and transmitting means for transmitting the change information to a plurality of clients other than the client that owns the object. [2] The information processing device according to [1], wherein the receiving means receives a synchronized object owned by a client when the client joins and stores it as a duplicate object on a recording medium. [3] The information processing device according to [1], wherein the receiving means duplicates the newly created object as a synchronized object when the client creates a new object and stores the duplicate object on a recording medium. [4] The information processing device according to [2] or [3], wherein the transmitting means transmits the duplicate object or the synchronized object stored on the recording medium to a plurality of clients other than the client that owns the object when the client joins or when the client creates a new object. [5] The receiving means is an information processing apparatus according to any one of [1] to [4] above, wherein the receiving means does not receive change information of the synchronized object from a client other than the client that owns the synchronized object. [6] The updating means is an information processing apparatus according to any one of [1] to [4] above, wherein the receiving means does not update the duplicate object when it receives change information of the synchronized object from a client other than the client that owns the synchronized object. [7] The transmitting means is an information processing apparatus according to any one of [1] to [4] above, wherein the receiving means does not transmit change information of the synchronized object to the client that owns the synchronized object when it receives change information of the synchronized object from a client other than the client that owns the synchronized object.[8] The information processing apparatus according to any one of [1] to [7], wherein the update means receives the difference between the synchronized object and the synchronized object before the change as change information of the synchronized object, and updates the duplicate object. [9] The information processing apparatus according to any one of [1] to [8], wherein when the client leaves, the ownership of the duplicate object corresponding to the synchronized object that the client owned is transferred to another client other than the said client based on predetermined rules, if the transfer of ownership is permitted.

[10] The information processing apparatus according to any one of [1] to [8], wherein when the client leaves, the transfer of ownership of the duplicate object corresponding to the synchronized object that the client owned is not permitted, and the transmission means instructs the other client to delete the object corresponding to the duplicate object that is held by the other client other than the said client.

[11] A data synchronization program for a computer connected via a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of a synchronized object from a client that owns the synchronized object; updating means for updating a duplicate object that is a copy of the synchronized object based on the change information; and transmitting means for transmitting the change information to the plurality of clients other than the client that owns the object.

[12] A data synchronization method for an information processing device connected via a network to a plurality of clients that synchronize objects with each other, comprising the steps of: receiving change information of a synchronized object from a client that owns the synchronized object; updating a duplicate object that is a copy of the synchronized object based on the change information; and transmitting the change information to the plurality of clients other than the client that owns the object.

[13] A data synchronization system comprising: a plurality of clients that synchronize objects with each other; an information processing device connected to the plurality of clients by a network, the information processing device comprising: receiving means for receiving change information of the synchronized object from a client that owns the synchronized object; updating means for updating a duplicate object that is a copy of the synchronized object based on the change information; and transmitting means for transmitting the change information to the plurality of clients other than the client that owns the object.

[14] The data synchronization system according to

[13] , wherein if the plurality of clients receive a duplicate object of a synchronized object owned by them from the information processing device, they refuse to receive the duplicate object.

[15] The data synchronization system according to

[13] or

[14] , wherein the plurality of clients transmit the difference between the synchronized object and the original synchronized object as change information of the synchronized object to the information processing device.

[16] A terminal device as a client that synchronizes objects with other clients, comprising: receiving means for receiving change information of the synchronized object from a client that owns the synchronized object; updating means for updating a duplicate object which is a copy of the synchronized object based on the change information; and transmitting means for transmitting the change information to a plurality of clients other than the client that owns the object, and being communicably connected to an information processing device.

[17] The terminal device as a client according to

[16] , which, when it receives a duplicate object of a synchronized object which it owns from the information processing device, rejects the receipt of the duplicate object.

[18] The terminal device as a client according to

[16] or

[17] , which transmits the difference between the synchronized object before the change and the original synchronized object as change information of the synchronized object to the information processing device.

[19] An information processing device connected via a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; updating means for updating a duplicate object that is a copy of the synchronized object based on the change information, if the sending client is defined in the management information as the client that owns the synchronized object, which is managed for each synchronized object; and transmitting means for transmitting the change information to a client other than the client that owns the object among the plurality of clients.

[20] An information processing device connected via a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; updating means for updating a duplicate object that is a copy of the synchronized object based on the change information; and transmitting means for transmitting change information to a client other than the client that owns the object, if the sending client is defined in the management information as the client that owns the synchronized object, which is managed for each synchronized object.

[21] The receiving means is the information processing apparatus according to

[19] or

[20] , wherein when a client joins a virtual space in which the plurality of clients are participating, the receiving means receives the synchronized object owned by the client and stores it on a recording medium as the duplicated object.

[22] The receiving means is the information processing apparatus according to

[19] or

[20] , wherein when the client creates a new object, the receiving means duplicates the newly created object as the synchronized object and stores the duplicated object on a recording medium.

[23] The information processing apparatus according to

[21] or

[22] wherein the transmitting means transmits the duplicate object stored on the recording medium or the synchronized object to the client other than the client that owns the object to the client when a client joins or when the client creates a new object.

[24] The information processing apparatus according to any one of

[19] to

[23] wherein the receiving means does not receive change information of the synchronized object from a client other than the client that owns the synchronized object.

[25] The information processing apparatus according to any one of

[19] to

[23] wherein the updating means does not update the duplicate object when the receiving means receives change information of the synchronized object from a client other than the client that owns the synchronized object.

[26] The information processing apparatus according to any one of

[19] to

[23] wherein the transmitting means does not transmit change information of the synchronized object to the client that owns the object when the receiving means receives change information of the synchronized object from a client other than the client that owns the synchronized object.

[27] The information processing apparatus according to any one of

[19] to

[26] , wherein the updating means receives the difference between the synchronized object and the synchronized object before the change as change information of the synchronized object, and updates the duplicate object.

[28] The information processing apparatus according to any one of

[19] to

[27] , wherein when the client leaves the virtual space in which the plurality of clients participate, the client transfers ownership of the duplicate object corresponding to the synchronized object that the client owned, to another client other than the said client based on predetermined rules, if the transfer of ownership is permitted.

[29] The information processing apparatus according to any one of

[19] to

[26] , wherein when the client exits the virtual space in which the client participates, the transfer of ownership of the copy object corresponding to the synchronized object that the client owned is not permitted, and the transmission means further functions as a management means to delete the copy object, wherein the transmission means instructs the other client to delete the object corresponding to the copy object that is held by the other client.

[30] A data synchronization program for a computer connected by a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; updating means for updating a copy object that has duplicated the synchronized object using the change information, which refers to management information managed for each synchronized object that defines the client that owns the synchronized object, and when the sending client is defined in the management information as the client that owns the synchronized object; and transmitting means for transmitting the change information to the plurality of clients other than the client that owns it.

[31] A computer connected via a network to a plurality of clients that synchronize objects with each other, comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; updating means for updating duplicate objects that are copies of the synchronized objects based on the change information; and a data synchronization program for causing a computer connected via a network to a plurality of clients that synchronize objects to each other to function as a receiving means for receiving change information of synchronized objects from the plurality of clients, and if the sending client is defined in the management information as the client that owns the synchronized object, the sending means for sending the change information to a client other than the client that owns the object among the plurality of clients.

[32] A data synchronization method in an information processing device connected via a network to a plurality of clients that synchronize objects with each other, comprising the steps of: receiving change information of an object to be synchronized from the plurality of clients; referring to management information managed for each object to be synchronized, which defines the client that owns the object to be synchronized, and if the sending client is defined in the management information as the client that owns the object to be synchronized, updating a duplicate object that has been duplicated from the object to be synchronized using the change information; and transmitting the change information to a client among the plurality of clients other than the client that owns the object.

[33] A data synchronization method in an information processing device connected via a network to a plurality of clients that synchronize objects with each other, comprising the steps of: receiving change information of an object to be synchronized from the plurality of clients; updating a duplicate object that has been duplicated from the object to be synchronized using the change information; referring to management information managed for each object to be synchronized, which defines the client that owns the object to be synchronized, and if the sending client is defined in the management information as the client that owns the object to be synchronized, transmitting the change information to a client among the plurality of clients other than the client that owns the object.

[34] A data synchronization system comprising: a plurality of clients that synchronize objects with each other; an information processing device connected to the plurality of clients by a network, the information processing device comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; management information managed for each synchronized object, which refers to the management information that defines the client that owns the synchronized object, and if the sending client is defined in the management information as the client that owns the synchronized object, updating a duplicate object that is a copy of the synchronized object based on the change information; and transmitting means for transmitting the change information to a client among the plurality of clients other than the client that owns the object.

[35] A data synchronization system comprising: a plurality of clients that synchronize objects with each other; an information processing device connected to the plurality of clients by a network, the information processing device comprising: receiving means for receiving change information of synchronized objects from the plurality of clients; updating means for updating duplicate objects which are duplicates of the synchronized objects based on the change information; and a transmission means for sending change information to a client other than the owner of the synchronized object, which is management information managed for each synchronized object and refers to the management information which defines a client that owns the synchronized object, and if the sending client is defined in the management information as a client that owns the synchronized object.

[36] The data synchronization system according to

[34] or

[35] , wherein the plurality of clients transmit the difference between the synchronized object before the change and the information processing device as change information of the synchronized object.

[37] A terminal device as a client that synchronizes objects with other clients via an information processing device, the terminal device as a client that is communicably connected to the information processing device, the information processing device having: receiving means for receiving information about changes to the synchronized object from the client that owns the synchronized object, when the sending client is defined in the management information as the client that owns the synchronized object; updating means for updating a duplicate object which is a copy of the synchronized object based on the change information; and transmitting means for transmitting the change information to a plurality of clients other than the client that owns the object.

[38] A terminal device as a client that synchronizes objects with other clients via an information processing device, wherein when a duplicate object which is a copy of a synchronized object is updated on its own client, the terminal device as a client that is communicable with the information processing device, the terminal device as a client that is communicable with the information processing device, the terminal device as a client that is communicable with the information processing device, the terminal device as a client that is communicable with the information processing device, the terminal device as a client that synchronizes objects which is a copy object, is updated on its own client, and the terminal device as a client that is communicable with the information processing device has management information which is managed for each of the synchronized objects which is a copy object which is a copy object, and if the self is defined in the management information which is a copy object which is a copy object which is a copy object, the copy object is communicable with the information processing device. Potential for industrial use

[0201] This invention provides an information processing device, a data synchronization program, a data synchronization method, a data synchronization system, a terminal device, an executable program, and an information processing system that suppress increases in processing and communication burdens in a configuration in which multiple user terminals participate and 3D data from multiple user terminals are synchronized.

[0202] 1: Server 2a, 2b, 2c: Terminal devices 3a, 3b, 3c: Users 4a, 4b, 4c: Avatars 10: Control unit 11: Storage unit 12: Memory 13: Communication unit 100: Terrain information management means 101: Terrain information editing means 102: Room management means 103: Participant management means 104: Data receiving means 105: Data transmission means 106: Replica management means 107: Data update means 110: Data management program 111: Terrain information 112: Room management information 113: Participant management information 114A: Replica management information 114B: Replica management information 120a, 120b, 120c: Replicas

Claims

An information processing device connected via a network to multiple clients that synchronize the basic data used to represent three-dimensional objects, Receiving means that receive from the aforementioned multiple clients either the difference in the basic data to be synchronized, the effect on the basic data, or conversion data that transforms the basic data, An information processing apparatus having an update means for updating the duplicated base data obtained by replicating the synchronization target using any of the received difference, the action, or the conversion data. 　 The information processing apparatus according to claim 1, further comprising a transmission means for transmitting any of the difference, the operation, or the conversion data to the plurality of clients. 　 The information processing apparatus according to claim 2, wherein the transmission means transmits the encoded difference, the action, or the conversion data without decoding it. 　 The system further includes a codec means for decoding the received difference, the effect, or the conversion data. If there are no problems with the result of checking the updated replication base data using the difference, the action, or the conversion data, The information processing apparatus according to claim 2, wherein the transmission means transmits any of the difference, the action, or the conversion data to the plurality of clients. 　 A data synchronization program executed on an information processing device connected via a network to multiple clients that synchronize the basic data used to represent a three-dimensional object, The aforementioned information processing device Receiving means that receive from the aforementioned multiple clients either the difference in the basic data to be synchronized, the effect on the basic data, or conversion data that transforms the basic data, A data synchronization program that functions as an update means for updating the duplicated base data of the synchronization target, which is a copy of the synchronization target, using any of the received difference, the action, or the conversion data. 　 A data synchronization method performed on an information processing device connected via a network to multiple clients that synchronize the basic data used to represent a three-dimensional object, The steps include receiving from the aforementioned multiple clients either the difference in the basic data to be synchronized, the effects on the basic data, or the transformation data that transforms the basic data, A data synchronization method comprising the step of updating the duplicated base data obtained by replicating the synchronization target using any of the received difference, the action, or the conversion data. 　 The basic data used to represent three-dimensional objects is connected via a network through multiple clients and a relay server that synchronize with each other. Receiving means that receive from the aforementioned multiple clients either the difference in the basic data to be synchronized, the effect on the basic data, or conversion data that transforms the basic data, A terminal device having an update means for updating the duplicated base data obtained by replicating the synchronization target using any of the received difference, the action, or the conversion data. 　 An execution means for updating the basic data of the synchronization target according to the execution content, A codec means that encodes either the difference, effect, or conversion data corresponding to the changes made to the execution means, The terminal device according to claim 7, further comprising a transmission means for transmitting the difference, the operation, or the conversion data to the plurality of clients. 　 The terminal device according to claim 8, wherein the execution means renders the basic data according to the execution content. 　 An executable program that runs on terminal devices connected via a network through multiple clients and a relay server, where the basic data for representing three-dimensional objects is synchronized with each other, The aforementioned terminal device, Receiving means that receive from the aforementioned multiple clients either the difference in the basic data to be synchronized, the effect on the basic data, or conversion data that transforms the basic data, An executable program that functions as an update means for updating the duplicated base data obtained by replicating the synchronization target using any of the received difference, the action, or the conversion data. 　 An information processing system comprising multiple clients and information processing devices connected via a network, which synchronize the basic data used to represent three-dimensional objects, The information processing device includes receiving means for receiving either the difference in the basic data to be synchronized, an action on the basic data, or conversion data for converting the basic data from the plurality of clients, and updating means for updating the duplicated basic data which is a copy of the synchronized data based on the received difference, action, or conversion data. The information processing system comprises a receiving means for receiving either the difference in the basic data to be synchronized, an action on the basic data, or conversion data that transforms the basic data from the plurality of clients other than itself, and an updating means for updating the replicated basic data which replicates the synchronization target using the received difference, action, or conversion data.

Images