Multi-person multi-scene real-time synchronization method and device for offline large-space XR product and storage medium

Abstract

The invention discloses a multi-user multi-scene real-time synchronization method and device for an offline large-space XR product and a storage medium, and belongs to the technical field of augmented reality, and the method comprises the steps: converting the user positions in different virtual scenes into a unified local coordinate system in a physical space, and enabling all users to share real-time position information in the physical space; grouping the users, managing the users in groups, synchronizing information of the users in real time based on a state-driven synchronization strategy, the information including virtual role information and virtual scene information of the users; different groups of users are mutually independent, do not share virtual scene information, but share virtual role information, and the virtual role information is used for generating a digital human avatar in a virtual scene and avoiding collision of others. The method has the advantages of high efficiency, light weight, high concurrency support and good expansibility, and is suitable for a multi-user interaction scene of an offline large-space XR product.

Description

Technical Field

[0001] The present invention relates to the field of extended reality technology, and in particular to a multi-person and multi-scene real-time synchronization method, device and storage medium for an offline large-space XR product. Background Art

[0002] Existing offline large-space XR products, such as VR theme parks and offline VR escape rooms, usually use fixed locations, fixed scenes, and timed experiences to conduct multi-person interactive VR experiences. However, this method has a huge problem of floor efficiency. When the number of people increases, they cannot experience it in the venue at the same time and can only wait for the previous group to finish. In traditional multi-person real-time synchronization scenarios, such as online multiplayer games, architectures based on DS (Dedicated Server) or LS (Listen Server) are usually used. These solutions generally rely on complex game engine communication protocols and synchronization solutions, involving a large number of user behavior states, physical simulations, and complex scene logic synchronization. However, in offline large-space XR products, the business requirements for real-time synchronization are relatively simple, mainly used to synchronize basic user information (such as location, posture, role, gender, etc.) and ensure that multiple people can avoid collisions in the same physical space.

[0003] Offline large-space XR products have the following characteristics: 1. High concurrency: More than 100 people may experience the same physical space at the same time, and they need to synchronize each other's location information in real time. 2. Multi-scene synchronization: Although users experience in different virtual scenes, they need to share location information in the physical space to avoid collisions between users. 3. Lightweight synchronization requirements: Only basic user information (such as location, posture, status, level, etc.) needs to be synchronized, without complex physical simulation and behavior prediction. 4. High efficiency and low latency: Offline large spaces require real-time synchronization, and too high a delay will reduce the user experience.

[0004] Existing game engines and multiplayer real-time synchronization solutions often fail to meet the above requirements, with problems such as complex architecture, high communication overhead, and strong dependence on specific engines. Therefore, a lightweight and efficient multiplayer multi-scene real-time synchronization method is needed to meet the needs of offline large-space XR products. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a multi-person and multi-scene real-time synchronization method, device and storage medium for an offline large-space XR product.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A multi-person multi-scene real-time synchronization method for an offline large-space XR product includes the following steps:

[0008] Convert user positions in different virtual scenes to a unified local coordinate system in physical space, so that all users can share real-time position information in one physical space;

[0009] Group users, manage users in groups, and synchronize information of users in the group in real time based on state-driven synchronization strategies. The information includes the user's virtual character information and virtual scene information. The virtual character information includes: position, posture, role, gender and group number; the virtual scene information includes: level number and timeline progress;

[0010] Users in different groups are independent of each other and do not share virtual scene information, but share virtual character information. The virtual character information is used to generate a digital human avatar to be used as a man, woman or child in a virtual scene, and to change the height of the virtual avatar according to the height characteristics to warn others and avoid collisions.

[0011] To optimize the above technical solutions, the specific measures taken also include:

[0012] Furthermore, the conversion of user positions in different virtual scenes into a unified local coordinate system in the physical space is specifically as follows:

[0013] Initialize the physical space to determine the origin and direction of the unified local coordinate system;

[0014] Calibrate each user's XR device so that the origin and orientation of the user's XR device are aligned with the origin and orientation of the unified local coordinate system;

[0015] The entry point in the virtual scene is used as the origin of the virtual scene, and the front and back directions and left and right directions of the virtual scene are calibrated to align them with the directions of the physical space;

[0016] Get the virtual scene coordinates (x_scene, y_scene, z_scene) of the XR device, and calculate the translation matrix T_scene2X and rotation matrix R_scene2X based on the calibration information of the virtual scene and the physical space;

[0017] Apply the following conversion formula to convert the coordinates in the virtual scene into coordinates in the unified local coordinate system:

[0018] [X,Y,Z]=T_scene2X*R_scene2X*[x_scene,y_scene,z_scene,1] T

[0019] Where [X, Y, Z] represents the coordinates in the unified local coordinate system, and the superscript T represents the transpose.

[0020] Furthermore, the group management of users specifically includes:

[0021] Dynamically adjust the number of users, levels or timelines of the group according to the background requirements. On the server side, use a hash table to store the user information of each group. Regularly clean up idle groups, which refers to groups where all users return to a static state. Each group regularly broadcasts the group's status and user information to all XR devices.

[0022] On the XR device side, switch the device state according to the server's instructions; in the reconnected or inserted state, obtain the latest group state from the server and update the scene and timeline of the XR device.

[0023] Furthermore, the state-driven synchronization strategy for real-time synchronization of information of users in the group is specifically:

[0024] After the user puts on the XR device, he scans the code and enters the information registered at the front desk; the user is randomly assigned to a group or enters a designated group;

[0025] If the number of users in a group reaches the upper limit, create a new group;

[0026] The server notifies the XR device to enter the ready state. The XR device loads user information and initializes the character, waiting for all users to complete initialization. The user switches from the ready state to the running state, enters the level and starts the story experience.

[0027] Real-time synchronization of the location information, posture, role, gender, group number, level number and timeline progress of the XR devices of users in the group;

[0028] Detect if the user is offline;

[0029] When the server detects that a user is offline, it marks the XR device status used by the user as disconnected and records the level number, timeline and current XR device ID. If the offline user is the only one in the group, the server uses the recorded level number, timeline and current XR device ID to reconnect. If there are other members in the group, the server uses the level numbers and timelines of other members to reconnect.

[0030] The new XR device scans the code to bind the user information, enters the insertion state, and is inserted into the level of the offline XR device. After the insertion is completed, the new XR device status switches to the running state. Or after the offline user changes the XR device, the server binds the user information and notifies to enter the reconnection state. After the new XR device loads the level and timeline of the offline device, it enters the running state.

[0031] In any state, the server can force the user to log out by triggering the forced exit state, and the XR device returns to the static state. For XR devices in the preparation state or reconnection state, a timeout detection is added, and the device automatically returns to the static state if it times out and does not enter the next state.

[0032] Furthermore, the calibration of each user's XR device is specifically as follows:

[0033] The origin of the user's XR device is aligned with its physical position, its direction is calibrated using the IMU data of the XR device, and aligned with the unified local coordinate system direction of the physical space.

[0034] Furthermore, the detecting whether the user is offline is specifically as follows:

[0035] The XR device periodically reports its status to the server through heartbeat packets. If the heartbeat packet times out, the server determines that the user is offline.

[0036] Furthermore, the server adopts a minimalist server architecture in a single site. If an unexpected situation such as server failure occurs, other supporting microservices such as self-checking and self-starting can help the server restart quickly. For offline operations, it can quickly restore to an operational state without users noticing.

[0037] Furthermore, the server may adopt a distributed server architecture in the case of multiple sites or a dramatic increase in business volume, with each server node being responsible for a fixed amount of business, thereby increasing the robustness and stability of the service.

[0038] Furthermore, the communication protocol between the server and the XR device includes UDP.

[0039] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the multi-person and multi-scene real-time synchronization method of the offline large-space XR product as described above is implemented.

[0040] The present invention also proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the multi-person and multi-scene real-time synchronization method for offline large-space XR products as described above.

[0041] The beneficial effects of the present invention are:

[0042] The present invention solves the complexity and limitation problems of traditional multi-person real-time synchronization methods in large-space XR application scenarios. Through a lightweight communication mechanism based on the UDP bare protocol and combined with a state-driven synchronization strategy, efficient real-time synchronization of multi-person information is achieved, including key fields such as user location information, posture, role, gender, group number, level number, and story progress. At the same time, the present invention converts the user positions in different scenes through a unified local coordinate system, so that all users can share real-time location information in a large offline space, thereby effectively preventing collisions between users. The present invention defines a variety of states (Idle, Ready, Running, Reconnect, Insert, Abort), and realizes dynamic management and rapid deployment of multiple groups of users through a state conversion mechanism, supporting functions such as device disconnection reconnection and insertion of specific scene timelines. The present invention is efficient, lightweight, high-concurrency support, and good scalability, and is suitable for multi-person interactive scenarios of offline large-space XR products. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the state-driven synchronization strategy;

[0044] Figure 2 It is a coordinate relationship diagram between the physical space and the virtual scene. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Embodiment 1

[0047] The present invention proposes a multi-person multi-scene real-time synchronization method for an offline large-space XR product, comprising the following steps:

[0048] Convert the user positions in different virtual scenes to a unified local coordinate system XROrigin in the physical space, so that all users share real-time location information in a physical space; because in offline large-space XR scenes, users may be distributed in different virtual scenes (levels), but all users share the same physical space. In order to avoid user collisions in the physical space and ensure the efficiency and consistency of multi-person real-time synchronization, it is necessary to design a unified coordinate system to convert the location information of all users from the coordinate system of their respective virtual scenes to a unified local coordinate system in the physical space. Specifically:

[0049] Initialize the physical space to determine the origin and direction of the unified local coordinate system; including: Select the reference point: Use the center point of the physical site or a fixed marker (such as a calibration mark in the site) as the XROrigin origin, determine the direction axis: Determine the X, Y, and Z direction axes of XROrigin through pre-calibration. X axis: horizontal forward direction in the site. Y axis: vertical upward direction. Z axis: horizontal lateral direction in the site.

[0050] Calibrate each user's XR device so that the origin and direction of the user's XR device are aligned with the origin and direction of the unified local coordinate system; the calibration of each user's XR device is specifically:

[0051] The origin of the user's XR device is aligned with its physical position, and its orientation is calibrated using the IMU data of the XR device and aligned with the orientation of the physical space.

[0052] The entry point in the virtual scene is used as the origin of the virtual scene, and the front and back and left and right directions of the virtual scene are calibrated to align them with the directions of the physical space;

[0053] Each XR device reports its position relative to the origin of the virtual scene. Combine the device's sensor data and the scene's calibration information to calculate the device's position in the virtual scene coordinate system. Get the XR device's virtual scene coordinates (x_scene, y_scene, z_scene), and calculate the translation matrix T_scene2X and rotation matrix R_scene2X based on the calibration information of the virtual scene and the physical space.

[0054] Apply the following conversion formula to convert the coordinates in the virtual scene into coordinates in the unified local coordinate system:

[0055] [X,Y,Z]=T_scene2X*R_scene2X*[x_scene,y_scene,z_scene,1] T

[0056] Where [X, Y, Z] represents the coordinates in the unified local coordinate system, and the superscript T represents the transpose.

[0057] The present invention also designs a fault-tolerant mechanism, including: using IMU data for real-time posture adjustment to reduce coordinate deviations caused by device tilt or jitter. The device position is reported to the server regularly, and the server performs error correction based on the position data of other users (such as recalibrating the device position through fixed reference points in the venue). By measuring the mapping relationship between virtual scene coordinates and physical space coordinates multiple times, the calibration error is automatically corrected. During operation, the virtual scene anchor point information is dynamically adjusted according to changes in the virtual scene. The anchor point information here represents the world position coordinate information and world rotation angle information of the origin of the XROrigin local coordinate system based on the virtual scene in different virtual scenes. When the XROrigin coordinates of two users are too close (such as the distance is less than a certain threshold), the client will display other characters in the scene to remind the user to adjust the position.

[0058] The application of the present invention in multiple scenarios includes:

[0059] (1) Unified coordinates across scenes

[0060] User A is in virtual scene 1, and user B is in virtual scene 2. After their positions in the virtual world are unified through the unified local coordinate system XROrigin, they can share their position information in real time.

[0061] For example: the world coordinates of user A in virtual scene 1 are [1, 2, 3], the world coordinates of the anchor point information in virtual scene 1 are [3, 4, 5], and XROrigin will be set to [3, 4, 5] in virtual scene 1. After the translation matrix transformation, the position of user A in the XROrigin local coordinate system is [-2, -2, -2]. The world coordinates of user B in virtual scene 2 are [2, 3, 4], and the world coordinates of the anchor point information in virtual scene 2 are [5, 6, 10]. After the translation matrix transformation, the position of user B in the XROrigin local coordinate system is [-3, -3, -6]. Although A and B are in different virtual scenes and different coordinate systems, after the matrix transformation in the anchor point information, both A and B can be represented by the XROrigin coordinate system, thereby avoiding collisions in the physical space. Similarly, the transformation of the rotation matrix can be superimposed on this basis.

[0062] (2) Dynamic scene switching

[0063] When the user switches from scene 1 to scene 2, only its scene anchor information needs to be updated, and the XROrigin coordinates remain unchanged.

[0064] (3) Synchronization within the group

[0065] After the level numbers and timelines of users in the same group are synchronized, each user's XROrigin coordinates are also updated in real time to ensure consistency of experience within the group.

[0066] In offline large-space XR scenarios, users usually participate in activities in groups, and each group of users may be in different levels, timelines or states. In order to meet the efficient management of multiple groups of users, it is necessary to design a flexible grouping and synchronization mechanism to group users, manage users in groups, and synchronize the information of users in the group in real time based on the state-driven synchronization strategy, including the user's location information, posture, role, gender, group number, level number and timeline progress.

[0067] State drive is the core synchronization mechanism of the system. Through the switching of states and the communication between the server and the client, dynamic management of real-time synchronization of multiple people and multiple scenes is realized. The following is a detailed description of each state, function description and state transition rules:

[0068] 1. State definition and function

[0069] 1.1Idle (static state)

[0070] Default state: All devices are initially in the Idle state, waiting for the user to scan the code before entering the next stage.

[0071] Features:

[0072] The client will not actively send data to the server in the Idle state, reducing the network burden.

[0073] The client loads the Idle scene and waits for the user to scan the code to activate it.

[0074] Trigger conditions:

[0075] The user is not bound (the code is not scanned).

[0076] After the user completes the experience, the system is forced to return to the Idle state.

[0077] Possible next status:

[0078] Ready: After scanning the code and entering the user information, the server notifies the client to enter the Ready state.

[0079] Reconnect: When the device is re-bound to user information, it directly enters the Reconnect state.

[0080] Abort: Return to Idle when forced to exit.

[0081] 1.2Ready

[0082] The state entered after scanning the code: The user enters information (such as name, gender, role, etc.) through the barcode scanner, the server sends the user information to the client, and the client enters the Ready state.

[0083] Features:

[0084] The client initializes user information and loads configurations such as user role, gender, height, etc.

[0085] The client starts real-time synchronization with the server and sends its own status information (such as ID, position, rotation, etc.) to the server.

[0086] Trigger conditions:

[0087] The user scans the code successfully, and the server notifies the client to enter the Ready state.

[0088] Possible next status:

[0089] Running: Normal process, the user enters the experience level.

[0090] Insert: The user re-enters a specific level after being disconnected.

[0091] Abort: Return to Idle when forced to exit.

[0092] 1.3Running (Running Status)

[0093] User formal experience state: the user enters the level, and the program drives the story linearly.

[0094] Features:

[0095] When the client is in the Running state, it continues to synchronize with the server in real time, including location information, posture information, etc.

[0096] The user's current level and progress (timeline) are broadcasted by the server to all group members to ensure synchronization within the group.

[0097] Trigger conditions:

[0098] Enter normally from Ready.

[0099] Enter from the Insert state.

[0100] Resume from the Reconnect state.

[0101] Possible next status:

[0102] Abort: Return to Idle when forced to exit.

[0103] Leaving (not in the current design, but can be considered): At the end of the experience, the system cleans up the process and returns to Idle.

[0104] 1.4Insert (Insert state)

[0105] Re-entering a certain level after being disconnected: After a single user is disconnected during the experience, it is used to reinsert into a specific level and timeline.

[0106] Features:

[0107] The server specifies the level and story progress (timeline) for the user to rejoin based on the user information before the disconnection.

[0108] The client loads the specified level and synchronizes it to the corresponding timeline.

[0109] Trigger conditions:

[0110] Reconnect after the user is disconnected.

[0111] Possible next status:

[0112] Running: After the insertion is completed, the system enters the normal experience state.

[0113] Abort: Return to Idle when forced to exit.

[0114] 1.5Reconnect (Reconnection Status)

[0115] Reconnection status after device replacement: The user's device is disconnected (such as the headset is out of power or the device is down), and the new device is re-bound to enter the Reconnect state.

[0116] Features:

[0117] The new device (such as VR_002) inherits all the information of the offline device (such as VR_001), including user ID, role, level number (level) and timeline progress (timeline).

[0118] The client loads the level and timeline of the offline device and restores it to its previous state.

[0119] For new devices, the state changes directly from Idle to Reconnect.

[0120] Trigger conditions:

[0121] The offline device is replaced, and the server binds the new device and notifies the client to Reconnect.

[0122] Possible next status:

[0123] Running: After successfully reconnecting, the system enters the normal experience state.

[0124] Abort: Return to Idle when forced to exit.

[0125] 1.6Abort (forced exit status)

[0126] Emergency exit in any state: used to interrupt the user's current state and force return to Idle.

[0127] Features:

[0128] The client immediately exits the current scene and resets all user information.

[0129] The server notifies the client to release resources and return to the Idle state.

[0130] Trigger conditions:

[0131] Administrators directly interrupt the user experience through the background.

[0132] Possible next status:

[0133] Idle: Return to the initial state after exiting.

[0134] The state transition rules are shown in Table 1:

[0135] Table 1 State transition rules

[0136]

[0137]

[0138] The group management users specifically include:

[0139] Dynamically adjust the number of users, levels or timelines of the group according to the background requirements. On the server side, use a hash table to store the user information of each group. Regularly clean up idle groups, which refers to groups where all users return to a static state. Each group regularly broadcasts the group's status and user information to all XR devices.

[0140] On the XR device side, switch the device state according to the server's instructions; in the reconnected or inserted state, obtain the latest group state from the server and update the scene and timeline of the XR device.

[0141] The core information of each group is maintained by the server, and the data structure is as follows:

[0142]

[0143] The state-driven synchronization strategy for real-time synchronization of information of users in the group is specifically as follows:

[0144] After the user puts on the XR device, he scans the code and enters the information registered at the front desk; the user is randomly assigned to a group or enters a designated group;

[0145] If the number of users in a group reaches the upper limit, create a new group;

[0146] The server notifies the XR device to enter the ready state. The XR device loads user information and initializes the character, waiting for all users to complete initialization. The user switches from the ready state to the running state, enters the level and starts the story experience.

[0147] Real-time synchronization of the location information, posture, role, gender, group number, level number and timeline progress of the XR devices of users in the group;

[0148] The data structure of the user's information is as follows, and supports extension:

[0149]

[0150] Field Description:

[0151] state: player status, including iIdle, Ready, Running, Reconnect, Insert, and Abort.

[0152] groupId: User group ID, used for batch management of multiple groups of users.

[0153] level: The current level number, used to distinguish different level scenes.

[0154] timeline: The current story progress, used to synchronize the timeline in the virtual scene.

[0155] Detect whether the user is offline; specifically:

[0156] The XR device periodically reports its status to the server through heartbeat packets. If the heartbeat packet times out, the server determines that the user is offline.

[0157] When the server detects that a user is offline, it marks the XR device status used by the user as disconnected and records the level number, timeline, and current XR device ID. If the offline user is the only one in the group, the recorded level number, timeline, and current XR device ID are used to reconnect; if there are other members in the group, the level numbers and timelines of other members of the group are used to reconnect.

[0158] The new XR device scans the code to bind the user information, enters the insertion state, and is inserted into the level of the offline XR device. After the insertion is completed, the new XR device state switches to the running state. Or after the offline user changes the XR device, the server binds the user information and sends a reconnection request to the server. Example request:

[0159]

[0160] After the new XR device enters the reconnection state, the server synchronizes the level and timeline of the group to ensure that the new XR device is consistent with other users in the group.

[0161] After the new XR device loads the level and timeline of the offline device, it enters the running state;

[0162] In any state, the server can force the user to log out by triggering a forced exit state, and the XR device returns to a static state. For XR devices in a ready state or a reconnected state, a timeout detection is added, and the device automatically returns to a static state if it times out without entering the next state. Due to the high pressure of offline operations, calculations are performed from morning to night, so the server design is required to be extremely simple, with only one server. The advantage of this is that the cost is low, and it has the functions of one-click restart and heartbeat self-start. Once the service goes down, it will be restarted and put back into operation in a very short time, and this process will not exceed 1 minute.

[0163] For high-concurrency scenarios, further optimize the communication overhead of state switching, such as reducing unnecessary data synchronization in Ready or Reconnect.

[0164] The synchronization frequency is dynamically adjusted according to the speed of the user's movement. For example, the synchronization frequency is lowered when stationary and increased when moving, further optimizing the communication overhead.

[0165] Users in different groups are independent of each other and do not share level numbers and timeline progress. For example, group A is at timeline 3.5 of level 1, and group B is at timeline 1.0 of level 2, without interfering with each other. However, they share virtual character information, which is used to generate digital human avatars to be used as men, women or children in virtual scenes, and to change the height of the virtual avatars according to their height characteristics, so as to warn others and avoid collisions.

[0166] The present invention also proposes a dynamic insertion mechanism, which is used in the following scenarios: 1) Single user rejoining a level: a user wants to be inserted from a specific level and timeline after being offline. 2) New user joining a group: an administrator wants to dynamically add a new user to a running group.

[0167] The process of dynamic insertion is as follows:

[0168] Insert request: The client sends an insert request to the server through a barcode scanner or background operation.

[0169] Sample request:

[0170]

[0171] On the server side, check whether the status of the target group is running, update the level and timeline of the device, and switch its status from idle to insert. After the insertion is completed, the device status changes to running and is added to the synchronization logic of the target group.

[0172] On the XR device side, the inserted user obtains the latest level and timeline of the group from the server to ensure consistency with other users in the group.

[0173] Embodiment 2

[0174] The present invention proposes a multi-person multi-scene real-time synchronization system for offline large-space XR products. The system implements a modular design and includes:

[0175] The user management module is used for user registration, grouping, status management, and processing of disconnection reconnection and dynamic insertion; it supports more user attributes (such as device type, permission classification, etc.).

[0176] The state management module is used for the state switching logic of users and groups (such as idle, ready, running, reconnect, etc.);

[0177] The synchronization module is used to synchronize data between the user and the server (such as location information, level progress).

[0178] The coordinate transformation module is used to process the local coordinate system transformation of XROrigin and map the device coordinates to a unified physical space coordinate system.

[0179] The scene management module is responsible for loading and switching virtual scenes, as well as calibrating scenes and physical spaces.

[0180] The background management module is used to provide administrators with real-time management capabilities for users, groups, and scenes (such as forced interruption, level adjustment, etc.).

[0181] Define the interaction between modules through interfaces to avoid hard coding. For example, the state synchronization module provides a standard interface to support adding new states or modifying state switching logic in the future without modifying the code of other modules.

[0182] Use event-driven model or publish / subscribe (Pub / Sub) mechanism to simplify communication between modules.

[0183] For example, when a user goes offline, a disconnection event is triggered, and other modules (such as the group management module) automatically respond by subscribing to the event.

[0184] Make key system parameters configurable (such as group size, number of levels, timeline resolution, etc.), and adjust them dynamically through configuration files or background to avoid code changes. When introducing new features, ensure that existing features are not affected. For example, when adding dynamic insertion logic, it does not affect the traditional disconnection reconnection function.

[0185] The modules communicate with each other through standard interfaces. The example interface design is shown in Table 2:

[0186] Table 2 Interface design

[0187]

[0188] The system proposed in this embodiment has good scalability, and the functional expansion scenarios are as follows:

[0189] 1. Add new status

[0190] If you need to support the pause state: In the state management module, define a new state pause. Modify the state flow rules to allow switching from running to pause, and from pause back to running. In the synchronization module, pause the synchronization of user location information, but keep the heartbeat detection.

[0191] 2. Support more user attributes

[0192] If you need to support user permission classification (such as administrators and ordinary users): in the user management module, add the user attribute role. Modify the group management logic to allow administrators to dynamically adjust the status or attributes of the group.

[0193] 3. Compatible with more devices

[0194] If you need to support mobile AR applications: In the coordinate conversion module, add the logic of calibrating the physical position captured by the mobile phone camera. In the synchronization module, expand the data protocol to support the location information format of the mobile phone.

[0195] 4. Dynamic scene expansion

[0196] If you need to support dynamically generated virtual scenes: In the scene management module, add scene generation logic (such as programmatic generation or AI modeling). Modify the scene loading interface to support dynamically generated scene IDs.

[0197] 5. Technology expansion direction

[0198] 5.1 Data synchronization optimization

[0199] 5.1.1 Differential synchronization:

[0200] Only synchronize the user's incremental data (such as location changes, status changes, etc.) to reduce network bandwidth usage. For example, if the user's location information has not changed, no synchronization package will be sent.

[0201] 5.1.2 Distributed Synchronization:

[0202] When the business scales up to a large scale, in large-scale user scenarios, a distributed server architecture is adopted, where each server is responsible for the synchronization operations of a part of users. For example, different groups are assigned to different server nodes.

[0203] 5.2 Higher Fault Tolerance

[0204] 5.2.1 Reconnect after disconnection:

[0205] Supports automatic reconnection after a short period of disconnection and restores to the latest state. For example: after a user is offline, the server retains its state for 30 seconds and waits for the device to reconnect.

[0206] 5.2.2 Redundancy Mechanism:

[0207] Add redundancy mechanisms to key modules (such as state management and data synchronization) to prevent single point failures from causing system crashes. For example, the current state and historical state of the state machine are saved at the same time, and can be rolled back to the previous state in case of an abnormality.

[0208] 5.3 Multi-platform support

[0209] 5.3.1 Cross-platform compatibility:

[0210] Supports access to multiple devices such as PC, VR, AR, and mobile terminals. In the synchronization module, a unified device interface is defined, and each device implements its own data processing logic.

[0211] 5.3.2 Cloud Rendering and Edge Computing:

[0212] For devices with weaker performance (such as mobile phones), some computing tasks (such as scene rendering) are placed in the cloud to complete. Use edge computing to reduce latency and improve the real-time performance of multi-device collaboration.

[0213] 5.4 Data Analysis and Optimization

[0214] 5.4.1 Real-time data analysis: record each user's experience data (such as movement trajectory, dwell time, etc.) for subsequent optimization. For example: analyze which levels have a high user dropout rate and optimize related designs.

[0215] 5.4.2 Machine learning assisted decision making:

[0216] Use machine learning models to predict user behavior and dynamically adjust group configuration or level difficulty. For example, automatically adjust the boundary prompts of the physical venue based on the user's movement speed.

[0217] Embodiment 3

[0218] The present invention proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-person and multi-scene real-time synchronization method for an offline large-space XR product as described in Example 1 is implemented.

[0219] Embodiment 4

[0220] The present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the multi-person and multi-scene real-time synchronization method for an offline large-space XR product as described in Example 1.

[0221] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0222] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0223] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A multi-person multi-scene real-time synchronization method for an offline large-space XR product, characterized in that: The following steps are involved: Convert user positions in different virtual scenes to a unified local coordinate system in physical space, so that all users can share real-time position information in one physical space; Group users, manage users in groups, and synchronize information of users in the group in real time based on state-driven synchronization strategies. The information includes the user's virtual character information and virtual scene information. The virtual character information includes: position, posture, role, gender and group number; the virtual scene information includes: level number and timeline progress; Users in different groups are independent of each other and do not share virtual scene information, but share virtual character information. The virtual character information is used to generate a digital human avatar and change the height of the digital human avatar according to the height characteristics to warn others and avoid collisions.

2. The method for real-time synchronization of multiple people and multiple scenes of an offline large-space XR product as claimed in claim 1, characterized in that: The method of converting the user positions in different virtual scenes into a unified local coordinate system in the physical space is specifically as follows: Initialize the physical space to determine the origin and direction of the unified local coordinate system; Calibrate each user's XR device so that the origin and orientation of the user's XR device are aligned with the origin and orientation of the unified local coordinate system; The entry point in the virtual scene is used as the origin of the virtual scene, and the front and back directions and left and right directions of the virtual scene are calibrated to align them with the directions of the physical space; Get the virtual scene coordinates (x_scene, y_scene, z_scene) of the XR device, and calculate the translation matrix T_scene2X and rotation matrix R_scene2X based on the calibration information of the virtual scene and the physical space; Apply the following conversion formula to convert the coordinates in the virtual scene into coordinates in the unified local coordinate system: [X, Y, Z] = T_scene2X * R_scene2X * [x_scene, y_scene, z_scene, 1] T Where [X, Y, Z] represents the coordinates in the unified local coordinate system, and the superscript T represents the transpose.

3. The method for real-time synchronization of multiple people and multiple scenes of an offline large-space XR product as claimed in claim 1, characterized in that: The group management users specifically include: Dynamically adjust the number of users, levels or timelines of the group according to the background requirements. On the server side, use a hash table to store the user information of each group. Regularly clean up idle groups, which refers to groups where all users return to a static state. Each group regularly broadcasts the group's status and user information to all XR devices. On the XR device side, switch the device state according to the server's instructions; in the reconnected or inserted state, obtain the latest group state from the server and update the scene and timeline of the XR device.

4. The method for real-time synchronization of multiple people and multiple scenes of an offline large-space XR product as claimed in claim 3, characterized in that: The state-driven synchronization strategy for real-time synchronization of information of users in the group is specifically as follows: After the user puts on the XR device, he scans the code and enters the information registered at the front desk; the user is randomly assigned to a group or enters a designated group; If the number of users in a group reaches the upper limit, create a new group; The server notifies the XR device to enter the ready state. The XR device loads user information and initializes the character, waiting for all users to complete initialization. The user switches from the ready state to the running state, enters the level and starts the story experience. Real-time synchronization of the location information, posture, role, gender, group number, level number and timeline progress of the XR devices of users in the group; Detect if the user is offline; When the server detects that a user is offline, it marks the XR device status used by the user as disconnected and records the level number, timeline and current XR device ID. If the offline user is the only one in the group, the server uses the recorded level number, timeline and current XR device ID to reconnect. If there are other members in the group, the server uses the level numbers and timelines of other members to reconnect. The new XR device scans the code to bind the user information, enters the insertion state, and is inserted into the level of the offline XR device. After the insertion is completed, the new XR device status switches to the running state. Or after the offline user changes the XR device, the server binds the user information and notifies to enter the reconnection state. After the new XR device loads the level and timeline of the offline device, it enters the running state. In any state, the server can force the user to log out by triggering the forced exit state, and the XR device returns to the static state. For XR devices in the ready state or reconnection state, a timeout detection is added, and the device automatically returns to the static state if it times out and does not enter the next state.

5. The method for real-time synchronization of multiple people and multiple scenes of offline large-space XR products as claimed in claim 2, characterized in that: The calibration of each user's XR device is specifically as follows: The user's XR device origin is aligned with its physical position, its orientation is calibrated using the XR device's IMU data, and aligned with the unified local coordinate system orientation of the physical space.

6. The method for real-time synchronization of multiple people and multiple scenes of an offline large-space XR product as claimed in claim 4, characterized in that: The specific steps of detecting whether a user is offline are as follows: The XR device periodically reports its status to the server through heartbeat packets. If the heartbeat packet times out, the server determines that the user is offline.

7. The method for real-time synchronization of multiple people and multiple scenes of an offline large-space XR product as claimed in claim 3, characterized in that: The communication protocol between the server and the XR device includes UDP.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-person and multi-scene real-time synchronization method for an offline large-space XR product as described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the multi-person and multi-scene real-time synchronization method for offline large-space XR products as described in any one of claims 1-7.

Images