Multi-user space redirection roaming system based on ai prediction

Abstract

The application discloses a multi-user space redirection roaming system based on AI prediction, wherein virtual position data is collected in real time, user pose data is input into a trajectory prediction model based on a multi-agent interactive perception long short-term memory network, the trajectory prediction model explicitly encodes the space-time interaction influence between users through a graph attention mechanism, and individual motion inertia features and group interaction features are fused, the redirection roaming path is gain-adjusted according to the individualized dizziness sensitivity parameters of each user, and redirection control instructions of each user are generated; the redirection control instructions are sent to virtual reality display devices of each user respectively to drive the visual redirection presentation of the virtual scene. The visual dizziness problem in the VR roaming process is effectively reduced, 10+ users can simultaneously experience individualized dizziness control, and the efficiency is improved to 40%+.

Description

Technical Field

[0001] This invention relates to the field of redirection roaming technology, and in particular to a multi-user spatial redirection roaming system based on AI prediction. Background Technology

[0002] Current traditional multi-user VR roaming systems suffer from significant technical shortcomings. Most systems rely solely on real-time pose data from a single user for path correction, failing to adequately consider the spatiotemporal motion interactions between multiple users and thus unable to accurately predict group movement trends. Existing trajectory prediction models often employ basic, single-sequence networks, lacking the ability to perceive the interactive relationships between multiple user agents and struggling to encode dynamic spatiotemporal interference characteristics between users, resulting in low accuracy and significant lag in trajectory prediction. Furthermore, traditional path planning algorithms suffer from poor versatility, failing to optimize collaborative paths for multi-user parallel roaming needs, and are prone to virtual path conflicts and roaming stutters. This leads to low spatial utilization, low accuracy, and low efficiency in multi-user spatial redirection roaming. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a multi-user spatial redirection roaming system based on AI prediction.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following modules in the aforementioned AI-based multi-user spatial redirection roaming system:

[0005] The real-time data acquisition module is used to collect user pose data of multiple users in physical space in real time. The user pose data includes the three-dimensional spatial position, walking direction, walking speed and head orientation of each user.

[0006] The motion trajectory prediction module is used to input the user pose data into a trajectory prediction model based on a MAI-LSTM multi-agent interactive perception long short-term memory network. The trajectory prediction model explicitly encodes the spatiotemporal interaction influence between users through a graph attention mechanism and integrates individual motion inertia features and group interaction features to predict the future motion trajectory of each user.

[0007] The collaborative path planning module is used to generate redirection roaming paths for each user based on future motion trajectories using a multi-agent collaborative path planning algorithm.

[0008] The personalized gain adjustment module is used to adjust the gain of the redirection roaming path according to each user's personalized dizziness sensitivity parameters and generate redirection control commands for each user.

[0009] The instruction output module is used to send the redirection control instructions to the virtual reality display devices of each user to drive the visual redirection of the virtual scene.

[0010] Furthermore, in the aforementioned AI-based multi-user spatial redirection roaming system, the motion trajectory prediction module includes the following sub-modules:

[0011] The individual motion coding submodule is used to map the historical pose temporal data of each user into an individual motion feature vector. The individual motion coding module adopts a two-layer LSTM structure to extract temporal features from the position sequence and the pose sequence respectively.

[0012] The Interactive Perception Graph Encoding Submodule is used to construct a spatiotemporal interaction graph with each user as a node. It uses a graph attention network (GAT) to calculate the interaction influence weight between any two users and aggregates the individual motion feature vectors on the spatiotemporal interaction graph to generate an enhanced feature vector containing group interaction information.

[0013] The gated feature fusion submodule is used to adaptively weight and fuse the individual motion feature vector and the enhanced feature vector through a learnable gating mechanism to generate a fused feature vector.

[0014] The interactive perception decoding submodule is used to input the fused feature vector into the LSTM decoder with physical space boundary constraints, and decode and output the predicted trajectory coordinate sequence of each user within the future time window.

[0015] Furthermore, in the aforementioned AI-predictive multi-user spatial redirection roaming system, the interaction-aware graph encoding submodule includes the following units:

[0016] The adjacency matrix construction unit is used to construct an initial adjacency matrix with each user as a node and the inverse of the spatial distance between users. When the spatial distance between any two users is greater than the preset truncation distance threshold, the edge weight between them is reset to zero.

[0017] The attention coefficient calculation unit is used to calculate the relative velocity vector and relative position vector between each user pair, and then concatenates the relative velocity vector and relative position vector and inputs them into the attention scoring function to generate the original attention coefficient.

[0018] The normalization processing unit is used to normalize the original attention coefficients using a sparse Softmax function to obtain normalized interaction influence weights, so that each user only pays attention to the K neighboring users who have the greatest impact on their movement, where K is a preset positive integer.

[0019] Furthermore, in the aforementioned AI-based multi-user spatial redirection roaming system, the motion trajectory prediction module includes the following sub-modules:

[0020] The data acquisition submodule is used to collect historical multi-user roaming trajectory datasets. The datasets contain the pose and time-series data of multiple groups of users walking synchronously, and label each group of data with real collision event tags between users.

[0021] A construction submodule is used to construct a composite loss function, which includes trajectory displacement loss, interaction consistency loss, and collision prior loss. The trajectory displacement loss is used to predict the mean square error between the trajectory coordinates and the actual trajectory coordinates. The interaction consistency loss is used to predict the difference between the minimum distance between user pairs in the trajectory and the actual minimum distance. The collision prior loss is an additional penalty term applied when the predicted trajectory causes the user spacing to be less than the safety threshold.

[0022] The training submodule is used to optimize the training of the composite loss function using the Backpropagation Time Trial (BPTT) algorithm combined with a gradient pruning strategy.

[0023] Furthermore, in the aforementioned AI-predictive multi-user spatial redirection roaming system, the collaborative path planning module includes the following sub-modules:

[0024] The state space construction submodule is used to construct the state space of a multi-agent system with each user as an independent intelligent agent. The state space includes the current position, predicted trajectory, velocity vector and direction vector of each user.

[0025] The cost function construction submodule is used to construct the cost function, which includes a weighted sum of space utilization cost, collision risk cost, redirection smoothness cost, and reset frequency cost;

[0026] The information exchange submodule is used to perform collaborative information exchange among agents using a distributed multi-agent optimization algorithm, and iteratively solve for each user redirection path that minimizes the cost function.

[0027] Furthermore, in the aforementioned AI-based prediction-based multi-user space redirection roaming system, the distributed multi-agent optimization algorithm in the information exchange submodule utilizes the MADRL (Multi-Agent Deep Reinforcement Learning) framework, where each agent makes collaborative decisions by sharing global state information to minimize the global cost function.

[0028] Furthermore, in the aforementioned AI-predictive multi-user spatial redirection roaming system, the personalized gain adjustment module includes the following sub-modules:

[0029] The calibration test submodule is used to perform a dizziness sensitivity calibration test when the user first uses the device, and to record the user's subjective comfort score and physiological response data under different redirection gain conditions.

[0030] The vector generation submodule is used to generate personalized dizziness sensitivity feature vectors for each user based on the calibration test results.

[0031] The dynamic adjustment submodule is used to dynamically adjust the redirection gain threshold and gain change rate for each user based on the personalized dizziness sensitivity feature vector.

[0032] Furthermore, in the AI-predictive multi-user spatial redirection roaming method, the method includes the following steps:

[0033] Real-time acquisition of user pose data of multiple users in physical space, including the three-dimensional spatial position, walking direction, walking speed and head orientation of each user;

[0034] The user pose data is input into a trajectory prediction model based on a MAI-LSTM multi-agent interactive perception long short-term memory network. The trajectory prediction model explicitly encodes the spatiotemporal interaction influence between users through a graph attention mechanism and integrates individual motion inertia features and group interaction features to predict the future motion trajectory of each user.

[0035] Based on future motion trajectories, a multi-agent collaborative path planning algorithm is used to generate redirection roaming paths for each user.

[0036] Based on each user's personalized dizziness sensitivity parameters, the gain of the redirection roaming path is adjusted to generate redirection control commands for each user;

[0037] The redirection control command is sent to each user's virtual reality display device to drive the visual redirection of the virtual scene.

[0038] Furthermore, in the AI-predictive multi-user spatial redirection roaming method, the method includes the following steps:

[0039] Each user is treated as an independent intelligent agent, and a state space for a multi-agent system is constructed. The state space includes the current position, predicted trajectory, velocity vector, and direction vector of each user.

[0040] Construct a cost function, which includes a weighted sum of space utilization cost, collision risk cost, redirection smoothness cost, and reset frequency cost;

[0041] By utilizing a distributed multi-agent optimization algorithm, collaborative information exchange is performed among the agents to iteratively solve for each user redirection path that minimizes the cost function.

[0042] Furthermore, in the AI-predictive multi-user spatial redirection roaming method, the method includes the following steps:

[0043] A dizziness sensitivity calibration test was performed when the user first used the device, and the user's subjective comfort score and physiological response data were recorded under different redirection gain conditions.

[0044] Based on the calibration test results, a personalized dizziness sensitivity feature vector is generated for each user.

[0045] Based on the personalized dizziness sensitivity feature vector, the redirection gain threshold and gain change rate of each user are dynamically adjusted.

[0046] Its beneficial effects lie in the fact that, through the real-time data acquisition module, it comprehensively collects multi-dimensional data on the user's physical pose and virtual position, covering parameters such as three-dimensional position, walking state, and head orientation, providing comprehensive and accurate data support for trajectory prediction and path planning, thus solving the problem of single-dimensional data acquisition in traditional methods. Secondly, it employs a MAI-LSTM multi-agent interactive perception network to construct a prediction model, relying on graph attention mechanisms to explicitly encode the spatiotemporal interaction characteristics of multiple users, integrating individual motion inertia and group interaction patterns, significantly improving the accuracy and real-time performance of predicting future motion trajectories for multiple users, and effectively avoiding roaming conflicts caused by lag in motion prediction. Simultaneously, the multi-agent collaborative path planning algorithm can generate differentiated paths for multi-user parallel roaming scenarios, achieving intelligent collaborative avoidance of paths among multiple users, completely resolving the defects of path conflicts and roaming stutters in traditional systems, and significantly improving the smoothness and stability of multi-user collaborative roaming. Furthermore, the personalized gain adjustment module adapts the redirection path based on the user's dizziness sensitivity parameters, achieving personalized roaming control adjustments for each user, effectively reducing visual dizziness during VR roaming and improving the comfort of the user's immersive experience. Ultimately, precise command output drives visual redirection in the virtual scene, enabling large-scale, highly adaptable, and highly stable multi-user virtual roaming within a limited physical space. This significantly expands the application scope and practical value of VR multi-user collaborative scenarios. However, this technology has drawbacks, including potential collisions with multiple users, passive responses leading to inconsistent experiences, and low space utilization (industry average <30%). Key technological highlights include: LSTM neural network prediction of user trajectories (intervention 500ms-2000ms in advance), multi-agent collaborative path planning supporting simultaneous experiences for 10+ users, and personalized dizziness control, increasing space utilization to over 40%. Attached Figure Description

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0048] Figure 1 This is a schematic diagram of the first embodiment of the multi-user spatial redirection roaming system based on AI prediction in this invention.

[0049] Figure 2 This is a schematic diagram of the second embodiment of the multi-user spatial redirection roaming system based on AI prediction in this invention.

[0050] Figure 3 This is a schematic diagram of the first embodiment of the multi-user spatial redirection roaming system based on AI prediction in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0053] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, a multi-user spatial redirection roaming system based on AI prediction includes the following modules:

[0054] The real-time data acquisition module is used to collect user pose data of multiple users in the physical space in real time. The user pose data includes the three-dimensional spatial position, walking direction, walking speed and head orientation of each user.

[0055] Specifically, this embodiment also includes the objects and content of the data collection: the system simultaneously collects the pose data of multiple users in the physical space (the three-dimensional spatial position, walking direction, walking speed and head orientation of each user) as well as the virtual position data of each user in the virtual space.

[0056] Physical spatial pose data is acquired through external positioning systems deployed in the physical environment, such as optical motion capture systems, ultra-wideband (UWB) indoor positioning systems, or LiDAR arrays. The six-degree-of-freedom pose information of each user is output in real time using the multi-base station triangulation principle.38 Virtual spatial location data is acquired through the fusion of the inertial measurement unit (IMU) and visual odometry (VO) built into the head-mounted display (HMD) worn by each user, recording the user's spatial coordinates and orientation in the virtual scene in real time. After the two types of data are aligned using a unified temporal synchronization mechanism, a structured data stream is formed and sent to the trajectory prediction module in step two.

[0057] The acquisition frequency is typically set to 30Hz-60Hz to ensure that subtle motion changes during the user's walking process can be captured. Each set of data is indexed by a timestamp and includes the user ID, physical space three-dimensional coordinates (x, y, z), walking direction vector (dx, dy, dz), walking speed scalar, head orientation quaternion, and corresponding coordinates in virtual space, forming a complete time-series data frame.

[0058] The motion trajectory prediction module is used to input the user pose data into a trajectory prediction model based on a MAI-LSTM multi-agent interactive perception long short-term memory network. The trajectory prediction model explicitly encodes the spatiotemporal interaction influence between users through a graph attention mechanism and integrates individual motion inertia features and group interaction features to predict the future motion trajectory of each user.

[0059] Specifically, this embodiment also includes an individual motion encoding submodule, used to map the historical pose time-series data of each user into individual motion feature vectors. The individual motion encoding submodule adopts a two-layer LSTM structure to extract temporal features from the position sequence and pose sequence respectively; an interaction-aware graph encoding submodule, used to construct a spatiotemporal interaction graph with each user as a node, using a graph attention network (GAT) to calculate the interaction influence weight between any two users, and performing neighborhood aggregation on the spatiotemporal interaction graph to generate an enhanced feature vector containing group interaction information; a gated feature fusion submodule, used to adaptively weight and fuse the individual motion feature vector and the enhanced feature vector through a learnable gating mechanism to generate a fused feature vector; and an interaction-aware decoding submodule, used to input the fused feature vector into an LSTM decoder with physical space boundary constraints, and decode and output the predicted trajectory coordinate sequence of each user within the future time window.

[0060] The adjacency matrix construction unit is used to construct an initial adjacency matrix with each user as a node and the inverse of the spatial distance between users. When the spatial distance between any two users is greater than a preset truncation distance threshold, the edge weight between them is reset to zero. The attention coefficient calculation unit is used to calculate the relative velocity vector and relative position vector between each user pair. The relative velocity vector and relative position vector are concatenated and input into the attention scoring function to generate the original attention coefficient. The normalization processing unit is used to normalize the original attention coefficient using the sparsity Softmax function to obtain the normalized interaction influence weight, so that each user only pays attention to the K neighboring users who have the greatest influence on their movement, where K is a preset positive integer.

[0061] The data acquisition submodule is used to collect historical multi-user roaming trajectory datasets. The dataset contains the pose and temporal data of multiple groups of users walking synchronously, and labels the actual collision events between users for each data group. The construction submodule is used to construct a composite loss function, which includes trajectory displacement loss, interaction consistency loss, and collision prior loss. The trajectory displacement loss is used to predict the mean square error between the trajectory coordinates and the actual trajectory coordinates. The interaction consistency loss is used to predict the difference between the minimum distance between user pairs in the trajectory and the actual minimum distance. The collision prior loss is an additional penalty term applied when the predicted trajectory causes the user distance to be less than a safe threshold. The training submodule is used to optimize and train the composite loss function using the backpropagation time time (BPTT) algorithm combined with a gradient pruning strategy.

[0062] The proposed MAI-LSTM network explicitly models the spatiotemporal interaction relationships between users by introducing a graph attention mechanism and integrating individual motion inertial features with group interaction features to achieve high-precision prediction of future trajectories for multiple users. It consists of the following four sequentially connected modules:

[0063] The individual motion encoding module is responsible for converting each user's historical pose data into individual motion feature vectors. It employs a two-layer LSTM structure. The first LSTM layer extracts temporal features from the user's position sequence (changes in 3D spatial coordinates over time), capturing the macroscopic motion trend of the user's walking. The second LSTM layer extracts temporal features from the user's posture sequence (changes in walking direction and head orientation over time), capturing the patterns of posture changes. The outputs of the two LSTM layers are concatenated along the feature dimension to form a vector representing the individual motion inertia characteristics of that user. The core function of this module is to preserve the unique motion patterns of each user, such as individual differences like one user's habit of walking quickly while another user's habit of turning slowly.

[0064] The Interactive Perceptual Graph Encoding Module is a key innovative component that distinguishes MAI-LSTM from traditional LSTM. It constructs a spatiotemporal interaction graph with each user as a node. Each node in the graph represents a user, and the edges between nodes represent the possibility of interaction between users.

[0065] Building upon this, a graph attention network (GAT) is employed to calculate the interaction influence weights between any two users. The calculation of attention weights is based on the spatial distance and relative motion state between users—the closer the distance and the greater the relative speed, the higher the interaction weight. Simultaneously, a sparsity processing mechanism is introduced, filtering out user pairs that are too far apart by setting a preset truncation distance threshold. This ensures that each user only focuses on the K nearest neighbors (K is a preset positive integer) that have the greatest impact on their motion, reducing the computational complexity from O(N²) to O(N·K) and guaranteeing that the system supports real-time concurrent prediction for more than 10 users.

[0066] After obtaining the interaction influence weights, the output individual motion feature vectors are aggregated in the spatiotemporal interaction graph. That is, each user not only uses its own motion features, but also weighted and fused with the motion features of its neighboring users to generate an enhanced feature vector containing group interaction information. For example, when other users in front of a user slow down, the module will capture this interaction signal and reflect it in the enhanced features.

[0067] The gated feature fusion module adaptively weights and fuses individual motion feature vectors with enhanced feature vectors using a learnable gating mechanism. The core idea of ​​this gating mechanism is that the network automatically learns during training whether to rely more on individual motion inertia or group interaction information in different scenarios. For example, when a user is in a densely populated area, the gating mechanism automatically increases the weight of the enhanced feature vector (i.e., relies more on group interaction information); when a user is in an open area, the gating mechanism automatically increases the weight of the individual motion feature vector (i.e., relies more on its own motion inertia). The fused feature vector is then used as input to the decoder.

[0068] The interactive sensing decoding module integrates feature vector inputs with an LSTM decoder8 that incorporates physical space boundary constraints. The decoder outputs a sequence of predicted trajectory coordinates for each user within a time window of 500ms to 2000ms using an autoregressive approach. Physical space boundary constraints ensure that the predicted trajectory does not exceed the effective range of physical space, avoiding invalid predictions. The final output is the predicted 3D coordinates for each user at each time step within the prediction time window.

[0069] Training method of MAI-LSTM network: The network training adopts a composite loss function, which includes three sub-losses: (1) trajectory displacement loss - the error between the predicted trajectory coordinates and the actual trajectory coordinates; (2) interaction consistency loss - the difference between the minimum distance between user pairs in the predicted trajectory and the actual minimum distance; (3) collision prior loss - an additional penalty term applied when the predicted trajectory causes the user distance to be less than the safety threshold. The training adopts the backpropagation time time (BPTT) algorithm combined with the gradient pruning strategy.

[0070] During the inference phase, the system backpropagates the prediction errors of the most recent N frames to some of the network's learnable parameters, performing lightweight online updates to the model to adapt to the user's real-time changes in gait patterns and fatigue levels during walking.

[0071] The collaborative path planning module is used to generate redirection roaming paths for each user based on future motion trajectories using a multi-agent collaborative path planning algorithm.

[0072] Specifically, this embodiment also includes a state space construction submodule, used to construct the state space of a multi-agent system with each user as an independent agent. The state space includes the current position, predicted trajectory, velocity vector, and direction vector of each user. A cost function construction submodule is used to construct a cost function, which includes a weighted sum of space utilization cost, collision risk cost, redirection smoothness cost, and reset frequency cost. An information exchange submodule is used to use a distributed multi-agent optimization algorithm to perform collaborative information exchange among the agents and iteratively solve for the redirection path of each user that minimizes the cost function.

[0073] The distributed multi-agent optimization algorithm in the information exchange submodule utilizes the MADRL framework of multi-agent deep reinforcement learning. Each agent makes collaborative decisions by sharing global state information in order to minimize the global cost function.

[0074] An optimized redirection roaming path is generated for each user using a multi-agent cooperative path planning algorithm.

[0075] The core principle of redirection roaming is that, as the user walks, the system introduces subtle visual-physical path deviations (i.e., redirection gain) at the user's visual perception level. This creates a subtle difference between the user's actual walking trajectory in physical space and their perceived trajectory in virtual space, thereby unconsciously guiding the user to a usable area in physical space. Traditional redirection methods are designed only for single users. In multi-user scenarios, each user's redirection instructions are independent, easily leading to collisions between users. This invention solves this problem through multi-agent cooperative planning.

[0076] Each user is treated as an independent intelligent agent, and a state space for a multi-agent system is constructed. The state space includes the current position, predicted trajectory, velocity vector, and direction vector of each user.

[0077] Construct a multi-objective cost function that includes a weighted sum of the following sub-costs: 38

[0078] Space utilization cost—characterizes the efficiency of each user's use of physical space, and encourages users to be dispersed and guided to different areas of the physical space to avoid crowding;

[0079] Collision risk cost — characterizes the deviation between the minimum distance between any two users' predicted trajectories and a preset safety threshold, ensuring that users maintain a safe distance;

[0080] Redirection smoothness cost—characterizes the rate of curvature change of the redirection path, avoiding sudden changes in direction that could cause user discomfort;

[0081] Reset frequency cost — represents the expected number of times the system resets the user's location per unit of time, minimizing reset operations to maintain a consistent experience.

[0082] A dynamic potential field weight is introduced into the cost function. Based on the real-time density distribution of each user in the physical space, the distribution of repulsive force weight in the environment is dynamically adjusted. When a user approaches a dense area, the repulsive force intensity is increased, so that the user naturally moves away from the crowded area.

[0083] A distributed multi-agent optimization algorithm is employed to facilitate collaborative information exchange among the agents, iteratively solving for the user redirection paths that minimize the cost function. The specific implementation framework is Multi-Agent Deep Reinforcement Learning (MADRL), where agents collaborate by sharing global state information. This framework supports concurrent redirection and roaming services for more than 10 users.

[0084] The personalized gain adjustment module is used to adjust the gain of the redirection roaming path according to each user's personalized dizziness sensitivity parameters and generate redirection control commands for each user.

[0085] Specifically, this embodiment also includes a calibration test submodule, which is used to perform a dizziness sensitivity calibration test when the user first uses the device, and record the user's subjective comfort score and physiological response data under different redirection gain conditions; a vector generation submodule, which is used to generate personalized dizziness sensitivity feature vectors for each user based on the calibration test results; and a dynamic adjustment submodule, which is used to dynamically adjust the redirection gain threshold and gain change rate of each user according to the personalized dizziness sensitivity feature vectors.

[0086] The generated redirection paths are individually gain-adjusted to generate the final redirection control commands for each user.

[0087] Redirection gain includes curvature gain and bending gain. Curvature gain makes the turning radius perceived by the user in virtual space smaller than the actual turning radius of walking, thus achieving a larger virtual turning radius in a smaller physical space; bending gain makes the user perceive a slightly curved path when walking straight. However, different users have vastly different sensitivities to these gains—sensitive users may experience dizziness even with small gains, while tolerant users can tolerate larger gains.

[0088] When a user first uses the system, a dizziness sensitivity calibration test is performed. During the test, the system applies different intensities of redirection gain to the user and records the user's subjective comfort score (out of 110) and physiological response data (heart rate variability, skin conductance, etc.). Based on the calibration test results, a personalized dizziness sensitivity feature vector is generated for each user.

[0089] During actual operation, the system dynamically adjusts the maximum amplitude and rate of change of curvature gain and bending gain based on each user's personalized vertigo sensitivity feature vector. For users with high vertigo sensitivity, the system limits the gain change amplitude to a lower level and changes gain at a smoother rate to avoid users experiencing vertigo due to perceived intense visual-vestibular conflict; for users with high tolerance, the system allows for larger gain change amplitudes and faster change rates to maximize space utilization. This module ensures that users with different vertigo sensitivities can have a comfortable roaming experience.

[0090] The instruction output module is used to send redirection control instructions to each user's virtual reality display device to drive the visual redirection of the virtual scene.

[0091] Specifically, in this embodiment, the redirection control commands of each user are sent to the corresponding HMD device worn by the user in a low-latency manner via a wireless transmission protocol.

[0092] After receiving control commands, the HMD adjusts the rendering parameters of the virtual camera in real time—specifically, it applies an offset to the orientation and position of the virtual camera corresponding to the redirection command during each frame of rendering. This causes the visual scene in the virtual space to exhibit a slightly different orientation when the user walks in a certain direction in the physical space. This visual-physical deviation is small enough (usually within a few degrees) that the user will not perceive the manipulation, but over a sufficiently long timescale, the user's actual walking trajectory is unconsciously guided to the usable area of ​​the physical space.

[0093] Its beneficial effects lie in the fact that, through the real-time data acquisition module, it comprehensively collects multi-dimensional data on the user's physical pose and virtual position, covering parameters such as three-dimensional position, walking state, and head orientation, providing comprehensive and accurate data support for trajectory prediction and path planning, thus solving the problem of single-dimensional data acquisition in traditional methods. Secondly, it employs a MAI-LSTM multi-agent interactive perception network to construct a prediction model, relying on graph attention mechanisms to explicitly encode the spatiotemporal interaction characteristics of multiple users, integrating individual motion inertia and group interaction patterns, significantly improving the accuracy and real-time performance of predicting future motion trajectories for multiple users, and effectively avoiding roaming conflicts caused by lag in motion prediction. Simultaneously, the multi-agent collaborative path planning algorithm can generate differentiated paths for multi-user parallel roaming scenarios, achieving intelligent collaborative avoidance of paths among multiple users, completely resolving the defects of path conflicts and roaming stutters in traditional systems, and significantly improving the smoothness and stability of multi-user collaborative roaming. Furthermore, the personalized gain adjustment module adapts the redirection path based on the user's dizziness sensitivity parameters, achieving personalized roaming control adjustments for each user, effectively reducing visual dizziness during VR roaming and improving the comfort of the user's immersive experience. Ultimately, precise command output drives visual redirection in the virtual scene, enabling large-scale, highly adaptable, and highly stable multi-user virtual roaming within a limited physical space. This significantly expands the application scope and practical value of VR multi-user collaborative scenarios. However, this technology has drawbacks, including potential collisions with multiple users, passive responses leading to inconsistent experiences, and low space utilization (industry average <30%). Key technological highlights include: LSTM neural network prediction of user trajectories (intervention 500ms-2000ms in advance), multi-agent collaborative path planning supporting simultaneous experiences for 10+ users, and personalized dizziness control, increasing space utilization to over 40%.

[0094] Please see Figure 2 In an AI-based multi-user spatial redirection roaming system, the motion trajectory prediction module includes the following sub-modules:

[0095] The individual motion coding submodule is used to map the historical pose time-series data of each user into individual motion feature vectors. The individual motion coding module adopts a two-layer LSTM structure to extract time-series features from the position sequence and the pose sequence respectively.

[0096] The Interactive Perception Graph Encoding Submodule is used to construct a spatiotemporal interaction graph with each user as a node. It uses a graph attention network (GAT) to calculate the interaction influence weight between any two users and aggregates the individual motion feature vectors on the spatiotemporal interaction graph to generate an enhanced feature vector containing group interaction information.

[0097] The gated feature fusion submodule is used to adaptively weight and fuse individual motion feature vectors and enhanced feature vectors through a learnable gating mechanism to generate a fused feature vector.

[0098] The interactive perception decoding submodule is used to input the fused feature vector into the LSTM decoder with physical space boundary constraints, and decode and output the predicted trajectory coordinate sequence of each user within the future time window.

[0099] Please see Figure 3 In an AI-predictive multi-user spatial redirection roaming system, the interaction-aware graph coding submodule includes the following units:

[0100] The adjacency matrix construction unit is used to construct an initial adjacency matrix with each user as a node and the inverse of the spatial distance between users. When the spatial distance between any two users is greater than the preset truncation distance threshold, the edge weight between them is reset to zero.

[0101] The attention coefficient calculation unit is used to calculate the relative velocity vector and relative position vector between each user pair. The relative velocity vector and relative position vector are concatenated and then input into the attention scoring function to generate the original attention coefficient.

[0102] The normalization processing unit is used to normalize the original attention coefficients using a sparse Softmax function to obtain normalized interaction influence weights, so that each user only pays attention to the K neighboring users who have the greatest impact on their movement, where K is a preset positive integer.

[0103] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and equivalents.

Claims

1. A multi-user spatial redirection roaming system based on AI prediction, characterized in that, The multi-user space redirection roaming system includes the following modules: The real-time data acquisition module is used to collect user pose data of multiple users in physical space in real time. The user pose data includes the three-dimensional spatial position, walking direction, walking speed and head orientation of each user. The motion trajectory prediction module is used to input the user pose data into a trajectory prediction model based on a MAI-LSTM multi-agent interactive perception long short-term memory network. The trajectory prediction model explicitly encodes the spatiotemporal interaction influence between users through a graph attention mechanism and integrates individual motion inertia features and group interaction features to predict the future motion trajectory of each user. The collaborative path planning module is used to generate redirection roaming paths for each user based on future motion trajectories using a multi-agent collaborative path planning algorithm. The personalized gain adjustment module is used to adjust the gain of the redirection roaming path according to each user's personalized dizziness sensitivity parameters and generate redirection control commands for each user. The instruction output module is used to send the redirection control instructions to the virtual reality display devices of each user to drive the visual redirection of the virtual scene.

2. The multi-user spatial redirection roaming system based on AI prediction as described in claim 1, characterized in that, The motion trajectory prediction module Includes the following sub-modules: The individual motion coding submodule is used to map the historical pose temporal data of each user into an individual motion feature vector. The individual motion coding module adopts a two-layer LSTM structure to extract temporal features from the position sequence and the pose sequence respectively. The Interactive Perception Graph Encoding Submodule is used to construct a spatiotemporal interaction graph with each user as a node. It uses a graph attention network (GAT) to calculate the interaction influence weight between any two users and aggregates the individual motion feature vectors on the spatiotemporal interaction graph to generate an enhanced feature vector containing group interaction information. The gated feature fusion submodule is used to adaptively weight and fuse the individual motion feature vector and the enhanced feature vector through a learnable gating mechanism to generate a fused feature vector. The interactive perception decoding submodule is used to input the fused feature vector into the LSTM decoder with physical space boundary constraints, and decode and output the predicted trajectory coordinate sequence of each user within the future time window.

3. The multi-user spatial redirection roaming system based on AI prediction as described in claim 2, characterized in that, The interactive perception graph encoding submodule includes the following units: The adjacency matrix construction unit is used to construct an initial adjacency matrix with each user as a node and the inverse of the spatial distance between users. When the spatial distance between any two users is greater than the preset truncation distance threshold, the edge weight between them is reset to zero. The attention coefficient calculation unit is used to calculate the relative velocity vector and relative position vector between each user pair, and then concatenates the relative velocity vector and relative position vector and inputs them into the attention scoring function to generate the original attention coefficient. The normalization processing unit is used to normalize the original attention coefficients using a sparse Softmax function to obtain normalized interaction influence weights, so that each user only pays attention to the K neighboring users who have the greatest impact on their movement, where K is a preset positive integer.

4. The multi-user spatial redirection roaming system based on AI prediction as described in claim 1, characterized in that, The motion trajectory prediction module includes the following sub-modules: The data acquisition submodule is used to collect historical multi-user roaming trajectory datasets. The datasets contain the pose and time-series data of multiple groups of users walking synchronously, and label each group of data with real collision event tags between users. A construction submodule is used to construct a composite loss function, which includes trajectory displacement loss, interaction consistency loss, and collision prior loss. The trajectory displacement loss is used to predict the mean square error between the trajectory coordinates and the actual trajectory coordinates. The interaction consistency loss is used to predict the difference between the minimum distance between user pairs in the trajectory and the actual minimum distance. The collision prior loss is an additional penalty term applied when the predicted trajectory causes the user spacing to be less than the safety threshold. The training submodule is used to optimize the training of the composite loss function using the Backpropagation Time Trial (BPTT) algorithm combined with a gradient pruning strategy.

5. The multi-user spatial redirection roaming system based on AI prediction as described in claim 1, characterized in that, The collaborative path planning module includes the following sub-modules: The state space construction submodule is used to construct the state space of a multi-agent system with each user as an independent intelligent agent. The state space includes the current position, predicted trajectory, velocity vector and direction vector of each user. The cost function construction submodule is used to construct the cost function, which includes a weighted sum of space utilization cost, collision risk cost, redirection smoothness cost, and reset frequency cost; The information exchange submodule is used to perform collaborative information exchange among agents using a distributed multi-agent optimization algorithm, and iteratively solve for each user redirection path that minimizes the cost function.

6. The multi-user spatial redirection roaming system based on AI prediction as described in claim 5, characterized in that, The distributed multi-agent optimization algorithm in the information exchange submodule utilizes the MADRL (Multi-Agent Deep Reinforcement Learning) framework, where agents collaborate to make decisions by sharing global state information in order to minimize the global cost function.

7. The multi-user spatial redirection roaming system based on AI prediction as described in claim 1, characterized in that, The personalized gain adjustment module includes the following sub-modules: The calibration test submodule is used to perform a dizziness sensitivity calibration test when the user first uses the device, and to record the user's subjective comfort score and physiological response data under different redirection gain conditions. The vector generation submodule is used to generate personalized dizziness sensitivity feature vectors for each user based on the calibration test results. The dynamic adjustment submodule is used to dynamically adjust the redirection gain threshold and gain change rate for each user based on the personalized dizziness sensitivity feature vector.

8. A method for implementing a multi-user spatial redirection roaming system based on AI prediction as described in claim 1, characterized in that, The method includes the following steps: Real-time acquisition of user pose data of multiple users in physical space, including the three-dimensional spatial position, walking direction, walking speed and head orientation of each user; The user pose data is input into a trajectory prediction model based on a MAI-LSTM multi-agent interactive perception long short-term memory network. The trajectory prediction model explicitly encodes the spatiotemporal interaction influence between users through a graph attention mechanism and integrates individual motion inertia features and group interaction features to predict the future motion trajectory of each user. Based on future motion trajectories, a multi-agent collaborative path planning algorithm is used to generate redirection roaming paths for each user. Based on each user's personalized dizziness sensitivity parameters, the gain of the redirection roaming path is adjusted to generate redirection control commands for each user; The redirection control commands are sent to each user's virtual reality display device to drive the visual redirection of the virtual scene.

9. A method for implementing a multi-user spatial redirection roaming system based on AI prediction as described in claim 8, characterized in that, The method includes the following steps: A multi-agent system state space is constructed with each user as an independent intelligent agent. The state space includes the current position, predicted trajectory, velocity vector, and direction vector of each user. Construct a cost function, which includes a weighted sum of space utilization cost, collision risk cost, redirection smoothness cost, and reset frequency cost; By utilizing a distributed multi-agent optimization algorithm, collaborative information exchange is performed among the agents to iteratively solve for each user redirection path that minimizes the cost function.

10. A method for implementing a multi-user spatial redirection roaming system based on AI prediction as described in claim 8, characterized in that, The method includes the following steps: A dizziness sensitivity calibration test was performed when the user first used the device, and the user's subjective comfort score and physiological response data were recorded under different redirection gain conditions. Based on the calibration test results, a personalized dizziness sensitivity feature vector is generated for each user. Based on the personalized dizziness sensitivity feature vector, the redirection gain threshold and gain change rate of each user are dynamically adjusted.

Images