An exhibition interaction method based on LBE technology XR large space

Abstract

The present invention discloses an exhibition interaction method for large-scale XR spaces based on LBE technology, which relates to the technical field of extended reality. The method comprises the following steps: aligning user location coordinates and environmental three-dimensional point cloud data in time and space using an ICP algorithm, filtering noise using a Gaussian filter, and forming a user location matrix; generating a spatial density distribution map using a PPO algorithm from the user location matrix, and dynamically meshing the spatial density distribution map to generate dynamic mesh layout parameters; inputting the dynamic mesh layout parameters into a chaotic prediction model, simulating user movement patterns using an improved Lorenz equation, extracting obstacle positions using a RANSAC algorithm, and simultaneously performing multi-dimensional feature mapping on user movement patterns and obstacle positions to generate a collision risk heat map. The present invention utilizes a chaotic prediction model and a RANSAC algorithm to improve the accuracy of user movement pattern prediction and the ability to identify obstacles in a changing environment, making the interactive experience smooth and natural while ensuring user safety.

Description

Technical Field

[0001] The present invention relates to the technical field of extended reality, and in particular to an exhibition interaction method based on LBE technology XR large space. Background Art

[0002] With the advancement of augmented reality (AR), virtual reality (VR), and extended reality (XR) technologies, location-based entertainment (LBE) has gradually become an emerging form of interactive entertainment. These technologies, particularly for exhibitions or experiential events held in large spaces, can provide a more immersive and personalized user experience. Over the past few decades, XR technology and LBE applications have achieved numerous remarkable achievements. On the one hand, the performance of XR devices has continued to improve, while their cost has gradually decreased, making high-quality XR experiences more affordable for more consumers. On the other hand, optimized software algorithms have greatly enhanced the realism and interactivity of the user experience. For example, modern XR devices enable high-precision positional tracking, allowing users to move freely within virtual environments without distortion. Improved rendering technology and 3D sound design further enhance the sense of immersion. Furthermore, with the development of artificial intelligence and machine learning technologies, XR systems can better understand and respond to user behavior, providing more personalized content and services. For example, intelligent recommendations can automatically adjust displayed content based on user preferences or predict their next actions by analyzing user behavior data. This has not only promoted the development of LBE technology but also laid the foundation for future research and innovation.

[0003] However, existing technologies still have some shortcomings. For one thing, the accuracy and real-time performance of user movement pattern predictions are insufficient. Existing technologies typically rely on location tracking technology and simple path prediction algorithms. While they perform well in static environments, their prediction accuracy and real-time responsiveness are insufficient in highly dynamic and ever-changing large-scale spaces. Furthermore, obstacle recognition and collision prevention mechanisms are inadequate. In large-scale exhibitions or interactive scenarios, ensuring participant safety while maintaining a high-quality interactive experience is crucial. However, traditional obstacle detection methods often focus on identifying static obstacles and have limited adaptability to rapidly changing environments. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an exhibition interaction method for large-space XR based on LBE technology to solve the problems of inaccurate movement mode prediction and imperfect anti-collision mechanism.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an exhibition interaction method for a large XR space based on LBE technology, which includes: aligning the user's position coordinates with the three-dimensional point cloud data of the environment through the ICP algorithm in time and space, and filtering the noise through Gaussian filtering to form a user position matrix;

[0008] The user position matrix is ​​used to generate a spatial density distribution map through the PPO algorithm, and the spatial density distribution map is dynamically meshed to generate dynamic mesh layout parameters;

[0009] The dynamic grid layout parameters are input into the chaotic prediction model. The user movement pattern is simulated by the improved Lorenz equation, and the obstacle positions are extracted by the RANSAC algorithm. At the same time, multi-dimensional feature mapping is performed on the user movement pattern and obstacle positions to generate a collision risk heat map.

[0010] Based on the collision risk heat map, a threshold segmentation algorithm is used to identify high-risk areas, obtain an interactive control priority list, and generate interactive adjustment instructions using the NSGA-II algorithm based on the interactive control priority list.

[0011] The interaction adjustment instructions are input into the XR rendering engine through the ROS middleware, and real-time interaction data is collected. The behavioral pattern of the real-time interaction data is analyzed using the triple exponential smoothing method, and an exhibition interaction report is generated through the Tableau engine.

[0012] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the user position matrix is ​​formed, the specific steps are as follows:

[0013] The ICP algorithm is used to map the user's location coordinates and the environment's 3D point cloud data into a unified coordinate system space and perform point cloud matching. The NTP protocol is used to perform spatiotemporal alignment simultaneously to generate a synchronized point cloud dataset.

[0014] Gaussian filtering uses convolution kernels to perform spatial smoothing and neighborhood enhancement on the synchronized point cloud dataset, identifies outliers, uses the SOR algorithm to filter noise based on the outliers, and converts the denoised point cloud dataset into a user position matrix through voxel index mapping.

[0015] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the generation of dynamic grid layout parameters, the specific steps are as follows:

[0016] The PPO algorithm is used to perform gradient updates on the user position matrix to obtain gradient trajectory data. Based on the gradient trajectory data, density calculation and spatial interpolation are performed through kernel density estimation to generate a spatial density distribution map.

[0017] The dynamic grid division method is used to adjust the grid density and control the deformation of the spatial density distribution map to obtain the grid deformation parameters; according to the grid deformation parameters, the hierarchical dynamic segmentation algorithm is used to dynamically segment the spatial density distribution map to generate dynamic grid layout parameters.

[0018] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the generation of the collision risk heat map, the specific steps are as follows:

[0019] In the PyTorch framework, the coupling layer, evolution layer, and perception layer are integrated through the Gaussian coupling mapping method to construct a chaos prediction model;

[0020] The coupling layer uses dynamic convolution kernels to extract features and assign weights to dynamic grid layout parameters to obtain the coupling parameter matrix;

[0021] The evolution layer uses the improved Lorenz equation to perform nonlinear dynamic modeling of the coupling parameter matrix and performs path probability calculation and pattern matching through dynamic time warping to obtain user mobility pattern data.

[0022] The perception layer uses the RANSAC algorithm to perform point cloud segmentation on the user's mobility pattern data to obtain the obstacle candidate point cloud, and then uses DBSCAN density clustering to perform bounding box fitting and spatial positioning to obtain the obstacle position;

[0023] The user mobility pattern data and obstacle positions are fused to generate a spatiotemporal coupling vector, and the spatial coupling vector is mapped into a spatial trajectory distribution using the phase space reconstruction method to generate a collision risk heat map.

[0024] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the step of obtaining the interaction control priority list is as follows:

[0025] The collision risk heat map is dynamically binarized using a threshold segmentation algorithm to obtain a binary mask. K-means clustering is used to extract the center point of the binary mask to generate a coordinate set of high-risk areas.

[0026] The fusion analytic hierarchy process is used to assign weights to the high-risk area coordinate set to generate a priority scoring matrix; the priority scoring matrix is ​​then sorted using principal component analysis to obtain a priority list of interactive regulation.

[0027] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the interaction adjustment instruction is generated by the NSGA-II algorithm, and the specific steps are as follows:

[0028] The NSGA-II algorithm is used to perform multi-objective trade-offs on the interactive control priority list to obtain a comprehensive score set. The comprehensive score set is divided into primary, secondary, and tertiary risk levels according to the three-level threshold.

[0029] The entropy weight method is used to map the risk levels to instruction weights and generate interactive adjustment instructions.

[0030] As a preferred solution of the exhibition interaction method based on LBE technology XR large space described in the present invention, wherein: the interaction adjustment instruction is input into the XR rendering engine through the ROS middleware, and real-time interaction data is collected. The specific steps are as follows:

[0031] The interactive adjustment instructions are matched to the XR rendering protocol format through the ROS middleware to generate an interactive instruction stream; the interactive instruction stream is input into the XR rendering engine using the OpenXRAPI interface;

[0032] The XR rendering engine drives dynamic exhibits and spatial navigation anchors through interactive instruction streams for exhibition interaction, and simultaneously uses the IMU inertial measurement unit to collect real-time interaction data.

[0033] As a preferred solution of the exhibition interaction method based on LBE technology XR large space of the present invention, wherein: the exhibition interaction report is generated by Tableau engine, the specific steps are as follows:

[0034] Triple exponential smoothing is used to separate the trends of real-time interaction data and obtain the interaction behavior feature set. The interaction behavior feature set is then analyzed for behavior patterns using a density clustering algorithm to generate a user behavior label matrix.

[0035] Based on the Tableau engine, the LOD dynamic rendering strategy is used to issue abnormal pattern warnings for the user behavior label matrix, obtain abnormal pattern data, integrate real-time interaction data and abnormal pattern data through a cross-source data mixing mechanism, and generate an exhibition interaction report.

[0036] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the exhibition interaction method for large-space XR based on LBE technology as described in the first aspect of the present invention is implemented.

[0037] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the exhibition interaction method for large-space XR based on LBE technology as described in the first aspect of the present invention is implemented.

[0038] The beneficial effects of the present invention are: through the chaos prediction model and the progressive Lorenz equation, the prediction accuracy of the user's movement pattern in a complex dynamic environment is significantly improved, making the interactive experience smoother and more natural; at the same time, the RANSAC algorithm is combined with multi-dimensional feature mapping to generate a detailed collision risk heat map, which enhances the ability to recognize obstacles in a rapidly changing environment and the collision warning effect, thereby ensuring the safety of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a flowchart of the exhibition interaction method in large-space XR based on LBE technology.

[0041] Figure 2 Flowchart of the process for generating parameters for dynamic grid layouts.

[0042] Figure 3 Flowchart of the process for interactive tuning instruction generation.

[0043] Figure 4 Flowchart of the collision risk heatmap generation process. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0047] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides an exhibition interaction method for a large space based on LBE technology XR, including the following steps:

[0048] S1, align the user position coordinates and the environment 3D point cloud data in time and space through the ICP algorithm, and filter the noise through Gaussian filtering to form a user position matrix.

[0049] The specific steps include:

[0050] The first step is to collect user position coordinates and environmental 3D point cloud data. First, in the XR environment, an optical tracking unit is installed on the ceiling of the exhibition space to collect position data through reflected infrared light. An inertial measurement unit is installed in the user's wearable device to collect relative displacement data through an accelerometer. The position and relative displacement data are aligned using the Precision Time Protocol (PTP) and fused using a sensor fusion algorithm to generate user position coordinate data.

[0051] Secondly, an RGB-D camera is placed in a fixed position in the corner of the exhibition space to capture depth information and color images of the scene, collecting environmental image data. A laser scanner is placed on the edge of the exhibition space to collect geometric structure data by emitting laser beams and receiving reflected signals. A voxel grid algorithm is used to downsample the environmental image data and geometric structure data to obtain sparse point cloud data. The sparse point cloud data is then smoothed using a sliding window to generate 3D point cloud data.

[0052] The second step is to perform spatiotemporal alignment and noise filtering on the collected user location coordinates and 3D point cloud data. First, the ICP algorithm is used to search for corresponding features between the location coordinates and the environment's 3D point cloud data to obtain an initial point cloud correspondence set. The KD tree acceleration structure (K-Dimensional Tree) is then used to perform nearest neighbor fast indexing and spatial partitioning on the initial point cloud correspondence set to generate optimized feature matching pairs.

[0053] Based on the optimized feature matching pairs, the user position coordinates and the environment 3D point cloud data are mapped to a unified coordinate system space using SVD singular value decomposition. The optimized feature matching pairs are then transformed using the least squares method to generate rigid transformation parameters. The rigid transformation parameters are then point cloud registered and iteratively optimized using the resolution registration method to generate the parameters after point cloud registration.

[0054] The parameters after point cloud registration are timestamped using the NTP protocol to generate a timestamped point cloud data frame. The clock offset is calculated using a four-timestamp interaction mechanism: the local clock is read from the timestamped point cloud data frame to obtain the T1 time (the time when the client sends the request); the Marzullo algorithm is used to record the reception time and filter outliers of the T1 time to generate the T2 time (the time when the server receives the time); the distributed clock synchronization method is used to encapsulate the response of the T2 time and embed the timestamp to generate the T3 time (the time when the server responds); the Cristian algorithm is used to record the response reception of the T3 time to obtain the T4 time (the time when the client receives the time);

[0055] The T4 time is timestamped using least squares fitting to obtain the clock offset. Based on the clock offset, the local clock frequency is adjusted using a PID controller by changing the proportional-integral-derivative ratio. Based on the calibrated local clock frequency, the parameters of the point cloud after registration are spatiotemporally aligned using a cubic spline interpolation algorithm to generate a synchronized point cloud dataset.

[0056] Secondly, Gaussian filtering is used to filter noise from the synchronized point cloud dataset. Specifically, Gaussian filtering uses a convolution kernel to perform a three-dimensional convolution operation on the synchronized point cloud dataset to obtain preliminary smoothed point cloud data; bilateral filtering is used to perform secondary spatial smoothing on the preliminary smoothed point cloud data; and local surface fitting is used to perform a k-nearest neighbor search on the smoothed point cloud data to obtain the normal vector point cloud structure; the vector weighted enhancement method is used to perform neighborhood enhancement on the normal vector point cloud structure to generate enhanced point cloud data; the statistical outlier detection method is used to calculate the k-nearest neighbor density of the enhanced point cloud data, and outliers are identified based on the 3σ criterion;

[0057] The SOR algorithm is used to weight the probability density of outliers to generate a candidate noise point set. Conditional Euclidean clustering is used to group the candidate noise point set into adjacent points to obtain the final noise point set. Based on the final noise point set, the noise is filtered using the radius filtering method to obtain the denoised point cloud dataset.

[0058] The third step is to convert the denoised point cloud dataset into a user position matrix through the voxel index mapping method. Specifically, the denoised point cloud dataset is divided into a three-dimensional spatial grid through the voxel index mapping method to generate voxelized point cloud data; the voxelized point cloud data is spatially clustered using the nearest neighbor search method to obtain clustered point cloud clusters; the centroid coordinates of the clustered point cloud clusters are calculated using the weighted average method to generate a key position point set; and the key position point set is converted into a user position matrix using the homogeneous transformation method;

[0059] The user location matrix can not only accurately record the user's real-time location and orientation in the exhibition space, but also lay the foundation for subsequent analysis of user behavior patterns.

[0060] S2. Generate a spatial density distribution map using the user position matrix through the PPO algorithm, and perform dynamic grid segmentation on the spatial density distribution map to generate dynamic grid layout parameters.

[0061] The specific steps include:

[0062] The first step is to use the PPO algorithm to perform gradient updates on the user position matrix and obtain gradient trajectory data. Specifically, the user position matrix is ​​first smoothed in time and space through Z-score normalization and sliding window to generate a time series enhancement feature matrix. The time series enhancement feature matrix is ​​then analyzed and reduced in dimension through principal component analysis to generate an orthogonal state vector.

[0063] Based on the PPO algorithm, an Actor-Critic dual network is constructed in the PyTorch framework. The nn.Sequential parameter is used to call two independent three-layer fully connected neural networks. The two neural networks are weighted together through a parameter sharing mechanism to obtain a shared feature layer. Based on the shared feature layer, a branching network structure is used to separate the Actor (strategy head) and the Critic (value head), completing the construction of the Actor-Critic dual network.

[0064] Secondly, the constructed Actor-Critic dual network is trained: the PPO-Clip algorithm is used to perform importance sampling on the orthogonal state vector to obtain the gradient optimization target; based on the gradient optimization target, the Adam optimizer is used to jointly optimize the MSE loss of the Actor network and the Critic network to generate the parameter update gradient; in the backpropagation stage, the L2 norm of the parameter update gradient is constrained by PyTorch's autograd mechanism to obtain the standardized gradient flow; and the KL divergence monitor is used to dynamically decay the learning rate of the standardized gradient flow to ensure training stability and obtain the KL divergence value; when the KL divergence value reaches the convergence threshold, the training is terminated and the trained Actor-Critic dual network is output;

[0065] It should be noted that the convergence threshold is defined based on the stability of KL divergence and its value range is [0.005, 0.02];

[0066] Next, the orthogonal state vector is input into the trained Actor-Critic dual network through forward propagation. The Actor network consists of three fully connected layers. The input layer linearly transforms the orthogonal state vector using linear projection to obtain a 128-dimensional feature vector. The hidden layer performs nonlinear mapping on the 128-dimensional feature vector through Layer Normalization and ReLU activation to generate 256-dimensional activation features. The output layer parameterizes the 256-dimensional activation features into action space through Gaussian parameter separation and Tanh activation to obtain the action mean.

[0067] The shared feature layer of the critic network performs feature enhancement on the 128-dimensional feature vector through linear projection or convolution to generate 256-dimensional value features; the output layer performs scalar value regression on the 256-dimensional value features through a one-dimensional linear layer to obtain the state estimated value;

[0068] Monte Carlo sampling is used to perform spatiotemporal joint sampling of action means and state estimation values ​​to obtain reward triplets, which are then fitted with a sliding window polynomial to obtain a reward time curve. Savitzky-Golay filtering is used to suppress noise and calculate first-order derivatives of the reward time curve to generate smooth gradient change values. The central difference method is used to extract the main direction of the smooth gradient change value point by point to generate gradient trajectory data.

[0069] In the second step, density calculation and spatial interpolation are performed based on the gradient trajectory data through kernel density estimation to generate a spatial density distribution map. In the specific operation, the gradient trajectory data is spatially decomposed by the kernel density estimation method to obtain the initial density distribution; the initial density distribution is spatially discretized using bilinear interpolation to generate a discrete density field; the discrete density field is frequency-smoothed using fast Fourier transform (FFT), and the denoised discrete density field is iteratively updated point by point using inverse distance weighting (IDW) to perform density recalculation; based on the calculated discrete density, the discrete density is locally enhanced through gradient ascent optimization to generate an optimized density field;

[0070] Then, the local extreme value detection algorithm is used to extract the peak points of the optimized density field to obtain the key density nodes. The Delaunay triangulation is used to construct the topological structure of the key density nodes to generate a density topology grid. The natural neighbor interpolation (NNI) is used to perform high-fidelity spatial interpolation on the density topology grid to obtain the density distribution surface. The density distribution surface is then visualized using color mapping (Viridis color spectrum) to generate a spatial density distribution map.

[0071] The third step is to use the dynamic meshing method to adjust the mesh density and control the deformation of the spatial density distribution map to obtain the mesh deformation parameters. In the specific operation, first, the dynamic meshing method is used to perform initial meshing on the spatial density distribution map to obtain the basic mesh units; the Sobel operator is used to assign weights to the basic mesh units through the convolution kernel to generate a mesh weight matrix; next, the weighted Lloyd relaxation algorithm is used to extract the mesh center points of the mesh weight matrix to obtain the optimized mesh nodes; the Gaussian integral method is used to sum the density integrals of the optimized mesh nodes, and the mesh density is adjusted by mesh subdivision and merging to generate a non-uniform density mesh;

[0072] It should be noted that the Sobel operator is defined based on the grid cell gradient calculation and has a value range of [0,1];

[0073] Next, bilinear interpolation is used to interpolate vertex coordinates of the non-uniform density grid to obtain the intermediate grid of the transition. The nearest neighbor search is used to match the feature points of the intermediate grid of the transition to generate matching control point pairs. The matching control point pairs are rigidly transformed using the least squares method to obtain the preliminary deformation field. The preliminary deformation field is deformed by fitting the displacement vector using a quadratic polynomial, and it is iteratively optimized using the stepwise relaxation method to generate the grid deformation parameters.

[0074] The fourth step is to dynamically segment the spatial density distribution map using a hierarchical dynamic segmentation algorithm based on the mesh deformation parameters to generate dynamic mesh layout parameters. Specifically, the hierarchical dynamic segmentation algorithm is used to extract the displacement field features of the mesh deformation parameters, and K-means clustering is used to perform deformation division on the displacement field features to generate deformation region markers. Based on the deformation region markers, the spatial density distribution map is segmented and extracted using a regional mask operation, and mesh segmentation is performed using Blender's subdivision tool to obtain multi-resolution mesh units. Secondly, a UV unfolding tool (such as Blender's Smart UV Project) is used to parametrically unfold the multi-resolution mesh units to generate dynamic mesh layout parameters.

[0075] Dynamic grid layout parameters can not only significantly improve computational efficiency, but also enhance rendering quality, achieving a balance between high performance and high fidelity.

[0076] S3. Input the dynamic grid layout parameters into the chaos prediction model, simulate the user movement pattern through the improved Lorenz equation, and extract the obstacle position through the RANSAC algorithm. At the same time, perform multi-dimensional feature mapping on the user movement pattern and obstacle position to generate a collision risk heat map.

[0077] The specific steps include:

[0078] The first step is to build a chaos prediction model. Specifically, first, call PyTorch's nn.Linear through the initialized fully connected layer, and use the Gaussian coupling mapping method to project the parameter space to obtain the primary coupling weights. The Adam optimizer is used to iterate the primary coupling weights to generate optimized coupling weights. Based on the optimized coupling weights, the coupling layer, evolution layer, and perception layer are dynamically weighted and integrated through sigmoid activation and gating mechanisms, and the constructed chaos prediction model is synchronously output.

[0079] Secondly, the constructed chaos prediction model is trained. In the specific operation, the dynamic grid layout parameters are divided into a training set and a validation set using the hierarchical time series segmentation method; the training set is spatially transformed and enhanced using random affine transformation (RandomAffine) to obtain enhanced training samples; the enhanced training samples are forward propagated and backward updated using the AMP trainer to obtain the preliminarily optimized chaos prediction model parameters; on the validation set, the SmoothL1Loss loss function is used to calculate the loss of the preliminarily optimized chaos prediction model parameters to obtain the validation loss value, and the cosine annealing is used to dynamically adjust the learning rate of the validation loss value. During the adjustment process, when the validation loss value reaches 10 consecutive epochs (training rounds) and does not drop below the loss threshold, the training is terminated, and the trained chaos prediction model is output simultaneously using the export tool (such as torch.jit.script);

[0080] It should be noted that the loss threshold is defined based on the convergence characteristics of the validation loss value of the validation set, and the value range is [1e-5, 1e-3];

[0081] In the second step, the dynamic grid layout parameters are input into the trained chaos prediction model through the PyTorch DataLoader interface. First, a coupling parameter matrix is ​​generated in the coupling layer. Specifically, multi-scale feature extraction is performed on the dynamic grid layout parameters through dynamic convolution kernels, and a 3×3 convolution kernel group is used to capture local and global features. Then, a channel attention mechanism is used to adaptively assign weights to local and global features, and the attention weights of each channel are generated through Sigmoid activation. Based on the attention weights of each channel, the Hadamard product is used to perform feature weighted fusion to obtain a weighted feature map. The weighted feature map is input into a 1×1 convolutional dimensionality reduction layer through a batch normalization layer for feature compression and dimension alignment to generate a coupling parameter matrix.

[0082] Secondly, user mobility pattern data is obtained in the evolution layer. Specifically, the improved Lorenz equation is used to fit the phase space trajectory of the coupling parameter matrix to obtain the three-dimensional dynamic trajectory. The three-dimensional dynamic trajectory is then used to perform nonlinear dynamic modeling using a fourth-order Runge-Kutta integrator. The three-dimensional dynamic trajectory is then used to perform fourth-order incremental calculations using the Butcher table coefficients to obtain four intermediate incremental terms. The four intermediate incremental terms are linearly combined using a weighted average operator to obtain a state vector sequence. The state vector sequence is then time-stepped and integrated using the step-size control method to construct the dynamic phase space trajectory and complete the nonlinear dynamic modeling.

[0083] It should be noted that the Butcher table coefficients refer to the parameter combination used to calculate the intermediate incremental terms in the Runge-Kutta integrator, and their value range is a rational number between [0, 1]. The weighted average operator refers to the coefficient set for weighted summation of the intermediate incremental terms, and its value range is a positive number such that the sum of the coefficients is 1.

[0084] Dynamic time warping is used to align the dynamic phase space trajectories, and the Mahalanobis distance algorithm is used to measure similarity to obtain a path similarity matrix. Path probabilities are calculated on the path similarity matrix using a forward-backward algorithm to generate an optimal path probability distribution. GMM clustering is used to simulate the multimodal distribution of the optimal path probability distribution to generate candidate pattern clusters. Dynamic time warping is used to perform temporal pattern matching on the candidate pattern clusters to obtain the optimal alignment path. A sliding window is used to uniformly resample the optimal alignment path to generate user mobility pattern data.

[0085] Next, the obstacle position is obtained in the perception layer. Specifically, the RANSAC algorithm is used to perform plane fitting on the user mobility pattern data to obtain ground parameters. The ground parameters are normalized by the normal vector normalization method to construct the ground equation. Based on the ground equation, the user mobility pattern data is segmented into non-ground point clouds: the ground equation calculates the vertical distance from each point in the user mobility pattern data to the ground through vector projection. The vertical distance from each point to the ground is highly filtered using the height threshold segmentation method to screen out non-ground point clouds. The statistical outlier removal method is used to check the number of neighboring points of the non-ground point cloud to remove non-ground point clouds in the user mobility pattern data. Voxel grid filtering is then used to perform uniform downsampling to generate obstacle candidate point clouds.

[0086] The obstacle candidate point cloud is separated by Euclidean clustering to obtain independent obstacle point clouds. The independent obstacle point cloud is bounded by the minimum bounding rectangle algorithm, and a 3D oriented bounding box is generated by the Graham scanning method. The 3D oriented bounding box is then associated with consecutive frames using nearest neighbor matching to obtain the obstacle trajectory. The obstacle trajectory is spatially located using the rigid body transformation method to obtain the global coordinate trajectory. The global coordinate trajectory is aligned to the coordinate system through time-synchronized interpolation to generate the obstacle position.

[0087] The third step is to fuse the user mobility pattern data and obstacle positions to generate a spatiotemporal coupling vector. The spatial coupling vector is then mapped to a spatial trajectory distribution using the phase space reconstruction method to generate a collision risk heat map. Specifically, the relative motion parameters of the user mobility pattern data and obstacle positions are first extracted using the relative motion calculation method. The relative motion parameters are then multi-dimensionally combined using the Cartesian product combination method to generate the spatiotemporal coupling vector.

[0088] Then, the phase space reconstruction method is used to reconstruct the trajectory of the spatial coupling vector to obtain the dynamic motion pattern. The dynamic motion pattern is smoothed and resampled using cubic spline interpolation, and the dynamic motion pattern is mapped to the spatial trajectory distribution. The spatial trajectory distribution is then divided into density regions using spatial density clustering to obtain density clustering results. Based on the density clustering results, the threshold segmentation algorithm is used to identify high-risk areas in the spatial trajectory distribution. For example, when the density clustering result exceeds the density threshold, it is identified as a high-risk trajectory. The high-risk trajectory is rendered in gradient colors using Matplotlib color mapping to generate a collision risk heat map.

[0089] It should be noted that the density threshold is defined based on the standard deviation σ of the spatial trajectory distribution, and its value range is [1.5σ, 3σ].

[0090] S4. Based on the collision risk heat map, high-risk areas are identified through the threshold segmentation algorithm, and an interactive control priority list is obtained. Based on the interactive control priority list, interactive adjustment instructions are generated through the NSGA-II algorithm.

[0091] The first step is to identify high-risk areas based on the collision risk heat map through the threshold segmentation algorithm and obtain an interactive control priority list. In the specific operation, first, the collision risk heat map is divided into high-risk areas through the threshold segmentation algorithm: when the area in the collision risk heat map exceeds the dynamic threshold, it is defined as a high-risk area and marked through the neighborhood connected domain scanning method to obtain a preliminary risk area marking map; the preliminary risk area marking map is filled with holes and smoothed with boundaries using the morphological closed operation algorithm to generate an optimized risk area binary map; the optimized risk area binary map is clustered using the connected domain marking algorithm to obtain a set of valid high-risk areas; the set of valid high-risk areas is dynamically binarized through dynamic Otsu threshold segmentation to generate a binary mask;

[0092] It should be noted that the dynamic threshold is defined based on the regional grayscale standard deviation (σ) of the collision risk heat map, and the value range is [1.5σ, 3σ];

[0093] Secondly, K-means clustering is used to extract rasterized features from the binary mask, and principal component analysis is used to compress the feature space to obtain the feature vector after dimensionality reduction. The t-SNE manifold learning algorithm is used to perform nonlinear feature visualization on the feature vector after dimensionality reduction to generate a two-dimensional feature distribution map. Gaussian filtering is used to perform probability density clustering on the two-dimensional feature distribution map to obtain a set of potential feature clusters. The weighted centroid calculation method is used to extract the center point of the potential feature cluster set, and the spatial coordinate back projection is used to generate the coordinate set of the high-risk area.

[0094] In the second step, the fused hierarchical analysis method is used to assign weights to the high-risk area coordinate set. Based on the assigned weights, the high-risk area coordinate set is weighted linearly combined to generate a preliminary priority ranking table. The weighted average method is used to score-integrate the preliminary priority ranking table to obtain a weighted priority ranking table. The sliding window method is used to divide the weighted priority ranking table into multiple windows, and time series smoothing is performed in each window. The structured matrix transformation method is used to convert the smoothed weighted priority ranking table into triplets. All triplets are arranged in time series order through row-first stacking to finally form a priority scoring matrix.

[0095] Secondly, the priority rating matrix is ​​reduced in dimension through principal component analysis to obtain a simplified rating matrix. K-means clustering is used to group and classify the simplified rating matrix to generate priority grouping results. Based on the priority grouping results, a quick sorting algorithm is used to sort within the group to obtain ordered priority groups. The ordered priority groups are then globally integrated and sorted using a hierarchical sorting method to obtain a comprehensive priority sequence. The comprehensive priority sequence is simultaneously annotated with a timestamp tag for timeliness, and a binary search method is used for rapid retrieval to obtain a list of interactive control priorities.

[0096] The interactive regulation priority list can not only automatically recommend the dangerous areas that need to be processed with the highest priority, dynamically adjust the processing order according to the real-time situation, making safety prevention and control more intelligent and efficient;

[0097] In the third step, first, perform multi-objective trade-offs on the interactive regulation priority list through the NSGA-II algorithm to obtain a comprehensive scoring set. In specific operations, in the DEAP framework of Python, call the NSGA-II multi-objective optimization algorithm to divide the interactive regulation priority list into an initial population composed of multiple individuals; use the simulated binary crossover operator to perform gene recombination on the initial population to obtain a candidate solution set; use the polynomial mutation operator to enhance the diversity of the candidate solution set to generate a Pareto front solution set; stratify the Pareto front solution set through non-dominated sorting to generate a comprehensive scoring set;

[0098] It should be noted that the simulated binary crossover operator is defined based on the maximum difference amount between the parent individuals of the initial population; the polynomial mutation operator is defined based on the width of the feasible region of the initial population;

[0099] Secondly, for the obtained comprehensive scoring set, adopt a three-level threshold division strategy based on statistical distribution to divide the risk levels: divide the comprehensive scoring set into windows through the sliding window method and perform dynamic discretization within each window to obtain the standard deviation (P) of the comprehensive scoring set; define the three-level thresholds based on the 3σ principle of the standard deviation of the comprehensive scoring set. For example, set the first-level threshold as 2P. When the comprehensive score > 2P, it is classified as a first-level risk, and a first-level risk indicates a major safety hazard and requires emergency response and priority handling; set the second-level threshold as P~2P. When the comprehensive score is within the range of P~2P, it is classified as a second-level risk, and a second-level risk indicates a potential risk that needs attention and requires the formulation of planned prevention and control measures; set the third-level threshold as P. When the comprehensive score < P, it is classified as a third-level risk, and a third-level risk indicates a low risk that can be monitored and requires regular inspection and observation;

[0100] In the fourth step, use the entropy weight method to perform instruction weight mapping on the risk levels to generate interactive adjustment instructions. In specific operations, use the entropy weight method to assign weights to the risk levels to obtain a risk level weight vector; perform linear transformation on the risk level weight vector through the Min-Max normalization method to generate an instruction intensity set; use the weight mapping table to perform instruction weight mapping on the instruction intensity set to obtain a basic adjustment instruction set; use a time-series sliding window to smooth the basic adjustment instruction set and use an instruction encoder (ASCII protocol encapsulation) to format the instructions to generate interactive adjustment instructions.

[0101] S5. Input the interaction adjustment instructions into the XR rendering engine through the ROS middleware, collect real-time interaction data, use the triple exponential smoothing method to analyze the behavior patterns of the real-time interaction data, and generate an exhibition interaction report through the Tableau engine.

[0102] The first step is to match the interactive adjustment instructions to the XR rendering protocol format through the ROS middleware, generate an interactive instruction stream, and input it into the XR rendering engine. Specifically, the interactive adjustment instructions are serialized and encapsulated through the ROS middleware to obtain standard ROS message packets. The standard ROS message packets are matched to the XR rendering protocol format through the XR protocol converter (such as Protobuf3.0) to generate XR basic data frames. The Zstandard compression algorithm is used to optimize the bandwidth of the XR basic data frames to generate an interactive instruction stream. The interactive instruction stream is normalized into a spatial coordinate system using the OpenXR API interface to generate an interactive event stream that can be parsed by the XR engine. The interactive event stream is input into the XR rendering engine using the Runtime layer of Unity XR.

[0103] In the second step, the XR rendering engine drives dynamic exhibits and spatial navigation anchors through interactive instruction streams to interact with the exhibition. Specifically, the XR rendering engine uses the LZ4 decompression algorithm to parse and decode the interactive instruction stream, generating a 3D spatial interaction parameter set. It then uses the ARCore spatial mapping API to bind the 3D spatial interaction parameter set to physical coordinates and obtain a set of interactive anchor points. Finally, it uses Shader Graph visual programming to generate real-time visual effects for the interactive anchor points and obtain dynamic rendering instructions.

[0104] Based on dynamic rendering instructions, VFX Graph (visual effects tool) drives dynamic exhibits and spatial navigation anchors for exhibition interaction. For example, dynamic rendering instructions use ARCore's depth perception API to drive VFX Graph to detect the distance between the user and the exhibit. When the distance is detected to be less than one meter, VFX Graph triggers particle effects to simulate the virtual reorganization of the dynamic exhibits and sound and light feedback. At the same time, navigation anchors placed on the ground guide AR glasses to project golden arrows to guide users to the next exhibition area. As users follow the golden arrows, the navigation anchors adjust the arrow's direction in real time to avoid collisions with other visitors until the user completes the tour, completing the dynamic interaction of the exhibition.

[0105] It should be noted that dynamic exhibits are physical exhibits equipped with XR interactive components, such as robotic arms, projection screens, and audio and video equipment. Spatial navigation anchors refer to spatial positioning markers that play a role in precise spatial coordinate binding and persistent tracking in large-scale exhibition interactions.

[0106] During the interactive exhibition process, the IMU inertial measurement unit (IMU) is used to collect real-time interactive data. Specifically, the IMU collects user posture data in real time, and uses the Madgwick AHRS algorithm for posture calculation, outputting stable quaternion posture data. In Unity, the PhysX physics engine converts the quaternion posture data into Euler angles and performs motion interpolation to obtain smooth posture information. Dynamic time warping is used to match the smooth posture information to motion trajectories, generating real-time interactive data.

[0107] The third step is to use triple exponential smoothing to separate the trends of real-time interaction data, and analyze the behavior patterns through density clustering algorithm to generate a user behavior label matrix. Specifically, triple exponential smoothing performs multi-period weighted smoothing on real-time interaction data, and uses a sliding window to perform first-order gradient calculation to obtain a joint feature vector; the STL seasonal decomposition method is used to separate the trends of the joint feature vector to generate a trend time series matrix; the KL divergence detector is used to mark the behavior boundaries of the trend time series matrix to obtain a behavior segment tag set; the t-SNE dimensionality reduction is used to perform two-dimensional manifold embedding on the behavior segment tag set to generate an interaction behavior feature set;

[0108] The density clustering algorithm is used to perform spatial density scanning on the interactive behavior feature set to generate a contour map. The contour map is clustered and decomposed using the silhouette coefficient optimization method to obtain optimized density partitions. The optimized density partitions are spatially partitioned using the spectral clustering repartitioning method to generate stable behavior partitions. The stable behavior partitions are decomposed into behavioral patterns using the non-negative matrix decomposition method to generate basic behavior pattern basis vectors. The basic behavior pattern basis vectors are used for real-time behavior classification using cosine similarity matching to generate a user behavior label matrix.

[0109] The fourth step is to use the Tableau engine to generate an exhibition interaction report. Specifically, the LOD dynamic rendering strategy is used to aggregate the user behavior tag matrix in the time and space dimensions to obtain a time-sharing and partitioned behavior statistics table. The Tableau Prep data cleaner is used to fill missing values ​​and correct outliers in the time-sharing and partitioned behavior statistics table to generate a standardized behavior data set.

[0110] Based on the standardized behavior data set, the 3σ outlier detection algorithm is used to issue abnormal pattern warnings. For example, the standardized behavior data set is divided into windows of equal time using a sliding time window splitter, and the mean μ and standard deviation j of the dwell time and interaction frequency are calculated in the window using the Welford algorithm. A primary warning is triggered when the value of a single user behavior label matrix exceeds the range of μ±3j; an intermediate warning is triggered when the value of the user behavior label matrix exceeds the range of μ±4j for three consecutive times; and an advanced warning is triggered when the value of the user behavior label matrix exceeds the range of μ±5j for 5 consecutive minutes.

[0111] Real-time interactive data and abnormal pattern data are aligned and integrated through a cross-source data blending mechanism to generate an interactive data cube; the interactive data cube is multi-dimensionally aggregated using the pivot function in Tableau to obtain an interactive KPI matrix; the interactive KPI matrix is ​​rendered in multiple views using the ableau visualization tool (such as the VizQL language) to generate an interactive report component set; the interactive report component set is automatically laid out through the Tableau dashboard to generate an exhibition interactive report.

[0112] This embodiment also provides a computer device, which is suitable for the exhibition interaction method of large-space XR based on LBE technology, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the exhibition interaction method of large-space XR based on LBE technology proposed in the above embodiment.

[0113] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0114] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the exhibition interaction method for large-space XR based on LBE technology as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0115] In summary, the present invention significantly improves the prediction accuracy of users' mobility patterns in complex dynamic environments through: a chaotic prediction model and an improved Lorenz equation, making the interactive experience smoother and more natural; at the same time, it uses the RANSAC algorithm combined with multi-dimensional feature mapping to generate a detailed collision risk heat map, which enhances the ability to recognize obstacles in rapidly changing environments and the collision warning effect, thereby ensuring user safety.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for interactive exhibition in a large space based on LBE technology XR, characterized by: include, The user position coordinates and the environment 3D point cloud data are aligned in time and space using the ICP algorithm, and noise is filtered using Gaussian filtering to form a user position matrix. The user position matrix is ​​used to generate a spatial density distribution map through the PPO algorithm, and the spatial density distribution map is dynamically meshed to generate dynamic mesh layout parameters; The dynamic grid layout parameters are input into the chaotic prediction model. The user movement pattern is simulated by the improved Lorenz equation, and the obstacle positions are extracted by the RANSAC algorithm. At the same time, multi-dimensional feature mapping is performed on the user movement pattern and obstacle positions to generate a collision risk heat map. Based on the collision risk heat map, a threshold segmentation algorithm is used to identify high-risk areas, obtain an interactive control priority list, and generate interactive adjustment instructions using the NSGA-II algorithm based on the interactive control priority list. The interaction adjustment instructions are input into the XR rendering engine through the ROS middleware, and real-time interaction data is collected. The behavioral pattern of the real-time interaction data is analyzed using the triple exponential smoothing method, and an exhibition interaction report is generated through the Tableau engine.

2. The LBE technology-based XR large-space exhibition interaction method according to claim 1, characterized in that: The specific steps of forming the user position matrix are as follows: The ICP algorithm is used to map the user's location coordinates and the environment's 3D point cloud data into a unified coordinate system space and perform point cloud matching. The NTP protocol is used to perform spatiotemporal alignment simultaneously to generate a synchronized point cloud dataset. Gaussian filtering uses convolution kernels to perform spatial smoothing and neighborhood enhancement on the synchronized point cloud dataset, identifies outliers, uses the SOR algorithm to filter noise based on the outliers, and converts the denoised point cloud dataset into a user position matrix through voxel index mapping.

3. The LBE technology-based XR large-space exhibition interaction method according to claim 2, characterized in that: The specific steps of generating dynamic grid layout parameters are as follows: The PPO algorithm is used to perform gradient updates on the user position matrix to obtain gradient trajectory data. Based on the gradient trajectory data, density calculation and spatial interpolation are performed through kernel density estimation to generate a spatial density distribution map. The dynamic grid division method is used to adjust the grid density and control the deformation of the spatial density distribution map to obtain the grid deformation parameters; according to the grid deformation parameters, the hierarchical dynamic segmentation algorithm is used to dynamically segment the spatial density distribution map to generate dynamic grid layout parameters.

4. The LBE technology-based XR large-space exhibition interaction method according to claim 1, characterized in that: The specific steps for generating the collision risk heat map are as follows: In the PyTorch framework, the coupling layer, evolution layer, and perception layer are integrated through the Gaussian coupling mapping method to construct a chaos prediction model; The coupling layer uses dynamic convolution kernels to extract features and assign weights to dynamic grid layout parameters to obtain the coupling parameter matrix; The evolution layer uses the improved Lorenz equation to perform nonlinear dynamic modeling of the coupling parameter matrix and performs path probability calculation and pattern matching through dynamic time warping to obtain user mobility pattern data. The perception layer uses the RANSAC algorithm to perform point cloud segmentation on the user's mobility pattern data to obtain the obstacle candidate point cloud, and then uses DBSCAN density clustering to perform bounding box fitting and spatial positioning to obtain the obstacle position; The user mobility pattern data and obstacle positions are fused to generate a spatiotemporal coupling vector, and the spatial coupling vector is mapped into a spatial trajectory distribution using the phase space reconstruction method to generate a collision risk heat map.

5. The LBE technology-based XR large-space exhibition interaction method according to claim 1, characterized in that: The specific steps for obtaining the interaction control priority list are as follows: The collision risk heat map is dynamically binarized using a threshold segmentation algorithm to obtain a binary mask. K-means clustering is used to extract the center point of the binary mask to generate a coordinate set of high-risk areas. The fusion analytic hierarchy process is used to assign weights to the high-risk area coordinate set and generate a priority scoring matrix; The priority score matrix is ​​sorted through principal component analysis to obtain the interaction regulation priority list.

6. The LBE technology-based XR large-space exhibition interaction method according to claim 5, characterized in that: The interactive adjustment instructions are generated by the NSGA-II algorithm. The specific steps are as follows: The NSGA-II algorithm is used to perform multi-objective trade-offs on the interactive control priority list to obtain a comprehensive score set. The comprehensive score set is divided into primary, secondary, and tertiary risk levels according to the three-level threshold. The entropy weight method is used to map the risk levels to instruction weights and generate interactive adjustment instructions.

7. The LBE technology-based XR large-space exhibition interaction method according to claim 1, characterized in that: The interactive adjustment instructions are input into the XR rendering engine through the ROS middleware, and real-time interactive data is collected. The specific steps are as follows: The interactive adjustment instructions are matched to the XR rendering protocol format through the ROS middleware to generate an interactive instruction stream; the interactive instruction stream is input into the XR rendering engine using the OpenXRAPI interface; The XR rendering engine drives dynamic exhibits and spatial navigation anchors through interactive instruction streams for exhibition interaction, and simultaneously uses the IMU inertial measurement unit to collect real-time interaction data.

8. The LBE technology-based XR large-space exhibition interaction method according to claim 1, characterized in that: The specific steps for generating an exhibition interaction report through the Tableau engine are as follows: Triple exponential smoothing is used to separate the trends of real-time interaction data and obtain the interaction behavior feature set. The interaction behavior feature set is then analyzed for behavior patterns using a density clustering algorithm to generate a user behavior label matrix. Based on the Tableau engine, the LOD dynamic rendering strategy is used to issue abnormal pattern warnings for the user behavior label matrix, obtain abnormal pattern data, integrate real-time interaction data and abnormal pattern data through a cross-source data mixing mechanism, and generate an exhibition interaction report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the exhibition interaction method based on LBE technology XR large space described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the exhibition interaction method based on LBE technology XR large space according to any one of claims 1 to 8 are implemented.

Images