Immersive space virtual-reality interaction method based on metaverse

Abstract

The present invention discloses an immersive space virtual-reality interaction method based on metaverse, which specifically relates to the field of augmented reality indoor navigation, and is used to solve the problem of disconnection between dynamic human flow and SLAM map. The method generates a real-time human flow heat map through collection and denoising aggregation, and realizes accurate alignment of human flow data with SLAM map by combining timestamp synchronization and spatial point matching. Then, the indexes such as regional depth density and trajectory coherence manifold are used to identify and remove ghost interference, and a dynamic environment map is generated after mapping the three-dimensional coordinates. The navigation path is dynamically optimized through weighted graph search and re-planning mechanism, and human-computer interaction is performed with the help of AR coordinate conversion and immersive virtual arrows, so as to cover the diverse structures and uncertain human flow changes of indoor scenes, avoid travel obstruction or erroneous detour caused by human flow congestion, reduce the influence of environmental structural defects caused by SLAM errors or dynamic objects, and thus ensure the navigation stability and reliability in large-scale indoor spaces.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality indoor navigation, and more specifically, to an immersive space virtual-reality interaction method based on a metaverse. Background Art

[0002] In the augmented reality indoor navigation scenario, the static environment map (update frequency 1-5Hz) constructed by the SLAM (simultaneous localization and mapping) system through lidar or visual sensors and the high-frequency dynamic crowd flow data (update frequency 10-30Hz) collected by Wi-Fi probes are disconnected due to the separation of time and space references. Specifically, the following are the manifestations: 1. The mismatch in data update frequency makes the SLAM map unable to respond to sudden crowd gatherings (such as corridor congestion) in real time, and the virtual navigation path still points to the blocked area; 2. The coordinate system is not rigidly aligned, resulting in the inability to accurately map the dynamic crowd flow heat map to the SLAM three-dimensional space coordinates, and the navigation guidance at the intersection of multiple paths deviates from the physical space; 3. The residual interference of dynamic segmentation causes the short-term stay area in the SLAM point cloud to be misjudged as a semi-static obstacle (ghost), which is inconsistent with the crowd flow data.

[0003] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an immersive space virtual-reality interaction method based on the metaverse, which generates a real-time crowd heat map through collection and denoising aggregation, and realizes the precise alignment of crowd data with the SLAM map by combining timestamp synchronization and spatial point matching, and then uses indexes such as regional depth density and trajectory continuity manifold to identify and remove ghost interference, and generates a dynamic environment map after mapping the three-dimensional coordinates. The navigation path is dynamically optimized through weighted graph search and re-planning mechanism, and human-computer interaction is performed with the help of AR coordinate conversion and immersive virtual arrows, covering the diverse structures and uncertain crowd changes of indoor scenes, avoiding travel obstructions or erroneous detours caused by crowd congestion, and reducing the impact of environmental structural defects caused by SLAM errors or dynamic objects, thereby ensuring the navigation stability and reliability in large-scale indoor spaces to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An immersive space virtual-reality interaction method based on the metaverse includes the following steps:

[0007] S1: Use high-frequency sensor networks to collect dynamic crowd flow data and generate crowd flow heat maps through denoising and aggregation;

[0008] S2: Based on timestamp synchronization and spatial point matching, the crowd flow heat map and SLAM map are spatiotemporally aligned;

[0009] S3: Extract the density and trajectory coherence of the area from the SLAM map as key features, input these features into the pre-trained classification model to determine and remove the ghost interference area, and then map the aligned heat map point by point according to the three-dimensional coordinates to generate a dynamic environment map;

[0010] S4: Calculate the optimal navigation path in real time based on the dynamic environment map and user location, and present an immersive interactive experience with virtual arrows in the AR interface.

[0011] In a preferred embodiment, step S1 includes the following contents:

[0012] The high-frequency sensor network deploys multiple sensor nodes to capture the dynamic flow data of people in the environment in real time, denoise it, and aggregate the denoised data to generate a real-time heat map of the flow of people.

[0013] In a preferred embodiment, the polymerization process is as follows:

[0014] Assume the spatial position is The data for each time period is ,in Represents the time series number, and the aggregation process is performed by the following formula: ,in, Indicates at time Time, Location The accumulated heat value at For in time Dynamic flow data of people at the corresponding location at the time, It represents the total data collection points, and presents the aggregated data as a heat map through color mapping, which ultimately generates a real-time and accurate heat map of pedestrian flow.

[0015] In a preferred embodiment, step S2 includes the following contents:

[0016] S2.1, first synchronize the timestamps of the dynamic crowd flow heat map and SLAM map data;

[0017] S2.2, spatially align the crowd flow heat map and the SLAM map through coordinate transformation technology; if the SLAM map uses coordinate system, and the flow data is , then you need to use affine transformation or rotation matrix to map the coordinates and get the corresponding three-dimensional coordinates ; Divide the three-dimensional coordinate points of the SLAM map into grids and set a spatial grid , the same gridding process is performed on the heat map of human flow; each heat map point is mapped to the corresponding grid of the SLAM map by matching the overlapping areas of the grids, thus completing the precise alignment of the spatial points;

[0018] S2.3, after completing the timestamp synchronization and spatial point matching, perform time-space alignment. For each collected time point, the corresponding three-dimensional coordinates and cumulative heat value Matching is done based on time and space locations.

[0019] In a preferred embodiment, step S3 includes the following contents:

[0020] Key features include the regional depth density index and the trajectory coherence manifold index.

[0021] In a preferred embodiment, the process of obtaining the regional depth density index is as follows:

[0022] Select the space window in the SLAM map To cover the coordinates A certain range around, and count the number of occupied points and their distribution, first define the local occupancy measure : ,in Indicates location Is it an occupied point identified by SLAM? The value range is 0,1. If it is an occupied point, it is close to 1, otherwise it is close to 0; introduce density fluctuation Measures the stability of the local occupancy distribution and defines ,in For Window Inside The average value of Represents the number of pixels or grid cells within the window range; after completing the calculation of the local occupancy measure and density fluctuation, the regional depth density index is defined: .

[0023] In a preferred embodiment, the process of obtaining the trajectory coherence manifold index is:

[0024] In coordinates Collect SLAM trajectory points around , the local velocity vector is calculated based on its time sequence , first define the direction consistency : ,in Indicates The velocity vector of the trajectory point Indicates that in the SLAM map, the coordinates The total number of trajectory points captured in the area; a continuous tensor field is introduced and quantized in a simplified form as , to measure the temporal connection between trajectory points, define ,in Represents the difference between adjacent trajectory vectors; couples the directional consistency with the continuity tensor to define the trajectory coherence manifold index: .

[0025] In a preferred embodiment, after the calculation of the regional depth density index and the trajectory coherence manifold index is completed, the coordinates of each region in the SLAM map are Calculation judgment function To identify ghost interference; according to the pre-set abnormal range, if the judgment value falls within the abnormal range, the coordinates are determined The ghost area is removed and the coordinates in the heat map are transformed according to the coordinate transformation relationship. Projection to SLAM 3D coordinates ,Then the remaining effective thermal information is superimposed with the SLAM static structure to generate a dynamic environment map.

[0026] In a preferred embodiment, step S4 includes the following contents:

[0027] The dynamic environment map is first discretized into a graph structure consisting of nodes and edges. The nodes represent key locations or grid centers in the map, and the edges represent accessible paths between nodes. A cost function is defined on each edge to quantify the difficulty of passage, and finally a weighted graph that integrates the flow of people is constructed. On the above weighted graph, a heuristic search algorithm is used to calculate the optimal path. The calculated optimal path usually includes a series of key nodes or inflection points, whose three-dimensional coordinates are Align with the coordinate system of the AR display device and project it into the augmented reality space with the help of the coordinate transformation matrix; then render virtual arrows or direction marks in the AR interface to merge them with the real scene. When the device posture or viewing angle changes, the arrows will be updated accordingly.

[0028] The technical effects and advantages of the immersive space virtual-reality interaction method based on the metaverse of the present invention are as follows:

[0029] Through high-frequency sensor network acquisition and denoising aggregation, real-time crowd heat map is generated. Combined with timestamp synchronization and spatial point matching, accurate alignment of crowd data and SLAM map is achieved. Then, the regional depth density and trajectory coherence manifold indexes are used to identify and remove ghost interference. After mapping the three-dimensional coordinates, a dynamic environment map is generated. Then, the navigation path is dynamically optimized through weighted graph search and replanning mechanism, and human-computer interaction is carried out with the help of AR coordinate conversion and immersive virtual arrows, covering the diverse structures and uncertain crowd changes of indoor scenes. This solution combines high-concurrency sensor data with dynamic SLAM processing flow, improves the accuracy of path planning and the real-time and immersiveness of navigation guidance, avoids travel blockage or wrong detour caused by crowd congestion, reduces the impact of environmental structural defects caused by SLAM errors or dynamic objects, and ultimately enhances the accuracy, flexibility and autonomy of indoor navigation. At the same time, it integrates crowd thermal information and visual interaction needs, presents a more intuitive and efficient indoor navigation method for end users, and strengthens the synergy between environmental understanding and path optimization, thereby ensuring navigation stability and reliability in large-scale indoor spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the immersive space virtual-reality interaction method based on the metaverse of the present invention;

[0031] Figure 2 This is a schematic diagram of step S2 of the immersive space virtual-reality interaction method based on the metaverse of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Embodiment 1: Figure 1 The present invention provides an immersive space virtual-reality interaction method based on the metaverse, including:

[0034] S1: Use high-frequency sensor networks to collect dynamic crowd flow data, and generate crowd flow heat maps through denoising and aggregation.

[0035] S2: Based on timestamp synchronization and spatial point matching, the crowd flow heat map and the SLAM map are temporally and spatially aligned.

[0036] S3: The density and trajectory coherence of the area are extracted from the SLAM map as key features, and these features are input into the pre-trained classification model to determine and remove the ghost interference area. The aligned heat map is then mapped point by point according to the three-dimensional coordinates to generate a dynamic environment map.

[0037] S4: Calculate the optimal navigation path in real time based on the dynamic environment map and user location, and present an immersive interactive experience with virtual arrows in the AR interface.

[0038] In the augmented reality (AR) indoor navigation system, how to achieve real-time synchronization between virtual navigation guidance and actual crowd flow dynamics is the key to improving user experience. The current SLAM technology cannot respond to dynamic changes in the environment in real time, resulting in a disconnect between the virtual path and the actual congestion situation. To solve this problem, it is first necessary to collect crowd flow data in real time through a high-frequency sensor network, and combine it with advanced data processing technology to generate an accurate dynamic crowd flow heat map as the basis for subsequent spatiotemporal alignment and path planning. The core of this step is how to accurately collect dynamic crowd flow data, denoise and aggregate it, and form a real-time and accurate heat map.

[0039] SLAM (Simultaneous Localization and Mapping) map is a spatial representation constructed in real time in an unknown environment by sensors (such as lidar, camera, IMU, etc.), while determining the position of the sensor itself in the space. SLAM map includes an accurate description of the spatial layout of static objects in the environment, the location of obstacles and their three-dimensional geometric features, usually stored in the form of raster maps, point clouds or sparse feature points. The map relies on sensor data and motion models, and is continuously updated through recursive estimation and nonlinear optimization techniques to ensure high accuracy of environmental modeling and self-positioning in dynamic scenes, thereby providing a spatial information foundation for applications such as navigation, path planning and augmented reality.

[0040] Step S1 includes the following contents:

[0041] The high-frequency sensor network deploys multiple sensor nodes to capture the dynamic changes of people in the environment in real time. Common sensors include Wi-Fi probes, infrared sensors, Bluetooth positioning devices, and high-frequency laser radars. These sensors can continuously capture information about human flow and feed it back to the central system. In this process, the density and coverage of sensor nodes need to be adjusted according to the actual environment to ensure the comprehensiveness and accuracy of human flow data collection.

[0042] For example, a Wi-Fi probe can infer the movement trajectory and dwell time of a device by monitoring the Wi-Fi signal strength and the device's timestamp. Infrared sensors capture the dynamic changes of a person's presence by sensing changes in heat sources. Each sensor node establishes a transmission link through wireless communication, and uploads the captured signal data in real time to form a high-speed dynamic acquisition network.

[0043] In the process of collecting dynamic crowd flow data, due to factors such as environmental noise, sensor errors, signal interference, etc., the collected raw data may contain a large number of abnormal fluctuations or errors. In order to ensure the accuracy of subsequent processing, the raw data needs to be denoised first.

[0044] Data denoising can be performed by following the steps below:

[0045] 1) Time series filtering: Smoothing the time series data collected by each sensor node. Common denoising methods include Kalman filtering and median filtering. Kalman filtering can estimate the current state based on the state of the previous moment and dynamically adjust the weight of noise interference, thereby effectively filtering out unnecessary fluctuations.

[0046] 2) Spatial denoising: Jointly denoise the spatial data of multiple sensor nodes to eliminate errors caused by signal attenuation or occlusion in local areas. Based on the positional relationship of the spatial layout of sensors, spatial weighted average or spatial collaborative filtering methods can be used for optimization to avoid isolated noise points generated in specific areas.

[0047] Through the above method, the accuracy and consistency of collected data can be significantly improved, laying the foundation for subsequent aggregation and analysis.

[0048] The denoised data needs to be further aggregated to generate a real-time heat map of human traffic. The aggregation process integrates the data from different sensors to generate a unified space-time heat map.

[0049] 1) Data aggregation: For dynamic data collected in different time periods, it is necessary to classify them uniformly according to the spatial position (e.g., two-dimensional coordinates) and timestamp of each data point. By setting a spatial grid, the position of each person collected by each sensor (e.g., the strength and position of the Wi-Fi positioning signal) is mapped to the corresponding grid of the spatial grid. The population density data of the area will be accumulated in each grid, and the degree of dynamic change of the area can be expressed by simple summation or weighted average.

[0050] Assume the spatial position is The data for each time period is ,in Represents the time series number, and the aggregation process is performed by the following formula: ,in, Indicates at time Time, Location The accumulated heat value at For in time Dynamic flow data of people at the corresponding location at the time, Indicates the total data collection points.

[0051] 2) Heat map visualization: The aggregated data is presented as a heat map through color mapping. In this process, color scales can be used to represent different density areas, such as red for high-density areas and blue for low-density areas, to intuitively display the real-time distribution of people flow.

[0052] The heat map generated by this aggregation can accurately reflect the dynamic flow of people at each moment, providing precise data support for subsequent spatiotemporal alignment and path planning.

[0053] By collecting dynamic crowd flow data through a high-frequency sensor network, denoising and aggregating the data, a real-time and accurate crowd flow heat map is generated, providing a reliable basis for subsequent spatiotemporal alignment and navigation path adjustment. In this process, the real-time and accuracy of data collection are crucial, while denoising and aggregation technology ensures the accuracy and reliability of the data, avoiding the problem of virtual guidance being out of touch with the actual scene.

[0054] In augmented reality indoor navigation, the real-time updated dynamic crowd heat map and the static SLAM map must be accurately aligned in time and space to ensure that the virtual guidance is consistent with the actual environment. Currently, although SLAM technology can provide environmental maps, its update frequency is low and cannot reflect the dynamic changes in crowd flow. In order to achieve efficient navigation optimization, the dynamic crowd flow data and SLAM map must be synchronized with the spatial points through precise timestamps to eliminate the time and space differences and ensure that the virtual path accurately reflects the actual situation in a complex environment. The core of this step is how to handle the synchronization of two-dimensional information in time and space, and then complete high-precision time and space alignment.

[0055] like Figure 2 As shown, step S2 includes the following contents:

[0056] S2.1, in order to ensure that the dynamic crowd heat map and SLAM map data can accurately correspond, it is necessary to first synchronize the timestamps of the dynamic crowd heat map and SLAM map data. The SLAM system and dynamic sensors usually have their own collection cycles, and the time synchronization error between the two may affect the effect of data fusion. The time synchronization process can be performed in the following ways:

[0057] 1) Time alignment method: by obtaining the timestamp of SLAM map update and the timestamp of the crowd flow data collection , adjust the time difference between the two. , then the heat map data is time-compensated according to the time difference, so that all the crowd data are aligned with the time of SLAM map update.

[0058] The synchronization process is specifically manifested as follows: ,in It is the unified timestamp after synchronization.

[0059] 2) Interpolation method: If the timestamp of the crowd flow data collection is significantly different from the update frequency of the SLAM map, the missing time period can be estimated by linear interpolation. The interpolation algorithm can fill the gaps in the time series and ensure the continuity of the data flow.

[0060] S2.2, in the process of spatiotemporal alignment of crowd heat map and SLAM map, spatial point matching is the key link. SLAM map usually uses a three-dimensional coordinate system to describe the static environment, while dynamic crowd flow data often comes from a two-dimensional plane (such as the coordinates of Wi-Fi probes on the ground plane). Therefore, it is necessary to align the crowd heat map and SLAM map in space through coordinate transformation technology.

[0061] 1) Coordinate transformation: First determine the transformation relationship between the coordinate system of the SLAM map and the coordinate system of the area where the pedestrian flow data is collected. coordinate system, and the flow data is , then you need to use affine transformation or rotation matrix to map the coordinates and get the corresponding three-dimensional coordinates .

[0062] 2) The conversion formula is: ,in, is the transformed three-dimensional coordinate, is a two-dimensional crowd flow data point, matrix is the affine transformation matrix.

[0063] 3) Spatial grid matching: Divide the three-dimensional coordinate points of the SLAM map into grids and set a spatial grid , the same gridding process is performed on the heat map of human flow. Each heat map point is mapped to the corresponding grid of the SLAM map by matching the overlapping areas of the grids, thus completing the precise alignment of the spatial points.

[0064] S2.3, after completing timestamp synchronization and spatial point matching, the next step is to perform spatiotemporal alignment. This process requires combining time synchronization and spatial matching information to accurately combine dynamic crowd flow data with SLAM maps.

[0065] 1) Space-time fusion: Based on the unified timestamp and spatial coordinate system, the synchronized dynamic flow data is mapped to the corresponding position of the SLAM map. , the corresponding three-dimensional coordinates and cumulative heat value Matching is performed according to the time and space positions to generate a comprehensive time and space data set, which includes multi-dimensional information of time, space and crowd density.

[0066] 2) Resampling and optimization: In order to avoid the influence of inconsistent time steps or spatial position alignment errors, the time-space alignment process can be optimized through resampling methods. Use high-precision spatial interpolation technology to adjust the spatial distribution so that the changes in the dynamic heat map are more consistent with the terrain changes of the SLAM map.

[0067] In this process, the resampling method used can be selected according to the spatial density and time step, such as using Lagrange interpolation or spline interpolation to smooth spatial data to improve the accuracy of spatiotemporal alignment.

[0068] Step S2 achieves precise spatiotemporal alignment of dynamic crowd flow data with SLAM maps through timestamp synchronization and spatial point matching. Timestamp synchronization ensures the uniformity of data updates, and spatial point matching eliminates the impact of inconsistent coordinate systems. Spatiotemporal alignment provides reliable basic data for subsequent environmental map generation, ensures the seamless integration of dynamic environmental information and static environmental maps, and lays a solid foundation for the accuracy and real-time performance of AR navigation guidance.

[0069] Step S3 includes the following contents:

[0070] Key features include the regional depth density index and the trajectory coherence manifold index.

[0071] When identifying ghost regions, evaluating the distribution and local density of occupied points in the region helps to accurately determine the possibility of incorrect construction. To this end, the regional depth density index is introduced , which is calculated in stages using the following two formulas.

[0072] The process of obtaining the regional depth density index is as follows:

[0073] Select the space window in the SLAM map To cover the coordinates A certain range around the occupancy point is calculated, and the number and distribution of the occupancy points are counted. In order to avoid the deviation caused by the simple superposition of the occupancy points, a measure of the local distribution form is introduced. First, the local occupancy measure is defined : ,in Indicates location Is it an occupied point identified by SLAM? The value range is 0,1. If it is an occupied point, it is close to 1, otherwise it is close to 0. This integral reflects the coordinate The overall distribution scale of the surrounding local occupation points.

[0074] It is difficult to distinguish uniform distribution from discrete distribution simply by relying on occupation measure. Therefore, based on the calculation of local occupation measure, the density fluctuation is introduced A measure of the stability of the local occupancy distribution. definition ,in For Window Inside The average value of Represents the number of pixels or grid cells within the window range. This formula is used to measure the degree of dispersion of a local area. If the density fluctuation value is large, it means that the distribution of occupied points in the local area is uneven.

[0075] After completing the calculation of the local occupation measure and density fluctuation, the regional depth density index is defined: , here select and Nonlinear functions such as occupancy measure and fluctuation quantity are coupled. The larger the value, the denser and more uniform the distribution in the area. If the value is too low, it suggests the possibility of ghost images or structural defects.

[0076] The process of obtaining the trajectory coherence manifold index is:

[0077] In addition to density distribution, the coherence of SLAM trajectory information also needs to be evaluated. Introducing trajectory coherence manifold index , and is evaluated from two perspectives: directional consistency of local motion trajectories and global continuity.

[0078] In coordinates Collect SLAM trajectory points around , the local velocity vector is calculated based on its time sequence First define the direction consistency : ,in Indicates The velocity vector of the trajectory point Indicates that in the SLAM map, the coordinates The total number of trajectory points captured in the area. The numerator is the vector dot product, which reflects the consistency of the trajectory in direction. The higher the value, the more the local trajectory tends to a single direction.

[0079] Direction consistency cannot reflect the impact of large-scale changes in direction or interruptions. Therefore, a continuous tensor field is introduced on this basis and quantified in a simplified form as , to measure the temporal connection between trajectory points. definition ,in Represents the difference between adjacent trajectory vectors, and the exponential function is used to amplify or weaken this difference. If the trajectory suddenly changes or is interrupted in a local area, will increase, resulting in a decrease in the continuity tensor value.

[0080] The directional consistency is coupled with the continuity tensor to define the trajectory coherence manifold index: In the numerator, this structure uses the product of the directional consistency and the continuity tensor to measure the interaction between trajectory direction and continuity, and introduces the difference correction between the two in the denominator. If the difference between the two is too large, it will have a weakening effect, making the trajectory coherence manifold index more sensitive to trajectory mismatch.

[0081] After completing the calculation of the regional depth density index and trajectory coherence manifold index, the coordinates of each area in the SLAM map are Calculate the following judgment function To identify ghost interference:

[0082] ,

[0083] in To prevent small positive numbers with zero denominators, When the difference between the two items is large, the function value will have obvious positive and negative fluctuations. According to the pre-set abnormal range, if the judgment value falls within the abnormal range (for example, below a critical value or above an abnormal value), the coordinates are determined. The ghost area is identified and removed.

[0084] After removing the ghost area, the dynamic crowd flow heat map obtained by the spatiotemporal alignment in step S2 is mapped point by point to the corresponding position of the SLAM map according to the three-dimensional coordinates. The mapping process first converts the coordinates in the heat map into Projection to SLAM 3D coordinates , and then superimpose the remaining effective thermal information with the SLAM static structure to generate a dynamic environment map. This dynamic environment map can reflect the changes in personnel distribution and eliminate obvious abnormal structures, laying a higher reliability foundation for the subsequent navigation path planning.

[0085] By constructing the regional depth density index and trajectory coherence manifold index and combining them to form a complex judgment formula, the ghost interference area in the SLAM map can be more accurately eliminated. The generated dynamic environment map not only takes into account the accuracy of the static environment, but also reflects the real dynamic characteristics of human flow, providing more reliable scene support for augmented reality navigation.

[0086] The previous steps have obtained the environment map that integrates the dynamic information of the flow of people, and completed the identification and elimination of the ghost area. Step S4 aims to search for the optimal navigation path based on the dynamic environment map and the real-time location of the user, and present virtual arrow guidance in the augmented reality interface to achieve an immersive interactive experience. This process integrates path planning algorithms and AR coordinate mapping technologies, and can maintain high-precision navigation effects when the indoor environment changes in real time.

[0087] Step S4 includes the following contents:

[0088] The dynamic environment map is first discretized into a graph structure consisting of nodes and edges. Nodes represent key locations or grid centers in the map, and edges represent accessible paths between nodes. A cost function is defined on each edge to quantify the difficulty of passage. To characterize the impact of congestion, the cost function can be written as: ,in represents the geometric distance or walk length of the edge, Indicates the density of people flow within the coverage area of ​​this edge. The coefficient for penalizing congestion can be initialized or adaptively adjusted according to user preferences or scenario requirements. The above construction forms a weighted graph that integrates the flow of people, laying the foundation for the subsequent search for the optimal path.

[0089] On the above weighted graph, heuristic search algorithms (such as A*) or improved Dijkstra methods are used to calculate the optimal path. Heuristic search reduces the search space based on the geometric features of the map or local orientation information. If the heat of the flow of people changes significantly, the cost of the edge will be updated accordingly, which may cause the existing path to fail. At this time, the congestion of the key edges is re-evaluated and replanning is triggered to continuously maintain the optimal solution under dynamic conditions. Replanning can significantly improve path reliability in high congestion scenarios and avoid navigation stagnation or detour failure.

[0090] The calculated optimal path usually includes a series of key nodes or inflection points. Align with the coordinate system of the AR display device and project it into the augmented reality space with the help of the coordinate transformation matrix M. Then render virtual arrows or direction marks in the AR interface to merge them with the real scene. When the device posture or viewing angle changes, the arrows are updated accordingly. In this way, continuous action guidance is presented in the field of view, guiding the user to move forward along the optimal trajectory.

[0091] The coordinate transformation matrix M here refers to the three-dimensional coordinates used to align in augmented reality The mapping matrix converted to the coordinate system of the AR display device. Usually including rotation, translation, and scaling (or other projection) operations when necessary, so that the spatial points in the external environment can be correctly mapped to the view coordinate system of the device, thereby achieving accurate superposition of virtual and real in the device screen (or field of view).

[0092] Step S4 uses a weighted graph search algorithm to calculate the optimal navigation path in real time based on the dynamic environment map and crowd distribution, and uses AR coordinate mapping to overlay the guidance information on the real environment. When facing uncertain changes in crowd flow, the path planning process can maintain dynamic adaptation through cost function re-evaluation and re-planning mechanisms. Finally, the virtual arrows presented on the AR interface provide users with direct and accurate movement guidance, completing an immersive indoor navigation experience.

[0093] The present invention generates a real-time heat map of human flow through high-frequency sensor network acquisition and denoising aggregation, combines timestamp synchronization with spatial point matching to achieve accurate alignment of human flow data with SLAM maps, and then uses indexes such as regional depth density and trajectory coherence manifold to identify and remove ghost interference, maps three-dimensional coordinates to generate a dynamic environment map, and then dynamically optimizes the navigation path through weighted graph search and replanning mechanism, and uses AR coordinate conversion and immersive virtual arrows for human-computer interaction, covering the diverse structures and uncertain changes of human flow in indoor scenes. This solution combines high-concurrency sensor data with dynamic SLAM processing flow, improves the accuracy of path planning and the real-time and immersiveness of navigation guidance, avoids travel blockages or erroneous detours caused by human flow congestion, reduces the impact of environmental structural defects caused by SLAM errors or dynamic objects, and ultimately enhances the accuracy, flexibility and autonomy of indoor navigation. At the same time, it integrates human flow thermal information and visual interaction requirements, presents a more intuitive and efficient indoor navigation method for end users, and strengthens the synergy between environmental understanding and path optimization, thereby ensuring navigation stability and reliability in large-scale indoor spaces.

[0094] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0095] It should be noted that the system of the present invention can be deployed on the device itself to realize embedded applications, and can also be run on a PC or other terminal with a user interface, thereby meeting a variety of hardware environments and usage requirements.

[0096] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0097] It should be noted that, in this article, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0098] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An immersive space virtual-reality interaction method based on the metaverse, characterized in that: Includes steps: S1: Use high-frequency sensor networks to collect dynamic crowd flow data and generate crowd flow heat maps through denoising and aggregation; S2: Based on timestamp synchronization and spatial point matching, the crowd flow heat map and SLAM map are temporally and spatially aligned; S3: Extract the density and trajectory coherence of the area from the SLAM map as key features, input these features into the pre-trained classification model to determine and remove the ghost interference area, and then map the aligned heat map point by point according to the three-dimensional coordinates to generate a dynamic environment map; S4: Calculate the optimal navigation path in real time based on the dynamic environment map and user location, and present an immersive interactive experience with virtual arrows in the AR interface.

2. The immersive space virtual-reality interaction method based on the metaverse according to claim 1, characterized in that: Step S1 includes the following contents: The high-frequency sensor network deploys multiple sensor nodes to capture the dynamic flow data of people in the environment in real time, denoise it, and aggregate the denoised data to generate a real-time heat map of the flow of people.

3. The immersive space virtual-reality interaction method based on the metaverse according to claim 2, characterized in that: The aggregation process is as follows: Assume the spatial position is The data for each time period is ,in Represents the time series number, and the aggregation process is performed by the following formula: ,in, Indicates at time Time, Location The accumulated heat value at For in time Dynamic flow data of people at the corresponding location at the time, It represents the total data collection points, and presents the aggregated data as a heat map through color mapping, which ultimately generates a real-time and accurate heat map of pedestrian flow.

4. The immersive space virtual-reality interaction method based on the metaverse according to claim 3 is characterized in that: Step S2 includes the following contents: S2.1, first synchronize the timestamps of the dynamic crowd flow heat map and SLAM map data; S2.2, spatially align the crowd flow heat map and SLAM map through coordinate transformation technology; If the SLAM map uses coordinate system, and the flow data is , then you need to use affine transformation or rotation matrix to map the coordinates and get the corresponding three-dimensional coordinates ; Divide the three-dimensional coordinate points of the SLAM map into grids and set a spatial grid , the same gridding process is performed on the heat map of human flow; each heat map point is mapped to the corresponding grid of the SLAM map by matching the overlapping areas of the grids, thereby completing the precise alignment of the spatial points; S2.3, after completing the timestamp synchronization and spatial point matching, perform time-space alignment. For each collected time point, the corresponding three-dimensional coordinates and cumulative heat value Matching is done based on time and space locations.

5. The immersive space virtual-reality interaction method based on the metaverse according to claim 4, characterized in that: Step S3 includes the following contents: Key features include the regional depth density index and the trajectory coherence manifold index.

6. The immersive space virtual-reality interaction method based on the metaverse according to claim 5, characterized in that: The process of obtaining the regional depth density index is as follows: Select the space window in the SLAM map To cover the coordinates A certain range around, and count the number of occupied points and their distribution, first define the local occupancy measure : ,in Indicates location Whether it is an occupied point identified by SLAM, the value range is [0,1]; introduce density fluctuation Measures the stability of the local occupancy distribution and defines ,in For Window Inside The average value of Represents the number of pixels or grid cells within the window range; after completing the calculation of the local occupancy measure and density fluctuation, the regional depth density index is defined: 。 7. The immersive space virtual-reality interaction method based on the metaverse according to claim 6, characterized in that: The process of obtaining the trajectory coherence manifold index is: In coordinates Collect SLAM trajectory points around , the local velocity vector is calculated based on its time sequence , first define the direction consistency ,in Indicates The velocity vector of the trajectory point Indicates that in the SLAM map, the coordinates The total number of track points captured in the area; A continuous tensor field, quantized in simplified form as , to measure the temporal connection between trajectory points, define ,in Represents the difference between adjacent trajectory vectors; couples the directional consistency with the continuity tensor to define the trajectory coherence manifold index: .

8. The immersive space virtual-reality interaction method based on the metaverse according to claim 7, characterized in that: After completing the calculation of the regional depth density index and trajectory coherence manifold index, the coordinates of each area in the SLAM map are Calculation judgment function To identify ghost interference; according to the pre-set abnormal range, if the judgment value falls within the abnormal range, the coordinates are determined The ghost area is removed and the coordinates in the heat map are transformed according to the coordinate transformation relationship. Projection to SLAM 3D coordinates ,Then the remaining effective thermal information is superimposed with the SLAM static structure to generate a dynamic environment map.

9. The immersive space virtual-reality interaction method based on the metaverse according to claim 8, characterized in that: Step S4 includes the following contents: The dynamic environment map is first discretized into a graph structure consisting of nodes and edges. The nodes represent key locations or grid centers in the map, and the edges represent accessible paths between nodes. A cost function is defined on each edge to quantify the difficulty of passage, and finally a weighted graph that integrates the flow of people is constructed. On the above weighted graph, a heuristic search algorithm is used to calculate the optimal path. The calculated optimal path usually includes a series of key nodes or inflection points, whose three-dimensional coordinates are Align with the coordinate system of the AR display device and project into the augmented reality space with the help of the coordinate transformation matrix; Then render a virtual arrow or direction mark in the AR interface to integrate it with the real scene. When the device posture or perspective changes, the arrow will update its position accordingly.

Images