Virtual experience immersive tour method based on multi-source data fusion

Abstract

The present application relates to the technical field of virtual reality, in particular to a virtual experience immersive guide method based on multi-source data fusion, comprising: acquiring real-time data of each exhibition subarea according to global coordinates of a virtual reality scene; monitoring the marker types of each exhibit node in the exhibition subarea based on the visit times of exhibits and the behavior trajectories of users in consecutive time periods, and constructing the behavior correlation areas corresponding to each exhibit node according to the multi-time period data aggregation mode; taking the node list mapped by the behavior correlation areas as a guide, extracting multiple path nodes from the exhibition subarea to form preliminary guide paths; performing path re-planning according to the overlapping parts of different preliminary guide paths in the exhibition subarea to determine the guide paths after re-planning; and globally synchronizing the guide paths according to the updated time and position to obtain a multi-path link distribution map. The accuracy and processing efficiency of virtual reality guidance are improved.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, specifically to a virtual experience immersive guided tour method based on multi-source data fusion. Background Technology

[0002] Virtual reality technology can create and allow users to experience a simulated virtual environment, using computer-generated images and sounds to immerse them in the scene and feelings. However, in current technologies, when users tour a destination using VR, navigation methods such as arrows, markers, or trajectory lines are generally used. These methods emphasize one-way guidance and cannot capture changes in the user's interest during the tour.

[0003] For example, Chinese Patent Publication No. CN121165945A discloses a method, device, computer equipment, and medium for displaying navigation paths in a virtual scene. The method includes: collecting current user behavior data through VR device sensors and analyzing user intent based on the behavior data; dynamically generating the current path using path planning based on the user intent and virtual scene information; generating a special effects navigation path based on the current path; and generating semantic path guidance information using multimodal semantic fusion. This invention provides intuitive navigation guidance for users, improving the intuitiveness of path display and enhancing the immersive experience.

[0004] For example, Chinese Patent Publication No. CN118732900A discloses a scene analysis method and system based on virtual reality technology. Based on the roaming request from the user, the method determines the display space selected by the user in the virtual model corresponding to the target location. The virtual model includes multiple display spaces. It obtains the navigation strategy selected by the user based on scene elements within the display space, generates navigation display data based on the navigation strategy, and sends it to the user. The navigation strategy includes individual navigation strategies and global navigation strategies. Based on the user's explanation needs for the navigation display data, it arranges the corresponding scene elements to obtain a navigation explanation sequence. It then generates a customized digital human corresponding to the navigation explanation sequence and uses the customized digital human to explain the display space.

[0005] In existing technologies, obstacle avoidance is achieved through path effect rendering combined with addressing algorithms, emphasizing particle flow and change; or through strategy guidance, the strategy is projected onto multiple elements of the navigation interface, emphasizing the distribution of element layout. However, existing technologies tend to differentiate data under specific dimensions and are biased towards individual-driven data scheduling, which can easily lead to a disconnect between global division and individual differentiation. Furthermore, emphasizing obstacle avoidance in spatial regions can easily lead to the neglect of the adjacency relationship of spatial regions, thereby affecting the path disconnect and regional fragmentation in global planning. Summary of the Invention

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a virtual experience immersive tour guide method based on multi-source data fusion, including: S1, obtaining real-time data of each exhibition zone according to the global coordinates of the virtual reality scene.

[0007] S2 monitors the tagging type of each exhibit node within the exhibition area based on the number of visits to exhibits and user behavior trajectories over continuous time periods, and constructs the behavior association area corresponding to each exhibit node based on the aggregation of data from multiple time periods.

[0008] S3, guided by the list of nodes mapped to the behavior-related areas, extracts multiple path nodes from the exhibition partitions to form a preliminary tour guide path.

[0009] S4. Based on the overlapping parts of the exhibition zones with different initial guided tour routes, perform route replanning and determine the replanned guided tour routes.

[0010] S5 globally synchronizes the navigation path based on the update time and location to obtain a multi-path link distribution map.

[0011] The beneficial effects of this invention are as follows: First, this invention establishes a unified spatial benchmark by dividing continuous and non-overlapping functional zones based on virtual reality global coordinates; it adopts a hierarchical gradient import method for 3D models and rendered images, performs collision detection on all models, establishes interaction triggering rules based on gaze dwell time, and organizes interaction data into real-time data in a tree structure; it clarifies the hierarchical management and interaction triggering of virtual reality scene data, providing a reliable data foundation for subsequent behavior analysis and path planning.

[0012] II. This invention statistically analyzes the effective visit frequency of exhibits based on the number of interaction points and information density. It then extracts the changing trends of exhibit visits and user behavior trajectories through first- / second-order difference analysis. Simultaneously, it uses Pearson correlation coefficients to quantify trend consistency, identifying points of interest that exhibit consistency, as well as isolated, pass-through, and background exhibit nodes that exhibit inconsistencies. Finally, it performs cluster analysis on data from multiple time periods using a three-dimensional spatial-temporal-feature tensor to generate behavioral association regions. This avoids discrepancies between manually defined functional zones in open spaces and actual user flow, which can lead to distorted user behavior recognition and pattern deviations. It improves the reliability of user behavior association, further identifies behavioral association regions that conform to user flow patterns, solves the problem of false adjacency in open spaces, and enhances the accuracy of subsequent processing.

[0013] Third, this invention selects path nodes hierarchically based on node tag type, prioritizing points of interest and independent points of interest; it employs differentiated aggregation strategies for multi-type and single-type nodes; and it establishes connections based on the walking distance between nodes to generate a preliminary guided tour path that meets the minimum walking distance constraint. This avoids highly homogenized guided tour paths for different users, enhances the personalization of the paths, improves the adaptability of the guided tour path to user behavior, and strengthens the immersive experience of the virtual scene.

[0014] IV. This invention identifies spatially folded areas by detecting path width and height, filtering out impassable paths; based on path-sharing segment analysis, it replans paths with high overlap and filters candidate adjacent area pairs based on minimum spatial distance; it quantifies the number of guide paths crossing area boundaries and calculates the relative connectivity of paths; and it generates a global multi-path link distribution map based on connectivity priority. This achieves global collaborative distribution of all paths, further solving the problem of false adjacency in multi-path systems, and ultimately realizing the synergy between pedestrian flow distribution and global distribution. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0016] Figure 1 This is a flowchart illustrating a virtual experience immersive guided tour method based on multi-source data fusion.

[0017] Figure 2 This is a flowchart illustrating step S1 of the virtual experience immersive guided tour method based on multi-source data fusion.

[0018] Figure 3 This is a flowchart illustrating step S2 of the virtual experience immersive guided tour method based on multi-source data fusion.

[0019] Figure 4 This is a flowchart illustrating step S3 of the virtual experience immersive guided tour method based on multi-source data fusion;

[0020] Figure 5 This is a flowchart illustrating step S4 of the virtual experience immersive guided tour method based on multi-source data fusion.

[0021] Figure 6 This is a flowchart illustrating step S5 of the virtual experience immersive guided tour method based on multi-source data fusion. Detailed Implementation

[0022] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Where specific techniques or conditions are not specified in the embodiments, they shall be performed in accordance with the techniques or conditions described in the literature in the art or in accordance with the product manual.

[0023] See Figure 1 The virtual experience immersive tour guide method based on multi-source data fusion includes: S1, obtaining real-time data of each exhibition zone based on the global coordinates of the virtual reality scene.

[0024] S2 monitors the tagging type of each exhibit node within the exhibition area based on the number of visits to exhibits and user behavior trajectories over continuous time periods, and constructs the behavior association area corresponding to each exhibit node based on the aggregation of data from multiple time periods.

[0025] S3, guided by the list of nodes mapped to the behavior-related areas, extracts multiple path nodes from the exhibition partitions to form a preliminary tour guide path.

[0026] S4. Based on the overlapping parts of the exhibition zones with different initial guided tour routes, perform route replanning and determine the replanned guided tour routes.

[0027] S5 globally synchronizes the navigation path based on the update time and location to obtain a multi-path link distribution map.

[0028] In this invention, a three-dimensional coordinate system covering the entire area is established, and continuous non-overlapping exhibition zones are divided according to function. The boundaries, areas, and coordinates of the exhibits contained in each zone are recorded. The three-dimensional layers are imported and registered in the layered order of terrain → architecture → fixed exhibits → movable exhibits → special effects to establish real-time data mapping between exhibition zones and three-dimensional space.

[0029] like Figure 2 As shown, one implementation of step S1 includes: S11, dividing the exhibition areas into continuous and non-overlapping sections according to the three-dimensional coordinates of the virtual reality scene and functional labels.

[0030] S12, following the spatial gradient from entrance to exit, traverse each exhibition area, import the 3D models and rendered images of each location layer by layer according to functional labels, and record the range of data acquired from the current perspective.

[0031] S13, within the data acquisition range, perform collision detection on all 3D models, determine the interaction trigger rules based on the gaze dwell time, and treat the processed data as real-time data of the exhibition area.

[0032] Furthermore, collision detection involves determining whether there is clipping within the data range acquired from the current viewpoint, and whether the virtual model representing the visitor from the current viewpoint clips with the model of the exhibit. If clipping occurs, the content of the model that collides is displayed; otherwise, the process proceeds to the interaction trigger.

[0033] Furthermore, the interaction trigger rules represent the operations triggered based on the focus of the gaze. These operations include detailed explanations of any exhibit, audio guides, 3D rotating models, and other specific interactive operations. This data will be recorded as real-time data within the exhibition area to form a relatively complete exhibition area.

[0034] Specifically, when determining interaction trigger rules based on gaze dwell time, the gaze dwell time at any 3D model is used as the judgment condition. If the dwell time exceeds the preset dwell time, the interaction trigger rule is triggered. Based on the data content displayed according to the interaction trigger rule, the corresponding data content is sequentially connected in a tree structure and populated into the real-time data of the exhibition area. The preset dwell time can be initially set to 5 seconds. When the user continuously lingers on the 3D model corresponding to the exhibit, the text, images, videos, and other content annotated on that 3D model are displayed promptly.

[0035] The exhibition zones here are divided by function, based on the theme of each exhibition zone. All exhibits, spaces and explanatory information related to the theme are grouped into the same area, thus forming multiple exhibition zones with different themes and non-overlapping spaces.

[0036] In one embodiment of the present invention, by integrating the exhibit access content in the virtual space with the real-time visitor flow behavior trajectory, the specific interests of users at different exhibit nodes are quantified, thereby improving the ability of both to understand user behavior.

[0037] The exhibit access content includes characteristic values ​​such as the size of the corresponding 3D model of the exhibit, its screen occupancy, information density (the number of text + images + videos contained in the exhibit), and the number of interactive points. These characteristics represent the items and intuitive visual information that can be displayed when observing exhibits through virtual reality, and belong to the static data of the virtual reality scene. Here, one exhibit corresponds to multiple display contents, which are combined into a complete display unit through text + images + videos, and interactive points that can be operated by users are set in the display content to explain the actual composition and origin of the exhibit.

[0038] It should be noted that the word count here is not the pure word count, but represents the number of text boxes within a display content to explain how many text boxes are used to describe the text content of an exhibit.

[0039] The behavioral trajectory data includes real-time visitor flow characteristics such as crowd density, actual dwell time, and crowd direction. This is statistically analyzed by dividing the guided tour area into 1m x 1m grids. Crowd density is calculated by counting the number of bounding boxes of user avatars within a single grid in the virtual reality scene, and then calculating the average crowd density of each grid within a time window. This is essentially the ratio of user dwell time within a grid to the time window, quantifying the number of people passing through each location, and excluding instantaneous data with dwell times less than 1 second to compare the number of people passing through each grid unit. Dwell time is obtained by monitoring the continuous residence time of user avatars within a single grid unit, calculated by determining the average dwell time of users within a single grid within a specific time window. Crowd direction is determined by analyzing the displacement direction vector of user avatars and calculating the average direction vector of all users within the grid, which is then considered the crowd direction within the corresponding grid. The time window here is divided into 5-minute statistical segments. Each grid cell obtains the grid set corresponding to each exhibit node based on the intersection of the bounding box of the exhibit node and all grids, and synchronizes the data of the grid set to the corresponding exhibit node.

[0040] like Figure 3 As shown, one implementation of step S2 includes: S21, extracting the display content triggered by each exhibit node within a continuous time period from the real-time data of the current exhibition partition, and counting the effective access times of each exhibit node within the continuous time period based on the number of interaction points and information density of the display content.

[0041] S22. Based on the effective access count of the exhibit nodes, calculate the first-order and second-order difference values ​​for each time period to determine the trend of exhibit access.

[0042] S23 uses pedestrian density, dwell time, and pedestrian flow direction as feature values ​​of behavioral trajectory to determine the changing trend of behavioral trajectory.

[0043] S24. Compare the changing trends of behavioral trajectories and exhibit visits. Based on the comparison results of the changing trends and the node label type, combine them into behavioral association regions through three-dimensional tensor clustering.

[0044] Furthermore, the effective access count of the exhibit node includes: a weighted coefficient for each access calculated based on the normalized values ​​of the number of interaction points and information density; and the access count of the exhibit node is weighted using the weighted coefficient for each access to obtain the output effective access count.

[0045] Firstly, when calculating the weight coefficient of a single visit, the number of interaction points and information density can be weighted equally to balance the triggered display content and the interaction point operation scenarios. Secondly, the weighting of the number of visits essentially emphasizes the degree to which different locations are increased due to interaction and observation.

[0046] Furthermore, the first and second differences of the effective visit count represent the rate and acceleration of change of that number, and the trend of change of the effective visit count is defined according to the pattern of increasing rate of change, decreasing rate of change, increasing acceleration of change, and decreasing acceleration of change. Similarly, the trends of change of crowd density and dwell time are recorded in the same way; as for the trend of change of crowd direction, the direction vectors of users around the exhibit nodes are recorded, and the magnitude of the average length of the vector is used to explain the situation where users concentrate on moving towards a certain exhibit node. When the normalized value of the average length of the vector is in [0.9, 1.0], the direction is highly consistent, indicating that everyone is moving towards the specific exhibit; when the value is in (0.5, 0.9), the direction is roughly consistent, with most people moving towards the specific exhibit; when the value is in (0, 0.5], the direction is scattered, and the crowd flows randomly; it is necessary to use the comprehensive weighted value of crowd density, dwell time, and crowd direction to determine whether the current area change is consistent with the trend of visit count.

[0047] Specifically, the characteristic values ​​of pedestrian density, dwell time, and pedestrian direction are weighted and fused to determine the changing trend of the behavioral trajectory. The weights used for the three will be 0.4, 0.4, and 0.2 to quantify the behavioral trajectory around the exhibition node, and a trend sequence with the same number of effective visits will be set according to the first-order difference and second-order difference.

[0048] Furthermore, when comparing the changing trends of behavioral trajectories and exhibit visits, the trend values ​​corresponding to behavioral trajectories and exhibit visits are used as input parameters to calculate the Pearson correlation coefficient for each exhibit node in turn. That is, by using the weighted value of the effective number of visits and the behavioral trajectory as input parameters, the correlation coefficients of all exhibit nodes within the exhibition area are calculated respectively, and the first-order difference and second-order difference values ​​of the corresponding parameters are introduced. The average value of the calculated Pearson correlation coefficients is regarded as the content of trend calculation.

[0049] If the number of valid visits to an exhibit node and its Pearson correlation coefficient meet the consistency requirement, the corresponding exhibit node will be regarded as an interest tendency point and input into the comparison results as the basis for the combined behavior association area. The consistency requirement here means that the main factors of behavioral trajectory change when exhibit visits change. The exhibit node must meet at least the following two conditions: 1. Its Pearson correlation coefficient is greater than 0.4 for two or more consecutive time periods; 2. The number of valid visits in the corresponding time window is greater than or equal to 1.2 times the average number of valid visits in the exhibition partition.

[0050] If the consistency requirement is not met, the exhibit nodes are classified into types, further divided into isolated, passing, and background exhibit nodes; the comparison results are updated according to the label type of each exhibit node, and behavioral association areas are formed based on the updated content.

[0051] The tagging types include points of interest, isolated types, pass-by types, and background types. When consistency requirements are not met, other types besides points of interest need to be identified. Isolated types indicate a surge in exhibit visits, but a decrease in overall visitor flow and dwell time within the zone. This emphasizes that some exhibit nodes are detached from the overall zone and require additional tagging, being marked as independent points of interest in the behavior-related area. Pass-by types indicate a surge in visitor flow within a zone, but a decrease in visits to the exhibit itself. This indicates that the current exhibit is unattractive and should only be used as a transitional node, updated in the behavior-related area. Background types indicate no significant change in exhibit visits, but significant fluctuations in visitor flow within the zone. This represents slow change at the corresponding location and has no effect on driving user interest. These will serve as reference nodes for spatial navigation, not part of the initial guide processing, and will be spatially marked at the corresponding location.

[0052] Specifically, for isolated exhibit nodes, a surge in visitor traffic relative to the overall increase is considered when the number of valid visits exceeds 1.5 times the average number of visits to the zone and the dwell time exceeds twice the average dwell time of the exhibit. Exhibit nodes in zones showing a decrease in overall visitor traffic and a shortening of dwell time are then identified and marked. For pass-by exhibit nodes, nodes with fewer than 0.5 times the average number of visits to the zone and a dwell time of less than 30 seconds are identified and considered transition points. For background exhibit nodes, nodes with a coefficient of variation of valid visits less than 0.2 and a dwell time of less than 15 seconds within a continuous time period are identified and marked as reference points.

[0053] Furthermore, when updating the comparison results based on the tag type, the processing method includes: for each tag type, when the exhibit node corresponding to the tag type is updated, the node list for each time period is updated synchronously according to the time period corresponding to the exhibit node.

[0054] After updating these identified nodes to the behavior association region, based on their locations, exhibits conforming to the consistency description are first connected according to their spatial locations to form a preliminary behavior association region. Then, for all exhibit nodes involved in this region, the corresponding node labeling type is checked; transition points, reference points, and independent points of interest are identified. The node list used for path connection determination within the current behavior association region is updated, and subsequent path connectivity processing is completed based on this node list. Within the updated exhibit partition, the exhibit nodes mapped by the node list and the spatial regions of the exhibit nodes will be used as the output behavior association region, and integrated with data from multiple time periods for output.

[0055] Furthermore, for the marked node list, it is necessary to perform data statistics on the behavioral association regions output across multiple time periods. By aggregating feature values, the behavioral association regions output statically across multiple time periods are transformed into dynamic regions aggregated across multiple time periods, thereby improving the accuracy of behavioral features within each time period.

[0056] Specifically, when combined into a behavior-related area, the tour area is divided into multiple grids based on global coordinates. According to the comparison results of the changing trends, the data of each exhibit node is mapped to the grid cell where it is located.

[0057] In each time period, the feature values ​​of all grid cells are filled to form an initial three-dimensional tensor. This three-dimensional tensor represents three dimensions: spatial, temporal, and feature. The spatial dimension represents dividing the entire guided tour area into 1m×1m grid cells, with each cell corresponding to a spatial slice of the tensor. The temporal dimension represents historical data from the past hour, with each 5-minute period covering the previous hour. The feature dimension represents the feature values ​​corresponding to five parameters: visit frequency, average dwell time, number of interaction points, information density weighting coefficients for each visit, crowd density, and the Pearson correlation coefficient between behavioral trajectories and exhibit visits. The feature values ​​for each dimension are normalized to eliminate their dimensions.

[0058] Region clustering is performed based on the initial three-dimensional tensor after multiple time periods are superimposed to form the output behavior-related regions.

[0059] Specifically, after obtaining the feature values ​​for each time period, the grids updated for each time period are determined. Based on the updated feature values, these grids are clustered to obtain the behavioral association regions under multi-time aggregation.

[0060] When performing region clustering, the values ​​of the three-dimensional tensor in the feature dimension are used as a basis. The tensor is first transformed into low-dimensional feature values, and then all grid cells are clustered to obtain the behavior-related regions that are dynamically output in the current scene.

[0061] In other words, the 3D tensor is decomposed (Tucker decomposition) into a core tensor and multiple factor matrices, each representing a low-dimensional feature value. In the current scenario, the factor matrices include spatial factor matrices, temporal factor matrices, and feature factor matrices. Each row of the spatial factor matrix corresponds to a low-dimensional vector of a spatial grid; each row of the temporal factor matrix corresponds to a low-dimensional vector of a time period; and each row of the feature factor matrix corresponds to a low-dimensional vector of a behavioral feature. Since the core tensor captures the interaction between the three dimensions, it can be selected as the weight. The low-dimensional values ​​of the three factors are then weighted and clustered to obtain the behavioral association regions under multi-grid aggregation.

[0062] Furthermore, Tucker decomposition is used to reduce the dimensionality of the three-dimensional tensor, where the spatial dimension has a rank of 8-12, the temporal dimension has a rank of 4-6, and the feature dimension has a rank of 5-8. Each element of the core tensor represents the interaction strength of the corresponding spatial, temporal, and feature factors. During clustering, the row vectors of the spatial factor matrix are weighted and summed with the corresponding dimensions of the core tensor to obtain the final feature vector of each grid cell.

[0063] Furthermore, when weighting the core tensor and the spatial factor matrix, the interaction weights of time and features are calculated first, and then the weighted feature vectors are set by multiplying the interaction weights with the row vectors of the spatial factors. The interaction weights represent the product of the core tensor with the row vectors of the time factor matrix and the feature factor matrix in each time window, and are divided into the corresponding grids to obtain the interaction weights in each grid.

[0064] It should be noted that the clustering algorithm uses density clustering, which uses the feature vector obtained from the core vector and the spatial factor matrix as the clustering value to perform clustering processing on the feature vector of each grid unit; the minimum number of samples for density clustering is set based on the dimension of the current data entry, and the elbow value is automatically selected according to the k-distance map to obtain the neighborhood radius of the current cluster.

[0065] In one embodiment of the present invention, a node list of the synchronized behavior associated area is generated, and multiple path nodes are extracted from these nodes based on the interest tendency points, independent interest points, transition points and reference points identified in the node list, as a preliminary guide path for connecting multiple exhibition areas.

[0066] like Figure 4 As shown, one implementation of step S3 includes: S31, based on the tag type of the exhibit node, first divide the node list mapped from the behavior association area into multiple groups according to different exhibition partitions; select path nodes according to the number of nodes corresponding to different tag types to obtain the path nodes to be distinguished.

[0067] Furthermore, before establishing the connection relationship between path nodes, at least one set of exhibit nodes containing the tag type is monitored.

[0068] Determine if the tag type has changed. If it has changed, sort the tags from highest to lowest according to the number of tags corresponding to a single exhibit node.

[0069] For exhibit nodes containing multiple tag types, identify the exhibit node with the most tag types, invalidate the multiple tag types set on it to avoid data conflicts when selecting path nodes; and regard this exhibit node as the initial node for path node extraction, gradually connect other exhibit nodes, aggregate the exhibit nodes that can be connected, and use them as the output path nodes.

[0070] Among them, exhibit nodes with multiple tag types represent exhibit nodes that exhibit two or more tag types within three consecutive time windows. This type of node reflects the dynamic changes in user behavior. The more frequent the node is, the stronger the attraction of the corresponding location to the user, and it can be used as a reference for path adjustment.

[0071] If an exhibit node shows multiple changes in its label type, it indicates that the location is affected by multiple time periods or pedestrian behavior, reflecting the interests of a specific group or phenomena such as walking transitions. In this case, the node with the most label types represents the point with the most frequent changes in walking within the overall scene. It needs to be used as the initial node, connected to other exhibit nodes, to find nodes that can be successfully connected, and these nodes should be included in the path planning considerations.

[0072] For exhibit nodes containing a single tag type, select them as path nodes in order of the tag type description.

[0073] When selecting path nodes, the label type of the exhibit node is found from the node list mapped by the behavior association area. Interest tendency points and independent interest points are directly selected as the extracted path nodes. Transition points are only used as path nodes under spatial navigation when they are on the path between two adjacent interest tendency points or when they are located between interest tendency points and independent interest points.

[0074] S32: Based on the walking distance between any path nodes within the current exhibition area, establish the connection relationship between each path node; for any two path nodes, if there are no obstacles between the two points and the space is accessible, make the path nodes form a network of connections. If there are obstacles between path nodes or the space is inaccessible, mark them as inaccessible and cyclically check other paths to complete the establishment of connections between multiple points.

[0075] S33: Extract the path that meets the minimum walking distance from the path nodes to be connected as the output preliminary navigation path.

[0076] The output preliminary navigation path will be determined by checking all connectable path nodes and connecting the path nodes to form the shortest path.

[0077] In one embodiment of the present invention, the preliminary guide path is determined by performing multi-region connectivity judgment based on the overlapping parts of the preliminary guide path to obtain the adjusted guide path.

[0078] like Figure 5As shown, one implementation of step S4 includes: S41, traversing all node sequences of the preliminary tour path, extracting the three-dimensional coordinates and height information of the path, performing multi-layer spatial folding recognition, and determining the preliminary tour path after filtering out false overlapping paths.

[0079] Specifically, the node sequence of all preliminary tour paths is traversed to extract the three-dimensional coordinates of each edge; vertical partitioning is performed according to floor or elevation values ​​to establish a layered connectivity graph; path segments belonging to spatial folding are identified from the layered connectivity graph, which are the vertical overlapping areas of the upper and lower floors and multiple mezzanine floors of the spiral staircase.

[0080] At the same time, it checks whether there are direct cross-floor connections in these path segments: if there are no connecting edges such as stairs or elevators, it is confirmed as a false overlap caused by spatial folding.

[0081] Then, the path segments in the vertically folded area are forcibly assigned to their respective elevation layers, and false overlaps across layers are deleted to ensure that each path can only move along the walkable edge of its own layer.

[0082] If a path is broken due to the removal of false overlaps, the local A* algorithm is called within the layer where the break point is located to re-search for the same layer path from the break point to the next node and replace the original path segment to complete the initial navigation path for spatial folding repair.

[0083] Furthermore, the Z-axis value of the three-dimensional coordinates is used as the elevation value, and vertical partitioning is performed according to an elevation layer of 3 meters. For path segments at different elevation layers at the same horizontal position, if there are no connecting edges such as stairs or elevators, they are judged as spatial folding and false overlap. When replanning the broken path, the heuristic function of the A* algorithm adopts the Manhattan distance, and the step size is set to 0.5 meters to complete the search process from the break point to the next point.

[0084] S42, for the initial tour route excluding spatial folding, the route is replanned according to the shared road segments between the routes to obtain the replanned tour route.

[0085] Since different initial tour routes may share some sections, all routes need to be replanned according to the length and number of times each route segment is shared, in order to obtain the final output tour route.

[0086] Specifically, firstly, the path weight of each path segment is set according to the reciprocal of the walking distance. When any path segment is a shared segment, the frequency of the shared location coordinates relative to the historical data is recorded as the adjustment value of the shared path weight. The product of the adjustment value and the path weight is used as the new path weight. Then, all path segments are replanned according to the path weight, and the path with the largest sum of weights is used as the output navigation path.

[0087] In the replanned navigation path, the larger the reciprocal of the walking distance, the greater the probability that the user will take the shorter path; the greater the frequency of the shared road segments, the larger the adjustment value, which further strengthens the probability that the user will take the path; at this time, the adjustment value is represented by the sum of the frequency of occurrence and 1, to strengthen the probability value of choosing the corresponding path; finally, the path with the largest sum of weights will represent the path with the highest probability of being taken in the current scenario.

[0088] In one embodiment of the present invention, the replanned tour guide path is synchronized into a global multi-path link distribution map according to the connectivity of adjacent tour guide paths. This is because two exhibition areas that are physically adjacent may not have direct user movement even if they are in a connected state. If the global link distribution map is still constructed based on physical adjacency relationships rather than actual tour guide paths, incorrect connection relationships will be generated, leading to problems such as detours in path planning and failure in crowd flow scheduling, ultimately resulting in a significant decrease in overall tour guide efficiency and user adaptability.

[0089] Specifically, the global adjacency relationship of the exhibition partitions is initialized, and a judgment mark is set for physically adjacent area pairs; the input guide path, behavior-related areas, and other path information representing user behavior are loaded; any area pair in the loaded guide path is checked to see if it is in a connected state, and areas that are connected will proceed to the next judgment step; by checking the connectivity of adjacent guide paths, the behavior flow relationship between different area pairs is verified, and the behavior flow relationship is synchronized to the global path, and finally, a global link graph containing user behavior flow is updated to improve the accuracy of user-oriented navigation.

[0090] Specifically, such as Figure 6 As shown, one implementation of step S5 includes: S51, initializing the global adjacency relationship of the exhibition partition, filtering the adjacency relationship based on the minimum spatial distance of the exhibition partition, and obtaining candidate region pairs.

[0091] Specifically, when resetting the global adjacency relationship of all exhibition zones, the minimum spatial distance from any point on the boundary of one exhibition zone to any point on the boundary of another exhibition zone is used to check whether the boundaries of the two exhibition zones are in direct contact. If the minimum spatial distance is less than or equal to 2 meters or the area boundaries are in direct contact, they are considered as candidate area pairs; otherwise, they are considered as non-adjacent and excluded from subsequent processes.

[0092] S52, for each candidate region pair, according to the connectivity of adjacent guide paths, count the number of guide paths crossing its boundary, and quantify the relative connectivity of the paths of the candidate region pair.

[0093] S53, prioritize the candidate regions based on their relative connectivity and connect them to form a multi-path link distribution map.

[0094] The relative connectivity of paths represents the ratio of the number of navigation paths traversing a pair of regions to the total number of paths traversing that pair, quantifying the execution of user actions when navigation points to the next two regions. This part of the relative connectivity calculation is used to check for abnormal boundary crossings, examining the critical situations between the two.

[0095] Furthermore, when the relative connectivity of a path is greater than or equal to the historical average, two adjacent exhibition zones are considered priority guides, and their regional connection data is retained. If the relative connectivity of a path is less than the historical average, it means that the number of people walking in that area is low, so the priority of that area is reduced, and the less popular passage is marked in the overall map. As for guide paths that are physically connected but functionally prohibited from passage, their priority is set to 0, and the connection is removed from the global link graph. Finally, a multi-path link graph containing various identifiers is obtained to improve the path identification effect against false adjacencies.

[0096] Meanwhile, each exhibition area connected by the guide path will be synchronized to the multi-path link distribution map in real time based on the updated data, so as to update the guide routes of each path in the virtual space scene in real time, ensuring that the adjacency of the area always reflects the user behavior pattern and improves the adaptability and efficiency for users.

[0097] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A virtual experience immersive guided tour method based on multi-source data fusion, characterized in that, include: S1, based on the global coordinates of the virtual reality scene, obtains real-time data for each exhibition area; S2, based on the number of visits to exhibits and user behavior patterns over continuous time periods, monitors the tagging type of each exhibit node within the exhibition area, and constructs the corresponding behavioral association area for each exhibit node based on the aggregation of data from multiple time periods. The tagging types include interest tendency points, isolated types, passing-by types, and background types. Isolated types indicate a surge in exhibit visits, but a decrease in overall traffic and dwell time in the area. Passing-by types indicate a surge in traffic in the area, but a decrease in visits to the specific exhibit. Background types indicate no significant change in exhibit visits, but significant fluctuations in traffic in the area. S3, guided by the list of nodes mapped to the behavior-related areas, extracts multiple path nodes from the exhibition areas to form a preliminary tour guide path; S4. Based on the overlapping parts of the exhibition zones with different initial guided tour routes, perform route replanning to determine the replanned guided tour route. S5, globally synchronizes the navigation path based on the update time and location to obtain a multi-path link distribution map; The implementation methods for the behavior-related region in step S2 include: S21. Extract the display content triggered by each exhibit node within a continuous time period from the real-time data of the current exhibition area. Based on the number of interaction points and information density of the display content, count the number of effective visits to each exhibit node within a continuous time period. S22, based on the effective access count of the exhibit nodes, calculate the first-order difference and second-order difference values ​​for each time period to determine the changing trend of exhibit access; S23, using pedestrian density, dwell time and pedestrian direction as feature values ​​of behavioral trajectory to determine the changing trend of behavioral trajectory; S24, compare the changing trends of behavioral trajectories and exhibit visits, and based on the comparison results of the changing trends and node label types, combine them into behavioral association regions through three-dimensional tensor clustering; When comparing the changing trends of behavioral trajectories and exhibit visits, the trend values ​​corresponding to behavioral trajectories and exhibit visits are used as input parameters, and the Pearson correlation coefficient for each exhibit node is calculated sequentially. If the number of valid visits to an exhibit node and the Pearson correlation coefficient meet the consistency requirement, then the corresponding exhibit node will be regarded as an interest point and input into the comparison results. If the consistency requirement is not met, the exhibit nodes are classified by type, and other types besides interest tendency points need to be identified. The comparison results are updated according to the label type of each exhibit node, and behavioral association areas are formed based on the updated content. For each tag type, when the exhibit node corresponding to the tag type is updated, the node list for each time period is updated synchronously according to the time period corresponding to the exhibit node; When combining three-dimensional tensor clusters into behavioral association regions, the tour area is divided into multiple grids based on global coordinates. According to the comparison results of the changing trends, the data of each exhibit node is mapped to the grid cell where it is located. In each time period, the eigenvalues ​​of all grid cells are filled to form the initial three-dimensional tensor; Region clustering is performed based on the initial three-dimensional tensor after multiple time periods are superimposed to form the output behavior-related regions.

2. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 1, characterized in that, The implementation methods for real-time data of the exhibition partitions in step S1 include: S11, based on the three-dimensional coordinates of the virtual reality scene, divides the exhibition areas into continuous and non-overlapping sections according to functional labels; S12, traverse each exhibition area in spatial gradient order, import the 3D models and rendered images of each location in layers according to functional labels, and record the range of data acquired from the current viewpoint; S13, within the data acquisition range, perform collision detection on all 3D models, determine the interaction trigger rules based on the gaze dwell time, and treat the processed data as real-time data of the exhibition area.

3. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 2, characterized in that, When determining interaction trigger rules based on gaze duration, the implementation methods include: The interaction trigger rule is triggered when the gaze dwell time at any 3D model is greater than the preset dwell time. Based on the interactive trigger rules, the data content is displayed, and the corresponding data content is connected sequentially in a tree structure to fill the real-time data of the exhibition area.

4. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 1, characterized in that, The implementation of the preliminary navigation path in step S3 includes: S31, Select path nodes based on the label type of the exhibit nodes; S32, establish the connection relationship between each path node based on the walking distance between any path nodes within the current exhibition area; S33: Extract the path that meets the minimum walking distance from the path nodes to be connected as the output preliminary navigation path.

5. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 4, characterized in that, The methods for selecting path nodes include: Determine if the tag type has changed. If it has changed, sort the tags from highest to lowest according to the number of tags corresponding to a single exhibit node. For exhibit nodes containing multiple tag types, find the exhibit node containing the most tag types and treat it as the initial node for path node extraction. Then, connect other exhibit nodes step by step, and aggregate the exhibit nodes that can be connected as the output path nodes. For exhibit nodes containing a single tag type, select them as path nodes in order of the tag type description.

6. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 1, characterized in that, The navigation path in step S4 is implemented in the following ways: S41, traverse all node sequences of the preliminary tour route, extract the three-dimensional coordinates and height information of the route, perform multi-layer spatial folding recognition, and determine the preliminary tour route after filtering out false overlapping paths. S42, for the initial tour route excluding spatial folding, the route is replanned according to the shared road segments between the routes to obtain the replanned tour route.

7. The virtual experience immersive guided tour method based on multi-source data fusion according to claim 1, characterized in that, The implementation methods of the multi-path link distribution map in step S5 include: S51, initialize the global adjacency relationship of the exhibition partition, filter the adjacency relationship based on the minimum spatial distance of the exhibition partition, and obtain candidate region pairs; S52, For each candidate region pair, according to the connectivity of adjacent guide paths, count the number of guide paths crossing its boundary, and quantify the relative connectivity of the paths of the candidate region pair. S53, prioritize the candidate regions based on their relative connectivity and connect them to form a multi-path link distribution map.

Images