A method for constructing a virtual cultural tourism platform based on digital twins

Abstract

The present invention discloses a method for constructing a virtual cultural tourism platform based on digital twins, which relates to the field of digital twins and cultural tourism technology, including: dynamically allocating federated model accuracy weights through a federated learning framework based on a federated learning initialization instruction set, and simultaneously utilizing a video twin engine to calibrate a real-time video stream with a three-dimensional model to generate a virtual-to-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions; based on the virtual-to-real scene bidirectional mapping protocol, scanning the three-dimensional geometric structure of the physical space through ray detection and executing a mixed reality collision monitoring algorithm to generate a closed-loop interactive data set containing NPC behavior instructions and physical device control signals. The present invention dynamically allocates federated model accuracy weights, combines the real-time video stream frame rate fluctuation to trigger a target detection algorithm to perform semantic segmentation on key areas, calculates geometric alignment error values to divide accuracy levels, and enables seamless connection between the virtual environment and the real world.

Description

Technical Field

[0001] The present invention relates to the field of digital twin and cultural tourism technology, and in particular to a method for constructing a virtual cultural tourism platform based on digital twin. Background Art

[0002] Traditional cultural tourism experiences primarily rely on direct interaction between the physical space of a physical attraction and visitors. This model presents significant limitations in terms of information transmission efficiency, personalized service provision, and real-time feedback mechanisms. In recent years, digital twin technology has gradually emerged. By creating a digital mirror of the physical environment, it enables high-precision simulation and dynamic updating of the actual scene. However, existing cultural tourism applications of digital twin technology are often limited to static displays or limited interactive functions, failing to fully realize its potential to enhance the visitor experience.

[0003] One of the major challenges facing the tourism industry today is how to achieve efficient data sharing and analysis to support personalized visitor services while protecting user privacy. Existing tourism platforms often fail to effectively integrate data from diverse sources, resulting in a lack of targeted and flexible services. Furthermore, while some advanced virtual reality (VR) and augmented reality (AR) technologies have been applied to enhance the visitor experience, their application is often independent of actual operational management and decision support, limiting their potential for optimizing resource allocation and improving management efficiency. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for constructing a virtual cultural and tourism platform based on digital twins to solve the problem.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides a method for constructing a virtual cultural and tourism platform based on digital twins, which includes collecting real-time data streams from the entire cultural and tourism area, initializing a federated learning framework at the edge point, and generating a federated learning initialization instruction set; based on the federated learning initialization instruction set, dynamically allocating the accuracy weight of the federated model through the federated learning framework, and at the same time using the video twin engine to calibrate the real-time video stream with the three-dimensional model to generate a virtual-to-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions; based on the virtual-to-real scene bidirectional mapping protocol, scanning the three-dimensional geometric structure of the physical space through ray detection and executing a mixed reality collision monitoring algorithm to generate a closed-loop interaction data set containing NPC behavior instructions and physical equipment control signals; based on the closed-loop interaction data set, executing federated model training at each edge node, calculating the federated model gradient and uploading it to the federated master node, and the master node aggregating the gradient and updating the global federated model parameters; using the global federated model parameters to trigger dynamic navigation and risk warning, and synchronizing the optimized virtual-to-real scene bidirectional mapping protocol to multiple scenic spot digital twins through the metaverse protocol to build a cross-scene virtual cultural and tourism platform.

[0008] As an optimal solution for the method of constructing a virtual cultural and tourism platform based on digital twins described in the present invention, the real-time data stream of the entire cultural and tourism area includes tourist behavior thermal distribution data, environmental parameters, cultural relics status data and real-time video stream data.

[0009] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins according to the present invention, the steps of initializing the federated learning framework at the edge point and generating the federated learning initialization instruction set are as follows:

[0010] Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the cultural and tourism industry, and output a dimensionally aligned local training dataset.

[0011] Based on the local training dataset, through multimodal learning, a federated learning initialization instruction set containing the federated model architecture template, aggregation weight rules and communication protocol parameters is dynamically generated.

[0012] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the method includes: dynamically allocating the accuracy weight of the federated model through the federated learning framework based on the federated learning initialization instruction set, and calibrating the real-time video stream with the three-dimensional model using the video twin engine to generate a bidirectional mapping protocol of virtual and real scenes containing accuracy grading parameters and NPC control instructions. The specific steps are as follows:

[0013] Based on the federated learning initialization instruction set, the device performance parameters and data reliability indicators of the edge nodes are extracted, and the accuracy weight of the federated model is dynamically allocated;

[0014] When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the stability threshold, it triggers the target detection algorithm to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects;

[0015] Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided, and the accuracy grading parameters including mesh density and algorithm selection rules are generated;

[0016] Based on the accuracy grading parameters, the geometric alignment error value of the current scene and the federated model accuracy weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine including logical complexity;

[0017] The federated model precision weights, geometric alignment error values, and NPC behavior engine are encapsulated into structured data packets, synchronized to edge nodes and cloud master nodes through a unified spatiotemporal interface, and a bidirectional mapping protocol for virtual and real scenes containing precision grading parameters and NPC control instructions is generated.

[0018] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the method is based on a bidirectional mapping protocol between virtual and real scenes, scans the three-dimensional geometric structure of the physical space through ray detection and executes a mixed reality collision monitoring algorithm to generate a closed-loop interaction data set containing NPC behavior instructions and physical device control signals. The specific steps are as follows:

[0019] Based on the bidirectional mapping protocol of virtual and real scenes, the 3D geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted;

[0020] Align the surface point cloud data with the 3D model coordinate system and execute the mixed reality collision detection algorithm to calculate the collision probability;

[0021] Dynamically adjust the virtual NPC's path planning based on the collision probability and generate an instruction set that includes obstacle avoidance priority and movement speed;

[0022] The instruction set of obstacle avoidance priority and movement speed is converted into driving control signals of physical devices and encapsulated into a closed-loop interaction dataset containing NPC behavior instructions and physical device control signals.

[0023] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins according to the present invention, wherein: based on the closed-loop interactive data set, the federated model training is performed at each edge node, and the specific steps are as follows:

[0024] By parsing the closed-loop interaction dataset, we extract NPC behavior instructions, device control signals, and posture verification data, and generate standardized training samples.

[0025] Perform federated model training based on standardized training samples and add dynamic differential noise to the trained federated model;

[0026] The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and uploaded to the cloud aggregation node. The aggregation node performs a weighted average of the parameters of each node and generates the global federated model parameters.

[0027] The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and uploaded to the cloud aggregation node. The aggregation node performs a weighted average of the parameters of each node and generates the global federated model parameters.

[0028] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins according to the present invention, wherein: the federated model gradient is calculated and uploaded to the federated master node, and the master node aggregates the gradient and updates the global federated model parameters. The specific steps are as follows:

[0029] Add Laplace noise to the gradient calculated by the edge node to generate privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node;

[0030] A weighted sum is performed based on the encrypted gradients of each edge node, and the global federated model parameters are updated through natural gradient descent.

[0031] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the method uses global federated model parameters to trigger dynamic navigation and risk warning, and synchronizes the optimized virtual and real scene bidirectional mapping protocol to multiple scenic area digital twins through the metaverse protocol to build a cross-scene virtual cultural tourism platform. The specific steps are as follows:

[0032] Through the global federated model parameters, real-time analysis of tourist locations, scenic area capacity and environmental data, dynamic generation of guided paths, and real-time calculation of dynamic correlation parameters of scenic area tourist distribution and capacity risk;

[0033] Based on the navigation path and dynamic association parameters, they are encoded into bidirectional mapping parameters through the Metaverse protocol and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus;

[0034] Based on the synchronized digital twin nodes, a virtual cultural and tourism platform with cross-scene virtual and real interaction is built.

[0035] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the method for constructing a virtual cultural and tourism platform based on digital twins as described in the first aspect of the present invention.

[0036] In a third aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, wherein: when the computer program is executed by a processor, it implements any step of the method for constructing a virtual cultural and tourism platform based on digital twins as described in the first aspect of the present invention.

[0037] The beneficial effects of the present invention are: by dynamically allocating the accuracy weights of the federated model, combining the real-time video stream frame rate fluctuations to trigger the target detection algorithm to perform semantic segmentation on key areas, calculating the geometric alignment error value to divide the accuracy level, so that the virtual environment and the real world are seamlessly connected, and at the same time, the NPC behavior logic complexity is dynamically switched based on the accuracy grading parameters, which not only improves the realism and interactivity of the virtual character, but also reduces the computing cost by optimizing resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a flowchart of the method for constructing a virtual cultural and tourism platform based on digital twins in Example 1.

[0040] Figure 2 Flowchart of the federated model weight allocation and virtual-real scene mapping of the method for building a virtual cultural tourism platform based on digital twins in Example 1

[0041] Figure 3 This is a flowchart for collision detection and closed-loop interaction data set generation for the method for building a virtual cultural tourism platform based on digital twins in Example 1.

[0042] Figure 4 This is a flowchart of the federated model training and global federated model parameter update of the method for building a virtual cultural and tourism platform based on digital twins in Example 1. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0045] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0046] Example 1, with reference to Figures 1 to 4 This embodiment provides a method for constructing a virtual cultural tourism platform based on digital twins, comprising the following steps:

[0047] S1: Collect real-time data streams from the entire cultural and tourism area, initialize the federated learning framework at the edge point, and generate a federated learning initialization instruction set.

[0048] S1.1: The real-time data stream of the entire cultural and tourism area includes tourist behavior thermal distribution data, environmental parameters, cultural relics status data and real-time video stream data.

[0049] It should be noted that tourist behavior thermal distribution data can accurately capture the gathering and flow trends of tourists in different areas, providing a basis for optimizing the layout of scenic spots and the configuration of service facilities; environmental parameters include information such as temperature, humidity, and air quality, which helps to evaluate and improve the environmental quality of scenic spots; cultural relics status data involves the safety monitoring and protection status of cultural heritage and exhibits, supporting timely measures to prevent potential risks; and real-time video streaming data provides dynamic visual information, which can be used to monitor various activities occurring in scenic spots.

[0050] S1.2: Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the entire cultural and tourism domain, and output a dimensionally aligned local training dataset.

[0051] It should be noted that after initializing the federated learning framework and deploying the lightweight federated learning client on the edge node, the real-time data stream of the cultural and tourism industry is loaded and first cleaned and standardized to eliminate noise, fill missing values, and unify the data format to ensure that data from different sources has a consistent structure and semantic expression. The preset feature engineering rules are based on historical data analysis and task requirements. They are used to guide how to clean, standardize, and select key features of multi-dimensional heterogeneous data to ensure data consistency and validity. The preset feature engineering rules are determined by analyzing historical data patterns and specific application goals to adapt to the characteristics and requirements of different edge nodes. Based on preset feature engineering rules, multi-dimensional heterogeneous data (such as tourist behavior thermal distribution data, environmental parameters, cultural relic status data and real-time video stream data) are mapped into a unified feature space. According to task requirements, the most relevant features to the target are screened out from the multi-dimensional heterogeneous data to ensure the effectiveness and pertinence of the data set, and form a local training data set with consistent dimensions. In this process, the feature weights and dimensional distribution are dynamically adjusted to adapt to the computing power and data characteristics of different edge nodes. The final output local training data set not only achieves dimensional alignment, but also retains the diversity and representativeness of the data, laying a solid foundation for the efficient training of subsequent federated models and global parameter aggregation.

[0052] S1.3: Based on the local training dataset, through multimodal learning, dynamically generate a federated learning initialization instruction set containing the federated model architecture template, aggregation weight rules, and communication protocol parameters.

[0053] Furthermore, by first leveraging the multi-source, heterogeneous data in the local training dataset, a multimodal learning approach is employed to extract key information that reflects the essential characteristics of the data, ensuring that data from different sources can be effectively processed within a unified framework. Next, based on the extracted key features and task requirements, a federated model architecture template suitable for the current application scenario is constructed. This template defines the basic structure and algorithmic logic for collaboration between edge nodes during the federated learning process. At the same time, aggregation weighting rules are determined, assigning corresponding weights based on factors such as the data volume, quality, and computing power of each node, ensuring that the contribution of each node is fully considered during federated model aggregation. Finally, communication protocol parameters are set, defining details such as the format, frequency, and security mechanism for data transmission between nodes to ensure efficient and secure information exchange.

[0054] S2: Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework. At the same time, the video twin engine is used to calibrate the real-time video stream with the three-dimensional model to generate a bidirectional mapping protocol between virtual and real scenes containing accuracy grading parameters and NPC control instructions.

[0055] S2.1: Based on the federated learning initialization instruction set, extract the performance parameters and data reliability indicators of the edge nodes, and dynamically assign the federated model accuracy weight.

[0056] It should be noted that, first, according to the requirements of the federated learning initialization instruction set, device performance parameters are obtained from each edge node, including information such as processing speed, memory capacity, and network bandwidth, and data reliability indicators such as data integrity check results and historical error rates are collected.

[0057] Furthermore, based on the acquired processing speed, memory capacity, network bandwidth, data integrity check results, and historical error rates, scoring criteria are set according to historical data and task requirements, and each node's performance and data reliability are evaluated. Nodes with high performance and high reliability receive higher ratings. The evaluation result is a comprehensive score for each node, reflecting its potential contribution. This score is used to dynamically assign federated model accuracy weights and optimize federated model accuracy. Based on the evaluation results, federated model accuracy weights are dynamically assigned to each participating edge node, ensuring that nodes with higher device performance and more reliable data receive greater weights, thereby playing a more important role in the federated training process. This process ensures efficient resource utilization and maximized model accuracy during federated learning, allowing the final federated learning model to fully benefit from the strengths of each node. In this way, the characteristics of edge nodes are accurately considered and rationally utilized, improving overall learning efficiency and accuracy.

[0058] S2.2: When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the threshold, the target detection algorithm is triggered to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects.

[0059] The specific process involves triggering the object detection algorithm to perform semantic segmentation on key areas and extract the bounding box coordinates of moving objects when the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds a preset threshold. The threshold here is a benchmark value set based on the frame rate fluctuations of historical video stream data, which is used to determine whether the current frame rate fluctuates abnormally.

[0060] Specifically, the threshold is typically set at 10% of the average frame rate. If the frame rate of the live video stream fluctuates beyond this range (for example, if the average frame rate is 30 frames per second, the threshold range is 27 to 33 frames per second), further object detection and semantic segmentation operations are triggered to identify and mark semantic information in key areas, such as tourists, cultural relics, or dynamically changing parts of the environment. This allows for accurate differentiation and positioning of different objects in complex scenes. This process ensures that important information can be captured and processed in a timely manner even when the frame rate fluctuates abnormally.

[0061] S2.3: Based on the bounding box coordinates, calculate the geometric alignment error value and divide the accuracy level into several levels. Generate the accuracy grading parameter including the grid density and algorithm selection rules. The expression is:

[0062] ;

[0063] in, Represents the geometric alignment error value, represents the actual bounding box coordinates, represents the mapping bounding box coordinates, represents the federated model base size, represents the dynamic adjustment coefficient, Indicates the maximum federation weight value, Indicates the federated learning weight of the current node.

[0064] The specific process includes calculating the geometric alignment error value based on the difference between the actual coordinates of the bounding box and the digital twin mapping coordinates, and obtaining a comprehensive error evaluation result reflecting the data mapping accuracy by comparing the coordinate dimension offset of the target detection box and performing normalization processing; then dynamically dividing the accuracy level according to the comprehensive error evaluation result, using a fine-grained grid and a high-precision deep learning registration algorithm when the error is low, and giving the node a higher federated learning weight to enhance the contribution to the global federated model, switching to a balanced mode when the error is medium, using a medium grid density and a lightweight ICP registration algorithm and assigning a moderate weight, and enabling a sparse grid and a fast feature matching method and reducing the weight to reduce the impact of noise when the error is high, and finally generating an accuracy grading parameter including grid density configuration, algorithm selection rules and federated weight value to achieve error-adaptive resource allocation and federated collaborative optimization.

[0065] S2.4: Based on the precision grading parameters, the geometric alignment error value of the current scene and the federated model precision weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine containing logical complexity.

[0066] The specific process involves using precision grading parameters to perform a detailed analysis of the current scene, analyzing the deviation between actual object positions and expected positions, and determining a geometric alignment error value to reflect the level of scene consistency. Based on the federated model's precision weighting, the contribution ratio of edge nodes is adjusted to ensure that high-precision nodes receive a greater weight in model training. Subsequently, based on changes in scene consistency and error values, the NPC behavior engine mode is dynamically switched, optimizing its logical processing mechanism to make NPC behavior more natural and consistent with real-world requirements, adapting to the requirements of scenes of varying complexity. Next, based on the calculated geometric alignment error value and the federated model's precision weighting, the contribution ratio of different edge nodes in federated learning is adjusted to ensure that high-precision nodes play a greater role in model training. Subsequently, based on changes in scene consistency and error values, the NPC behavior engine's operating mode is dynamically switched to accommodate varying logical complexity requirements. During this process, the NPC behavior engine adjusts its internal logical processing mechanism based on the specific requirements of the current scene to optimize NPC behavior.

[0067] Furthermore, through this method based on precision grading parameters and detailed scene analysis, we can achieve accurate evaluation of geometric alignment error values, effectively and dynamically adjust the precision weights of the federated model, and ensure that the NPC behavior engine can flexibly respond to various complex scene changes.

[0068] S2.5: Encapsulate the federated model precision weights, geometric alignment error values, and NPC behavior engine into structured data packets, synchronize them to edge nodes and cloud master nodes through a unified spatiotemporal interface, and generate a bidirectional mapping protocol between virtual and real scenes containing precision grading parameters and NPC control instructions.

[0069] Furthermore, the federated model accuracy weights, geometric alignment error values, and NPC behavior engine are encapsulated into a structured data packet. This process involves integrating the contribution ratio of each edge node to federated learning (i.e., the federated model accuracy weights), the deviation between the actual and expected positions of objects in the current scene (i.e., the geometric alignment error values), and the NPC behavior logic adjusted based on this information (i.e., the NPC behavior engine) into a unified data structure. This ensures that all relevant information can be transmitted and processed in a consistent and standardized manner. Next, it is synchronized to the edge nodes and the cloud master node via a unified spatiotemporal interface. This involves sending the structured data packets to each edge node and the cloud master node using a predefined communication protocol (i.e., the unified spatiotemporal interface). The unified spatiotemporal interface ensures that all nodes receive the same information in both time and space and can make appropriate adjustments or responses based on this information. Based on this, a bidirectional mapping protocol for virtual and real scenes is generated, containing accuracy grading parameters and NPC control commands. Specifically, based on the information collected from each node (such as the federated model accuracy weights and geometric alignment error values), a set of rules (i.e., the bidirectional mapping protocol for virtual and real scenes) is formulated to guide the interaction between the virtual scene and the real world. The bidirectional mapping protocol between virtual and real scenes not only includes how to adjust the data processing method according to different accuracy requirements (i.e., accuracy grading parameters), but also includes specific operation guidelines for NPCs (i.e., NPC control instructions) to optimize NPC behavior performance in different scenarios.

[0070] Specifically, by integrating key parameters, we ensure comprehensive and consistent information. Then, we achieve efficient data synchronization through standard interfaces. Finally, we generate detailed interaction protocols based on this collected information, making the interaction between the virtual and real worlds more precise and effective. This series of steps works together to ensure high levels of data accuracy and rational NPC behavior in complex and changing environments.

[0071] S3: Based on the bidirectional mapping protocol between virtual and real scenes, it scans the three-dimensional geometric structure of the physical space through ray detection and executes the mixed reality collision detection algorithm to generate a closed-loop interaction dataset containing NPC behavior instructions and physical device control signals.

[0072] S3.1: Based on the bidirectional mapping protocol between virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted.

[0073] The specific process involves scanning the three-dimensional geometric structure of the physical space through ray detection based on a bidirectional mapping protocol between virtual and real scenes. This process relies first on the accuracy grading parameters in the protocol and the alignment between the virtual scene and the real environment. Ray detection emits multiple rays from the virtual camera or sensor position, covering key points in the target area, and obtains the spatial coordinates of the physical object surface based on the intersection information between the rays and the physical object surface. This process incorporates the geometric alignment error value in the bidirectional mapping protocol between virtual and real scenes to ensure that the ray detection results accurately reflect the geometric characteristics of the real physical space, while avoiding data deviations caused by error accumulation.

[0074] Specifically, after completing the ray detection, the surface point cloud data of the physical space object is further extracted. By sampling and filtering the spatial coordinates of all ray intersections, noise points are removed and valid data is retained, forming a dense three-dimensional point set. The surface point cloud data of the physical space object not only characterizes the surface morphology of the physical space object, but also provides basic support for subsequent mixed reality collision monitoring and virtual-reality interaction. By combining point cloud data with the accuracy grading parameters in the virtual-reality scene bidirectional mapping protocol, the point cloud density and sampling strategy can be dynamically adjusted, thereby optimizing computational efficiency while ensuring data accuracy.

[0075] S3.2: Align the surface point cloud data with the 3D model coordinate system and execute the mixed reality collision detection algorithm to calculate the collision probability, which is expressed as:

[0076] ;

[0077] in, represents the collision probability, represents the Sigmoid function, Represents the real-time pose of real objects in physical space, Indicates the current position of NPCs and objects in the virtual scene. Indicates the scene base size, Indicates continuous tracking time, represents the decay time constant, represents the time decay function, Relative speed, Indicates the current minimum Euclidean distance, Indicates the angle of motion direction.

[0078] The specific process involves precisely aligning surface point cloud data acquired in physical space with the 3D model in the virtual scene, ensuring their coordinate systems are consistent, thus providing an accurate basis for subsequent collision detection. Next, the collision probability is calculated using a sigmoid function. This function comprehensively considers the real-time pose (i.e., position and attitude) of real objects in physical space, as well as the current pose of NPCs and objects in the virtual scene, while also incorporating the scene's baseline dimensions as a reference scale. Furthermore, the mixed reality collision detection algorithm continuously tracks time and decay time constants, using a time decay function to adjust the temporal trend of the collision probability, reflecting the dynamic changes in collision likelihood within different time intervals. The collision probability calculation is further influenced by relative velocity, which describes the rate of approach between the physical object and the virtual character. Finally, the current minimum Euclidean distance is used to quantify the closest distance between the two objects, directly influencing the probability of a collision.

[0079] S3.3: Dynamically adjust the virtual NPC's path planning based on the collision probability and generate an instruction set including obstacle avoidance priority and movement speed.

[0080] Furthermore, when dynamically adjusting the virtual NPC's path planning based on collision probability, the system first assesses the potential risk of collision between the virtual NPC and real objects in the physical space based on the collision probability calculated by the mixed reality collision detection algorithm. When the collision probability is high, a safe path away from high-risk areas is prioritized for the NPC, and waypoints are recalculated to avoid obstacles. When the collision probability is low, the NPC is allowed to move along the original path while retaining a certain degree of flexibility to respond to unexpected situations. This process combines real-time pose information and geometric structure in the scene to ensure that path adjustments are both efficient and meet the needs of the actual environment.

[0081] Specifically, when generating an instruction set that includes obstacle avoidance priority and movement speed, the obstacle avoidance priority is first set according to the probability of collision. A high probability corresponds to a high priority, requiring the NPC to take avoidance measures as soon as possible, while a low probability allows for lower priority processing. Then, combined with the relative movement speed and the current minimum Euclidean distance, the NPC's movement speed is dynamically adjusted. When approaching an obstacle, it slows down appropriately to reduce the risk of collision, and speeds up in a safe area to improve operating efficiency. Finally, the obstacle avoidance priority and the adjusted movement speed are integrated into a specific instruction set to guide the NPC's behavioral decisions, thereby achieving a more intelligent and smooth virtual-reality interaction experience.

[0082] S3.4: Convert the instruction set of obstacle avoidance priority and movement speed into the driving control signal of the physical device, and encapsulate it into a closed-loop interaction data set containing NPC behavior instructions and physical device control signals.

[0083] The specific process involves mapping obstacle avoidance priorities and speed parameters defined in the instruction set into specific physical device operation instructions. For example, high-priority obstacle avoidance instructions may be converted into emergency braking or steering control signals, while movement speed adjustments correspond to motor speed or actuator power regulation. Through preset conversion rules and device interface protocols, logical instructions are further encoded into a language or signal format adapted for specific hardware, directly driving physical devices to complete corresponding actions, ensuring that the virtual NPC's behavior can be accurately executed in the real environment.

[0084] Furthermore, the converted drive control signals are integrated with the NPC's behavioral instructions to form a unified data structure. This data structure not only contains behavioral information such as the NPC's path planning and obstacle avoidance strategies, but also covers the actual control signals of the corresponding physical devices. The actual control signals are directly generated by analyzing the global federated model parameters and real-time data from edge nodes, combined with preset behavioral instructions. They are used to guide the physical devices to perform specific operations to achieve synchronization between virtual and real interactions. Next, the data structure is marked with timestamps and scene identifiers to ensure the consistency and real-time transmission and parsing of the instruction set between different nodes, ultimately generating a closed-loop interaction dataset.

[0085] S4: Based on the closed-loop interaction dataset, the federated model training is performed on each edge node. The federated model gradient is calculated and uploaded to the federated master node. The master node aggregates the gradient and updates the federated model.

[0086] S4.1: Extract NPC behavior instructions, device control signals, and posture verification data by parsing the closed-loop interaction dataset, and generate standardized training samples.

[0087] The specific process involves parsing the closed-loop interaction dataset and first classifying and extracting the NPC behavioral instructions, device control signals, and posture verification data contained in the closed-loop interaction dataset. Based on data fields and protocol rules, this process identifies and separates behavioral instructions related to NPC path planning and obstacle avoidance strategies, as well as drive control signals corresponding to physical device actions. It also extracts real-time posture information of virtual characters and real objects in the scene. After preliminary screening, this real-time posture information is further cleaned to remove noise and redundant information, ensuring the high accuracy and consistency of the extracted data, laying the foundation for subsequent training sample generation.

[0088] Furthermore, when generating standardized training samples, the extracted NPC behavior commands, device control signals, and posture verification data are encoded and normalized according to a unified format. By mapping data of different dimensions into the same feature space and combining timestamps and scene context information, a structured sample set is constructed. Furthermore, standardized training samples are annotated according to task requirements, such as by marking collision risk level or behavior execution effect, thus forming standardized training samples with clear labels.

[0089] S4.2: Perform federated model training based on the standardized training samples and add dynamic differential noise to the trained federated model.

[0090] It should be noted that the federated model is first trained using standardized training samples. These samples include preprocessed and annotated datasets, ensuring that each participating edge node learns from the same baseline. During training, each edge node updates its local model based on local data and aggregates the updated results into the global federated model to gradually optimize the federated model.

[0091] Specifically, this process involves calculating an appropriate amount of noise based on specific privacy protection requirements and applying it to the federated model. This approach effectively prevents potential privacy leaks without significantly impacting the performance of the federated model. Ultimately, through this federated model training based on standardized training samples and the dynamic differential noise addition steps, effective training and privacy protection of the federated model are achieved, ensuring the accuracy and security of the federated model in practical applications.

[0092] S4.3: Perform homomorphic encryption on the federated model with added noise, generate an encrypted parameter package, and upload it to the cloud aggregation node. The aggregation node performs a weighted average of the parameters of each node and generates the global federated model parameters, which are expressed as:

[0093] ;

[0094] in, represents the global federated model parameters, represents the homomorphic decryption function, represents the total number of edge nodes, represents the edge node index, represents the homomorphic encryption function, Indicates the Global federation model parameters for edge nodes, represents the differential privacy sensitivity, represents the privacy budget, Indicates the The federation weight of edge nodes, represents the standard Laplace noise generator, represents the homomorphic encryption public key, Represents a homomorphically encrypted private key.

[0095] The specific process involves homomorphically encrypting the noisy global federated model parameters during federated learning to generate an encrypted parameter package, ensuring data security during transmission. After the encrypted parameter package is uploaded to the cloud aggregation node, the aggregation node leverages the properties of homomorphic encryption to perform a weighted average on the encrypted parameters without decrypting them.

[0096] Specifically, the parameters of each edge node are weighted according to the federation weight, which reflects each edge node's contribution to the global federated model. This weight is combined with differential privacy sensitivity and privacy budget to ensure effective privacy protection. Finally, the aggregated results are restored to the global federated model parameters through a homomorphic decryption function, completing the update of the global federated model.

[0097] The global federated model parameters refer to the unified parameter values generated by weighted averaging of the global federated models of each edge node in federated learning.

[0098] S4.4: Add Laplace noise to the gradient calculated by the edge node to generate the privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node. The expression is:

[0099] ;

[0100] in, Indicates the The edge node is Noisy gradients during training epochs, Indicates the sequence number of the training round, Indicates the The edge node is The unadded noise of the wheel, represents the noise gradient, Indicates the Dynamic gradient sensitivity of round training, represents the dynamic adjustment function, Indicates the The federation weight of edge nodes, Indicates the Real-time resource indicators of edge nodes, represents the resource weight coupling function value, Indicates based on training rounds Dynamic noise control function, Represents rounds over time Dynamically decaying privacy budget function, represents a skewed normal distribution, Represents positional parameters, represents the scale parameter, represents the skewness parameter.

[0101] The specific process involves adding Laplace noise to the gradients calculated by edge nodes during each round of federated learning training to generate privacy-preserving gradients. This process uses a dynamic adjustment function and a dynamic noise control function based on training rounds to determine the appropriate amount of noise, ensuring that data privacy is protected while not affecting the performance of the global federated model.

[0102] Furthermore, the raw gradients of each edge node are weighted based on the federation weight, real-time resource metrics, and the resource-weight coupling function. An appropriate amount of Laplace noise is then added to make the data contribution of a single node difficult to reverse engineer. To further enhance security, the noisy gradients are encrypted using the Paillier encryption algorithm. The encrypted gradients are then uploaded to the cloud aggregation node, where they can be securely aggregated without leaking any private information. This approach not only effectively protects the data privacy of all participating parties but also facilitates efficient federated model training under multi-party collaboration.

[0103] S4.5: Perform weighted summation based on the encrypted gradients of each edge node and update the global federated model parameters through natural gradient descent. The expression is:

[0104] ;

[0105] in, Indicates the The global federated model parameters after round of updates, Indicates the Global federated model parameters during round training, represents the step size coefficient, represents the total number of edge nodes, represents the homomorphic decryption operation, Indicates the The noise gradient of edge nodes, Indicates the The federation weight of edge nodes, represents the homomorphic encryption private key, represents element-wise multiplication, Represents the Fisher information matrix The low-rank approximate inverse matrix of represents the Riemannian manifold regularization term.

[0106] The specific process involves performing a weighted summation of the encrypted gradients from each edge node during each training round, and updating the global federated model parameters via natural gradient descent. First, encrypted and noisy gradients are obtained from each edge node, and then a homomorphic decryption operation is used to restore the encrypted gradients to a processable form. The encrypted gradients are then weighted and summed based on each node's federation weight, ensuring that nodes with greater contributions have greater influence during the update process.

[0107] Next, they employed natural gradient descent, combined with a step-size coefficient to adjust the update amplitude. They also used the low-rank approximate inverse of the Fisher information matrix to optimize the update direction. Furthermore, a Riemannian manifold regularization term was implemented to maintain the well-defined geometric properties of the global federated model parameter space. This approach not only effectively updates the global federated model parameters, but also ensures data privacy protection and efficient global federated model parameter optimization through multi-party collaboration. This process comprehensively considers the data contributions and real-time resource availability of different nodes, ensuring that the global federated model parameters can continuously improve performance while preserving privacy.

[0108] S5: Use global federated model parameters to trigger dynamic navigation and risk warnings, and synchronize the optimized virtual and real scene bidirectional mapping protocol to multiple scenic spot digital twins through the Metaverse protocol to build a cross-scene virtual cultural tourism platform.

[0109] S5.1: Analyze visitor locations, scenic area capacity, and environmental data in real time through global federated model parameters, dynamically generate guide paths, and calculate dynamic correlation parameters of scenic area visitor distribution and capacity risk in real time. The expression is:

[0110] ;

[0111] in, Indicates the The dynamic correlation parameters of the scenic area tourist distribution and capacity risk, represents the smooth positive activation function, represents the mathematical expectation, represents the probability density function, Represents the spatial coordinate vector within the scenic area, Indicates in The global federated model parameters updated after the sub-global aggregation, represents the scenic area capacity threshold matrix, represents the path gradient penalty coefficient, Indicates the Wheel Guided Path The spatial gradient of Represents the element-wise multiplication algorithm for two vectors.

[0112] The specific process involves using federated aggregation to update the parameters of the global federated model to assess the current visitor distribution within a scenic area and its impact on its capacity. A smoothed positive activation function processes the mathematical expectation and probability density function, combined with the scenic area's spatial coordinate vectors, to accurately calculate the visitor density and potential capacity risk in each area. This process also considers the scenic area's capacity threshold matrix and path gradient penalty coefficients, ensuring that the generated guided paths not only guide tourists to avoid crowded areas but also optimize the overall tour experience.

[0113] Furthermore, a path gradient penalty coefficient is used to adjust the spatial gradient of the guided path, preventing it from being too concentrated or too dispersed. This effectively manages the distribution of visitors within the scenic area and reduces the risk of overload. Ultimately, through comprehensive analysis and calculation, not only does this achieve the goal of dynamically generating optimal guided paths, but it also enables real-time monitoring and adjustment of visitor distribution and capacity risks within the scenic area.

[0114] S5.2 is based on the navigation path and dynamic association parameters, which are encoded into bidirectional mapping parameters through the Metaverse protocol and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus.

[0115] The specific process involves structured integration of the generated guide paths with dynamic correlation parameters such as visitor distribution and capacity risk within scenic areas, ensuring a clear and consistent mapping between virtual and real scenes. Multi-scenic area digital twin nodes are formed by integrating the actual needs of physical scenic areas, leveraging blockchain technology to ensure data consistency and security, and utilizing the Metaverse protocol to standardize the processing of complex information. These nodes form a distributed network, enabling each scenic area to not only optimize its own operations locally but also collaborate efficiently with other scenic areas to improve overall service quality. Subsequently, utilizing the standardized encoding rules of the Metaverse protocol, the dynamic correlation parameters are converted into a unified format suitable for multi-scenic area digital twins, enabling seamless integration and collaborative management across different scenic areas. Furthermore, the encoded bidirectional mapping parameters are verified and synchronized through a blockchain consensus mechanism, ensuring that each scenic area's digital twin nodes can securely and reliably access and update data. This supports real-time interaction and optimized resource allocation across scenarios, laying the foundation for building a highly interconnected virtual cultural and tourism platform.

[0116] S5.2: Based on the synchronized digital twin node parameters, a virtual cultural tourism platform with cross-scenario virtual-reality interaction is constructed.

[0117] Furthermore, the synchronized parameters are used to update the digital twins of each scenic spot to ensure a high degree of consistency and dynamic linkage between the virtual scene and the real environment. Subsequently, the digital twin node parameters of different scenic spots are associated and integrated to form a unified cross-scene data framework, supporting tourists to switch and interact seamlessly between multiple scenic spots. On this basis, combined with the virtual-reality mapping protocol and interaction logic, real-time synchronization of virtual characters, guide paths, and physical equipment control signals is achieved, providing tourists with an immersive tour experience. In this way, not only is the efficient coordination and optimized allocation of resources in multiple scenic spots achieved, but the coherence and sense of participation of tourists in cross-scene tours are also enhanced, creating a highly intelligent and interactive virtual cultural tourism platform.

[0118] This embodiment also provides a computer device, which is suitable for the method of constructing a virtual cultural and tourism platform based on digital twins, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the method of constructing a virtual cultural and tourism platform based on digital twins proposed in the above embodiment.

[0119] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0120] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for constructing a virtual cultural and tourism platform based on digital twins as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0121] In summary, the present invention achieves seamless integration of the virtual environment and the real world by dynamically allocating precision weights to the federated model, triggering the target detection algorithm to perform semantic segmentation on key areas in combination with fluctuations in the frame rate of the real-time video stream, and calculating geometric alignment error values to divide the precision levels. Furthermore, the complexity of the NPC behavior logic is dynamically switched based on the precision grading parameters, thereby improving the realism and interactivity of the virtual characters and reducing the computing cost by optimizing resource usage.

[0122] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a virtual cultural tourism platform based on digital twins, characterized by: include, Collect real-time data streams from the entire cultural and tourism industry, initialize the federated learning framework at the edge, and generate a federated learning initialization instruction set; Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework. At the same time, the video twin engine is used to calibrate the real-time video stream with the 3D model to generate a bidirectional mapping protocol between virtual and real scenes containing accuracy grading parameters and NPC control instructions. The specific steps are as follows: Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided. The accuracy grading parameter including the grid density and algorithm selection rules is generated. The expression is: ； in, Represents the geometric alignment error value, represents the actual bounding box coordinates, represents the mapping bounding box coordinates, represents the federated model base size, represents the dynamic adjustment coefficient, Indicates the maximum federation weight value, Indicates the federated learning weight of the current node; Based on the bidirectional mapping protocol of virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned by ray detection and the mixed reality collision detection algorithm is executed to generate a closed-loop interaction dataset containing NPC behavior instructions and physical device control signals. The specific steps are as follows: Based on the bidirectional mapping protocol of virtual and real scenes, the 3D geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted; Based on the alignment of the surface point cloud data with the 3D model coordinate system, the mixed reality collision detection algorithm is executed to calculate the collision probability, which is expressed as: ； in, represents the collision probability, represents the Sigmoid function, Represents the real-time pose of real objects in physical space, Indicates the current position of NPCs and objects in the virtual scene. Indicates the scene base size, Indicates continuous tracking time, represents the decay time constant, represents the time decay function, Relative speed, Indicates the current minimum Euclidean distance, Indicates the angle of motion direction; Dynamically adjust the virtual NPC's path planning based on the collision probability and generate an instruction set that includes obstacle avoidance priority and movement speed; Convert the obstacle avoidance priority and movement speed instruction set into the driving control signal of the physical device, and encapsulate it into a closed-loop interaction data set containing NPC behavior instructions and physical device control signals; Based on the closed-loop interaction dataset, federated model training is performed on each edge node. The federated model gradient is calculated and uploaded to the federated master node. The master node aggregates the gradient and updates the global federated model parameters. Global federated model parameters are used to trigger dynamic navigation and risk warnings, and the optimized virtual and real scene bidirectional mapping protocol is synchronized to multiple scenic spot digital twins through the Metaverse protocol to build a cross-scene virtual cultural tourism platform.

2. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 1, characterized in that: The real-time data stream of the entire cultural and tourism area includes tourist behavior thermal distribution data, environmental parameters, cultural relics status data and real-time video stream data.

3. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 2, characterized in that: Initializing the federated learning framework at the edge point and generating the federated learning initialization instruction set are as follows: Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the cultural and tourism industry, and output a dimensionally aligned local training dataset. Based on the local training dataset, through multimodal learning, a federated learning initialization instruction set containing the federated model architecture template, aggregation weight rules and communication protocol parameters is dynamically generated.

4. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 3, characterized in that: Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework. At the same time, the video twin engine is used to calibrate the real-time video stream with the three-dimensional model to generate a virtual-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions. The specific steps are as follows: Based on the federated learning initialization instruction set, the performance parameters and data reliability indicators of edge nodes are extracted, and the accuracy weight of the federated model is dynamically allocated; When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the stability threshold, it triggers the target detection algorithm to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects; Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided, and the accuracy grading parameters including mesh density and algorithm selection rules are generated; Based on the accuracy grading parameters, the geometric alignment error value of the current scene and the federated model accuracy weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine including logical complexity; The federated model precision weights, geometric alignment error values, and NPC behavior engine are encapsulated into structured data packets, synchronized to edge nodes and cloud master nodes through a unified spatiotemporal interface, and a bidirectional mapping protocol for virtual and real scenes containing precision grading parameters and NPC control instructions is generated.

5. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 4, characterized in that: Based on the closed-loop interaction dataset, the federated model training is performed at each edge node. The specific steps are as follows: By parsing the closed-loop interaction dataset, we extract NPC behavior instructions, physical device control signals, and posture verification data, and generate standardized training samples. Perform federated model training based on standardized training samples and add dynamic differential noise to the trained federated model; The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and uploaded to the cloud aggregation node. The aggregation node performs a weighted average of the parameters of each node and generates the global federated model parameters.

6. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 5, characterized in that: The federated model gradient is calculated and uploaded to the federated master node. The master node aggregates the gradient and updates the global federated model parameters. The specific steps are as follows: Add Laplace noise to the gradient calculated by the edge node to generate privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node; A weighted sum is performed based on the encrypted gradients of each edge node, and the global federated model parameters are updated through natural gradient descent.

7. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 6, characterized in that: The global federated model parameters are used to trigger dynamic navigation and risk warnings, and the optimized virtual and real scene bidirectional mapping protocol is synchronized to multiple scenic spot digital twins through the metaverse protocol to build a cross-scene virtual cultural tourism platform. The specific steps are as follows: Through the global federated model parameters, real-time analysis of tourist locations, scenic area capacity and environmental data, dynamic generation of guided paths, and real-time calculation of dynamic correlation parameters of scenic area tourist distribution and capacity risk; Based on the navigation path and dynamic association parameters, they are encoded into bidirectional mapping parameters through the Metaverse protocol and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus; Based on the synchronized digital twin nodes, a virtual cultural and tourism platform with cross-scene virtual and real interaction is built.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for constructing a virtual cultural and tourism platform based on digital twins as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a virtual cultural and tourism platform based on digital twins according to any one of claims 1 to 7 are implemented.

Images