Virtual travel platform construction method based on digital twinning

Abstract

The invention discloses a digital twinning-based virtual travel platform construction method, which relates to the technical field of digital twinning and cultural travel, and comprises the following steps of: initializing an instruction set based on federated learning, dynamically distributing a federated model precision weight through a federated learning framework, and calibrating a real-time video stream and a three-dimensional model by using a video twinning engine to obtain a real-time video stream; generating a virtual-real scene bidirectional mapping protocol containing a precision grading parameter and an NPC control instruction; and based on a virtual-real scene bidirectional mapping protocol, scanning a physical space three-dimensional geometric structure through ray detection, executing a mixed reality collision monitoring algorithm, and generating a closed-loop interaction data set containing an NPC behavior instruction and a physical equipment control signal. According to the method, the precision weight of the federal model is dynamically allocated, a target detection algorithm is triggered by combining the real-time video stream frame rate fluctuation to perform semantic segmentation on a key region, and the geometric alignment error value is calculated to divide the precision level, so that the virtual environment and the real world are seamlessly jointed.

Description

Technical Field

[0001] The present invention relates to the field of digital twin and cultural tourism technology, and in particular to a method for constructing a virtual cultural tourism platform based on digital twin. Background Art

[0002] Traditional cultural tourism experience mainly relies on direct interaction between the physical space of the physical scenic spot and tourists. This model has significant limitations in terms of information transmission efficiency, personalized service provision, and real-time feedback mechanism. In recent years, digital twin technology has gradually emerged. By creating a digital mirror of the physical environment, it has achieved high-precision simulation and dynamic update of the actual scene. However, in existing cultural tourism applications, digital twin technology is mostly limited to static display or limited interactive functions, and has failed to fully utilize its potential to enhance the tourist experience.

[0003] At present, one of the main challenges facing the cultural and tourism industry is how to achieve efficient data sharing and analysis to support personalized tourist services while protecting user privacy. Existing cultural and tourism platforms are often unable to effectively integrate data from different sources, resulting in a lack of pertinence and flexibility in the services provided. In addition, although some advanced virtual reality (VR) and augmented reality (AR) technologies have been applied to enhance the tourist experience, the application of these technologies is usually independent of actual operational management and decision support, limiting their potential in optimizing resource allocation and improving management efficiency. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for constructing a virtual cultural and tourism platform based on digital twins to solve the problem.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In the first aspect, the present invention provides a method for constructing a virtual cultural and tourism platform based on digital twins, which includes collecting real-time data streams of the entire cultural and tourism domain, initializing a federated learning framework at the edge point, and generating a federated learning initialization instruction set; based on the federated learning initialization instruction set, dynamically allocating the accuracy weight of the federated model through the federated learning framework, and using the video twin engine to calibrate the real-time video stream with the three-dimensional model to generate a virtual-to-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions; based on the virtual-to-real scene bidirectional mapping protocol, scanning the three-dimensional geometric structure of the physical space through ray detection and executing a mixed reality collision monitoring algorithm to generate a closed-loop interactive data set containing NPC behavior instructions and physical device control signals; based on the closed-loop interactive data set, executing federated model training at each edge node, calculating the federated model gradient and uploading it to the federated master node, and the master node aggregates the gradient and updates the global federated model parameters; using the global federated model parameters to trigger dynamic navigation and risk warning, and synchronizing the optimized virtual-to-real scene bidirectional mapping protocol to multiple scenic spot digital twins through the metaverse protocol to build a cross-scene virtual cultural and tourism platform.

[0007] As a preferred solution of the method for constructing a virtual cultural and tourism platform based on digital twins described in the present invention, the real-time data stream of the entire cultural and tourism area includes thermal distribution data of tourist behavior, environmental parameters, cultural relics status data and real-time video stream data.

[0008] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the specific steps of initializing the federated learning framework at the edge point and generating a federated learning initialization instruction set are as follows: Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the entire cultural and tourism domain, and output a dimensionally aligned local training data set; Based on the local training data set, through multimodal learning, a federated learning initialization instruction set containing a federated model architecture template, aggregation weight rules, and communication protocol parameters is dynamically generated.

[0009] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the federated learning initialization instruction set is used to dynamically allocate the federated model accuracy weight through the federated learning framework, and the video twin engine is used to calibrate the real-time video stream with the three-dimensional model to generate a virtual-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions. The specific steps are as follows: Based on the federated learning initialization instruction set, the device performance parameters and data reliability indicators of the edge nodes are extracted, and the accuracy weight of the federated model is dynamically allocated; When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the stability threshold, it triggers the target detection algorithm to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects; Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided to generate the accuracy grading parameters including the grid density and algorithm selection rules; Based on the accuracy grading parameters, the geometric alignment error value of the current scene and the federated model accuracy weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine containing logical complexity; The federated model precision weights, geometric alignment error values, and NPC behavior engine are encapsulated into structured data packets, synchronized to edge nodes and cloud master nodes through a unified spatiotemporal interface, and a bidirectional mapping protocol for virtual and real scenes containing precision grading parameters and NPC control instructions is generated.

[0010] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the following specific steps are used to generate a closed-loop interactive data set containing NPC behavior instructions and physical device control signals based on the virtual-real scene bidirectional mapping protocol, by scanning the three-dimensional geometric structure of the physical space through ray detection and executing the mixed reality collision monitoring algorithm. Based on the bidirectional mapping protocol of virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted; Align the surface point cloud data with the 3D model coordinate system and execute the mixed reality collision detection algorithm to calculate the collision probability; According to the collision probability, the virtual NPC's path planning is dynamically adjusted, and an instruction set including obstacle avoidance priority and movement speed is generated; The instruction set of obstacle avoidance priority and moving speed is converted into driving control signals of physical devices and encapsulated into a closed-loop interaction data set containing NPC behavior instructions and physical device control signals.

[0011] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, wherein: based on the closed-loop interactive data set, the federated model training is performed at each edge node, and the specific steps are as follows: By parsing the closed-loop interaction dataset, NPC behavior instructions, device control signals, and posture verification data are extracted, and standardized training samples are generated; Perform federated model training based on standardized training samples and add dynamic differential noise to the trained federated model; The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and upload it to the cloud aggregation node. The aggregation node performs weighted average of the parameters of each node and generates the global federated model parameters.

[0012] The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and upload it to the cloud aggregation node. The aggregation node performs weighted average of the parameters of each node and generates the global federated model parameters.

[0013] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, wherein: the federated model gradient is calculated and uploaded to the federated master node, and the master node aggregates the gradient and updates the global federated model parameters. The specific steps are as follows: Add Laplace noise to the gradient calculated by the edge node to generate privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node; A weighted sum is performed based on the encrypted gradients of each edge node, and the global federated model parameters are updated through natural gradient descent.

[0014] As a preferred solution of the method for constructing a virtual cultural tourism platform based on digital twins described in the present invention, the global federated model parameters are used to trigger dynamic navigation and risk warning, and the optimized virtual and real scene bidirectional mapping protocol is synchronized to multiple scenic spot digital twins through the metaverse protocol to build a cross-scene virtual cultural tourism platform. The specific steps are as follows: Through the global federated model parameters, the tourist location, scenic area capacity and environmental data are analyzed in real time, the guide path is dynamically generated, and the dynamic correlation parameters of the scenic area tourist distribution and capacity risk are calculated in real time; Based on the guided path and dynamic association parameters, they are encoded into bidirectional mapping parameters through the Metaverse protocol and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus; Based on the synchronized digital twin nodes, a virtual cultural and tourism platform with cross-scene virtual and real interaction is built.

[0015] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the method for constructing a virtual cultural and tourism platform based on digital twins as described in the first aspect of the present invention.

[0016] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for constructing a virtual cultural and tourism platform based on digital twins as described in the first aspect of the present invention.

[0017] The beneficial effects of the present invention are as follows: by dynamically allocating the accuracy weights of the federated model, combining the real-time video stream frame rate fluctuation to trigger the target detection algorithm to perform semantic segmentation on key areas, calculating the geometric alignment error value to divide the accuracy level, so that the virtual environment and the real world are seamlessly connected, and at the same time, the NPC behavior logic complexity is dynamically switched based on the accuracy grading parameters, which not only improves the realism and interactivity of the virtual character, but also reduces the computing cost by optimizing resource usage. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 This is a flow chart of the method for constructing a virtual cultural and tourism platform based on digital twins in Example 1.

[0020] Figure 2 Flow chart of federated model weight allocation and virtual-real scene mapping of the method for building a virtual cultural tourism platform based on digital twins in Example 1 Figure 3 This is a flowchart for generating collision detection and closed-loop interaction data sets for the method for building a virtual cultural tourism platform based on digital twins in Example 1.

[0021] Figure 4 This is a flowchart of the federated model training and global federated model parameter update of the method for building a virtual cultural and tourism platform based on digital twins in Example 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0025] Example 1, reference Figure 1~Figure 4 , this embodiment provides a method for constructing a virtual cultural tourism platform based on digital twins, comprising the following steps: S1: Collect real-time data streams from the entire cultural and tourism domain, initialize the federated learning framework at the edge point, and generate a federated learning initialization instruction set.

[0026] S1.1: The real-time data stream of the entire cultural and tourism area includes tourist behavior thermal distribution data, environmental parameters, cultural relics status data and real-time video stream data.

[0027] It should be noted that the thermal distribution data of tourist behavior can accurately capture the gathering and flow trends of tourists in different areas, and provide a basis for optimizing the layout of scenic spots and the configuration of service facilities; environmental parameters include information such as temperature, humidity, and air quality, which helps to evaluate and improve the environmental quality of scenic spots; cultural relics status data involves the safety monitoring and protection status of cultural heritage and exhibits, and supports timely measures to prevent potential risks; and real-time video streaming data provides dynamic visual information, which can be used to monitor various activities occurring in scenic spots.

[0028] S1.2: Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the entire cultural and tourism domain, and output a dimensionally aligned local training data set.

[0029] It should be noted that after the edge node initializes the federated learning framework and deploys the lightweight federated learning client, the real-time data stream of the entire cultural and tourism domain is loaded, and the real-time data stream of the entire cultural and tourism domain is first cleaned and standardized to eliminate noise, fill missing values ​​and unify the data format to ensure that data from different sources have consistent structure and semantic expression; the preset feature engineering rules are based on the standards set by historical data analysis and task requirements, which are used to guide how to clean, standardize and select key features of multi-dimensional heterogeneous data to ensure data consistency and validity. The preset feature engineering rules are determined by analyzing previous data patterns and specific application goals to adapt to the characteristics and requirements of different edge nodes. Based on preset feature engineering rules, multi-dimensional heterogeneous data (such as thermal distribution data of tourist behavior, environmental parameters, cultural relics status data and real-time video stream data) are mapped into a unified feature space, and according to task requirements, the most relevant features to the target are screened out from the multi-dimensional heterogeneous data to ensure the effectiveness and pertinence of the data set and form a local training data set with consistent dimensions. In this process, the feature weights and dimensional distribution are dynamically adjusted to adapt to the computing power and data characteristics of different edge nodes. The final output local training data set not only achieves dimensional alignment, but also retains the diversity and representativeness of the data, laying a solid foundation for the efficient training of subsequent federated models and global parameter aggregation.

[0030] S1.3: Based on the local training data set, through multimodal learning, dynamically generate a federated learning initialization instruction set containing a federated model architecture template, aggregation weight rules, and communication protocol parameters.

[0031] Furthermore, we first take advantage of the characteristics of multi-source heterogeneous data in the local training data set and use multimodal learning methods to extract key information that can reflect the essential characteristics of the data, ensuring that data from different sources can be effectively processed under a unified framework. Then, based on the extracted key features and task requirements, we build a federated model architecture template suitable for the current application scenario. The federated model architecture template clarifies the basic structure and algorithm logic of the collaboration between edge nodes in the federated learning process. At the same time, we determine the aggregation weight rules, and assign corresponding weights based on factors such as the amount, quality, and computing power of each node to ensure that the contribution of each node is fully considered when the federated model is aggregated. Finally, we set the communication protocol parameters and define details such as the format, frequency, and security mechanism of data transmission between nodes to ensure efficient and secure information exchange.

[0032] S2: Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework. At the same time, the video twin engine is used to calibrate the real-time video stream with the three-dimensional model to generate a virtual-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions.

[0033] S2.1: Based on the federated learning initialization instruction set, the performance parameters and data reliability indicators of the edge nodes are extracted, and the accuracy weights of the federated model are dynamically allocated.

[0034] It should be explained that, first, according to the requirements of the federated learning initialization instruction set, device performance parameters are obtained from each edge node, including information such as processing speed, memory capacity, and network bandwidth, and data reliability indicators are collected, such as data integrity check results and historical error rates.

[0035] Furthermore, based on the acquired processing speed, memory capacity, network bandwidth, data integrity check results and historical error rate. Scoring criteria are set according to historical data and task requirements, and the performance and data reliability of each node are evaluated. Nodes with high performance and high reliability receive higher evaluations. The evaluation result is a comprehensive score for each node, reflecting its potential contribution, which is used to dynamically allocate federated model accuracy weights and optimize the accuracy of the federated model. According to the evaluation results, the federated model accuracy weights are dynamically allocated to each participating edge node to ensure that nodes with higher device performance and more reliable data obtain greater weights, thereby playing a more important role in the joint training process. This process ensures the effective use of resources and the maximization of model accuracy in the federated learning process, so that the final federated learning model can fully benefit from the advantages of each node. In this way, the precise consideration and rational use of edge node characteristics are achieved, and the overall learning efficiency and accuracy are improved.

[0036] S2.2: When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the threshold, the target detection algorithm is triggered to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects.

[0037] The specific process includes: when the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the preset threshold, it triggers the target detection algorithm to perform semantic segmentation on the key area and extract the bounding box coordinates of the moving object. The threshold here is a benchmark value set based on the frame rate fluctuation of the historical video stream data, which is used to determine whether the current frame rate fluctuates abnormally.

[0038] Specifically, the threshold is usually set to 10% above or below the average frame rate. That is, if the frame rate of the real-time video stream fluctuates beyond this range (for example, if the average frame rate is 30 frames per second, the threshold range is 27 to 33 frames per second), further object detection and semantic segmentation operations will be triggered to identify and mark the semantic information in key areas, such as tourists, cultural relics, or dynamically changing parts of the environment, so as to achieve accurate distinction and positioning of different objects in complex scenes. This process ensures that important information can be captured and processed in a timely manner when the frame rate fluctuates abnormally.

[0039] S2.3: Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided to generate the accuracy grading parameter including the grid density and algorithm selection rules. The expression is: ; in, Represents the geometric alignment error value, represents the actual bounding box coordinates, represents the mapping bounding box coordinates, represents the federated model base size, represents the dynamic adjustment coefficient, Indicates the maximum federation weight value, Represents the federated learning weight of the current node.

[0040] The specific process includes calculating the geometric alignment error value based on the difference between the actual coordinates of the bounding box and the mapped coordinates of the digital twin, and obtaining a comprehensive error evaluation result reflecting the data mapping accuracy by comparing the coordinate dimension offset of the target detection box and performing normalization processing; then, the accuracy level is dynamically divided according to the comprehensive error evaluation result. When the error is low, a fine-grained grid and a high-precision deep learning registration algorithm are used. At the same time, a higher federated learning weight is given to the node to enhance the contribution to the global federated model. When the error is medium, it switches to balanced mode using a medium grid density and a lightweight ICP registration algorithm and assigning a moderate weight. When the error is high, a sparse grid and a fast feature matching method are enabled and the weight is reduced to reduce the impact of noise. Finally, the accuracy grading parameters including grid density configuration, algorithm selection rules and federation weight values ​​are generated to achieve error-adaptive resource allocation and federated collaborative optimization.

[0041] S2.4: Based on the accuracy grading parameters, the geometric alignment error value of the current scene and the federated model accuracy weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine containing logical complexity.

[0042] The specific process includes using the precision grading parameters to conduct a detailed analysis of the current scene, analyzing the deviation between the actual object and the expected position, and determining the geometric alignment error value to reflect the level of scene consistency. Combined with the federated model precision weight, adjust the contribution ratio of the edge nodes to ensure that high-precision nodes occupy a larger proportion in model training. Subsequently, according to the changes in scene consistency and error values, dynamically switch the NPC behavior engine mode, optimize its logic processing mechanism, make the NPC behavior performance more natural and meet the needs of the actual environment, and adapt to the requirements of scenes of different complexity. Next, according to the calculated geometric alignment error value, combined with the federated model precision weight, adjust the contribution ratio of different edge nodes in federated learning to ensure that high-precision nodes play a greater role in model training. Subsequently, according to the changes in scene consistency and error values, dynamically switch the working mode of the NPC behavior engine to adapt to different logic complexity requirements. In this process, the NPC behavior engine will adjust the internal logic processing mechanism according to the specific requirements of the current scene to optimize the NPC's behavior performance.

[0043] Furthermore, through this method based on precision grading parameters and detailed scene analysis, an accurate evaluation of the geometric alignment error value is achieved, and the precision weight of the federated model is effectively and dynamically adjusted, while ensuring that the NPC behavior engine can flexibly respond to various complex scene changes.

[0044] S2.5: Encapsulate the federated model precision weights, geometric alignment error values, and NPC behavior engine into structured data packets, synchronize them to edge nodes and cloud master nodes through a unified spatiotemporal interface, and generate a bidirectional mapping protocol between virtual and real scenes containing precision grading parameters and NPC control instructions.

[0045] Furthermore, the federated model precision weight, geometric alignment error value and NPC behavior engine are encapsulated into a structured data packet. This process involves integrating the contribution ratio of each edge node in federated learning (i.e., the federated model precision weight), the deviation between the actual object and the expected position in the current scene (i.e., the geometric alignment error value), and the NPC behavior logic adjusted according to this information (i.e., the NPC behavior engine) into a unified data structure. This is done to ensure that all relevant information can be transmitted and processed in a consistent and standardized manner. Next, it is synchronized to the edge node and the cloud master node through the spatiotemporal unified interface. This means that the above structured data packets are sent to each edge node and the cloud master node using a predefined communication protocol (i.e., the spatiotemporal unified interface). The spatiotemporal unified interface ensures that all nodes can receive the same information in both time and space dimensions, and can make corresponding adjustments or responses based on this information. On this basis, a virtual-real scene bidirectional mapping protocol containing precision grading parameters and NPC control instructions is generated. Specifically, based on the information collected from each node (such as the federated model precision weight, geometric alignment error value, etc.), a set of rules (i.e., the virtual-real scene bidirectional mapping protocol) is formulated to guide the interaction between the virtual scene and the real world. The bidirectional mapping protocol between virtual and real scenes not only includes how to adjust the data processing method according to different accuracy requirements (i.e., accuracy grading parameters), but also includes specific operation guidelines for NPCs (i.e., NPC control instructions) to optimize the NPC's behavior performance in different scenarios.

[0046] Specifically, by integrating key parameters, the comprehensiveness and consistency of information are ensured; then, efficient data synchronization is achieved through standard interfaces; finally, detailed interaction protocols are generated based on the collected information, making the interaction between the virtual and real worlds more accurate and effective. This series of steps work together to ensure that a high level of data accuracy and the rationality of NPC behavior can be maintained in a complex and changing environment.

[0047] S3: Based on the bidirectional mapping protocol between virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection and the mixed reality collision monitoring algorithm is executed to generate a closed-loop interaction dataset containing NPC behavior instructions and physical device control signals.

[0048] S3.1: Based on the bidirectional mapping protocol between virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted.

[0049] The specific process includes: based on the virtual-real scene bidirectional mapping protocol, the process of scanning the three-dimensional geometric structure of the physical space through ray detection first depends on the accuracy grading parameters in the protocol and the alignment relationship between the virtual scene and the real environment. Ray detection emits multiple rays from the virtual camera or sensor position to cover the key points of the target area, and obtains the spatial coordinates of the surface of the physical space object based on the intersection information between the ray and the surface of the physical space object. This process combines the geometric alignment error value in the virtual-real scene bidirectional mapping protocol to ensure that the ray detection results can accurately reflect the geometric characteristics of the real physical space, while avoiding data deviations caused by error accumulation.

[0050] Specifically, after completing the ray detection, the surface point cloud data of the physical space object is further extracted. By sampling and filtering the spatial coordinates of all ray intersections, noise points are removed and valid data is retained to form a set of dense three-dimensional point sets. The surface point cloud data of the physical space object not only characterizes the surface morphology of the physical space object, but also provides basic support for subsequent mixed reality collision monitoring and virtual-reality interaction. By combining the point cloud data with the accuracy grading parameters in the virtual-reality scene bidirectional mapping protocol, the point cloud density and sampling strategy can be dynamically adjusted to optimize the computational efficiency while ensuring data accuracy.

[0051] S3.2: Align the surface point cloud data with the 3D model coordinate system and execute the mixed reality collision detection algorithm to calculate the collision probability, which is expressed as: ; in, represents the collision probability, represents the Sigmoid function, Represents the real-time pose of real objects in physical space, Indicates the current position of NPCs and objects in the virtual scene. Indicates the scene base size, Indicates continuous tracking time. represents the decay time constant, represents the time decay function, Relative speed of motion, represents the current minimum Euclidean distance, Indicates the angle of motion direction.

[0052] The specific process includes accurately aligning the surface point cloud data obtained in the physical space with the three-dimensional model in the virtual scene to ensure that the coordinate systems of the two are consistent, thereby providing an accurate basis for subsequent collision detection. Then, the Sigmoid function is used to calculate the collision probability. The Sigmoid function comprehensively considers the real-time pose (i.e., position and posture) of the real object in the physical space and the current pose of the NPC and objects in the virtual scene, and combines the scene baseline size as a reference scale. In addition, the mixed reality collision detection algorithm continuously tracks the time and decay time constant, and adjusts the trend of the collision probability over time through the time decay function to reflect the dynamic changes in the possibility of collision in different time intervals. The relative motion speed further affects the calculation of the collision probability, which describes the approach rate between the physical object and the virtual character. Finally, the current minimum Euclidean distance is used to quantify the closest distance between the two, which directly affects the possibility of collision.

[0053] S3.3: Dynamically adjust the path planning of the virtual NPC according to the collision probability, and generate an instruction set including obstacle avoidance priority and movement speed.

[0054] Furthermore, when dynamically adjusting the path planning of the virtual NPC based on the collision probability, the potential conflict risk between the virtual NPC and the real object in the physical space is first evaluated based on the collision probability value calculated by the mixed reality collision detection algorithm. When the collision probability is high, a safe path away from high-risk areas is planned for the NPC first, and obstacles are avoided by recalculating the path points; when the collision probability is low, the NPC is allowed to move along the original path while retaining a certain degree of flexibility to deal with emergencies. This process combines the real-time posture information and geometric structure in the scene to ensure that the path adjustment is both efficient and meets the needs of the actual environment.

[0055] Specifically, when generating an instruction set that includes obstacle avoidance priority and movement speed, first set the obstacle avoidance priority according to the probability of collision. A high probability corresponds to a high priority, requiring the NPC to take avoidance measures as soon as possible, while a low probability allows for lower priority processing. Next, combined with the relative motion speed and the current minimum Euclidean distance, the NPC's movement speed is dynamically adjusted, slowing down appropriately when approaching an obstacle to reduce the risk of collision, and speeding up in a safe area to improve operating efficiency. Finally, the obstacle avoidance priority and the adjusted movement speed are integrated into a specific instruction set to guide the NPC's behavioral decisions, thereby achieving a more intelligent and smooth virtual-reality interaction experience.

[0056] S3.4: Convert the instruction set of obstacle avoidance priority and movement speed into the driving control signal of the physical device, and encapsulate it into a closed-loop interaction data set containing NPC behavior instructions and physical device control signals.

[0057] The specific process includes mapping the obstacle avoidance priority and speed parameters defined in the instruction set to specific physical device operation instructions. For example, high-priority obstacle avoidance instructions may be converted into emergency braking or steering control signals, while the adjustment of movement speed corresponds to the motor speed or power regulation of the actuator. Through preset conversion rules and device interface protocols, logical instructions are further encoded into a language or signal format that adapts to specific hardware, thereby directly driving physical devices to complete corresponding actions, ensuring that the behavior of virtual NPCs can be accurately executed in a real environment.

[0058] Furthermore, the converted drive control signal is integrated with the NPC behavior instructions to form a unified data structure. The data structure not only contains the behavior information such as the NPC's path planning and obstacle avoidance strategy, but also covers the actual control signals of the corresponding physical devices. The actual control signals are directly generated by analyzing the global federated model parameters and the real-time data of the edge nodes, combined with the preset behavior instructions, and are used to guide the physical devices to perform specific operations to achieve the synchronization of virtual and real interactions. Then, the data structure is marked with timestamps and scene identifiers to ensure the consistency and real-time transmission and parsing of the instruction set between different nodes, and finally generate a closed-loop interaction data set.

[0059] S4: Based on the closed-loop interaction dataset, the federated model training is performed on each edge node, the federated model gradient is calculated and uploaded to the federated master node, and the master node aggregates the gradient and updates the federated model.

[0060] S4.1: By parsing the closed-loop interaction dataset, extract NPC behavior instructions, device control signals, and posture verification data, and generate standardized training samples.

[0061] The specific process includes first classifying and extracting the NPC behavior instructions, device control signals, and posture verification data contained in the closed-loop interaction data set by parsing the closed-loop interaction data set. This process identifies and separates the behavior instructions related to NPC path planning and obstacle avoidance strategies, as well as the drive control signals corresponding to the physical device actions, based on data fields and protocol rules. At the same time, it extracts the real-time posture information of virtual characters and real objects in the scene. After preliminary screening, the real-time posture information is further cleaned to remove noise and redundant information to ensure that the extracted data has high accuracy and consistency, laying the foundation for subsequent training sample generation.

[0062] Furthermore, when generating standardized training samples, the extracted NPC behavior instructions, device control signals, and posture verification data are encoded and normalized in a unified format. By mapping data of different dimensions to the same feature space and combining timestamps and scene context information, a structured sample set is constructed. In addition, standardized training samples are annotated according to task requirements, such as marking collision risk levels or behavior execution effects, thereby forming standardized training samples with clear labels.

[0063] S4.2: Perform federated model training based on standardized training samples, and add dynamic differential noise to the trained federated model.

[0064] It should be noted that, first, the federated model is trained using standardized training samples, which contain preprocessed and annotated data sets to ensure that each participating edge node can learn on the same benchmark. During the training process, each edge node updates its local model based on local data and aggregates the updated results into the global federated model to gradually optimize the federated model.

[0065] Specifically, this process involves calculating the appropriate amount of noise based on specific privacy protection requirements and applying it to the federated model. In this way, potential privacy leakage risks can be effectively prevented without significantly affecting the performance of the federated model. Finally, after this federated model training based on standardized training samples and dynamic differential noise addition steps, effective training and privacy protection of the federated model are achieved, ensuring the accuracy and security of the federated model in practical applications.

[0066] S4.3: Perform homomorphic encryption on the federated model with added noise, generate an encrypted parameter package and upload it to the cloud aggregation node. The aggregation node performs weighted average of the parameters of each node and generates the global federated model parameters, which are expressed as: ; in, represents the global federated model parameters, represents the homomorphic decryption function, Represents the total number of edge nodes, represents the edge node index, represents the homomorphic encryption function, Indicates The global federation model parameters of edge nodes, represents the differential privacy sensitivity, represents the privacy budget, Indicates The federation weight of edge nodes, represents the standard Laplace noise generator, represents the homomorphic encryption public key, Represents a homomorphic encryption private key.

[0067] The specific process includes: in the federated learning process, the global federated model parameters with added noise are first homomorphically encrypted to generate an encrypted parameter package to ensure the security of data during transmission. After the encrypted parameter package is uploaded to the cloud aggregation node, the aggregation node uses the characteristics of homomorphic encryption to directly perform weighted averaging operations on the encrypted parameters without decryption.

[0068] Specifically, the parameters of each edge node are weighted according to the federation weight, which reflects the contribution of each edge node to the global federation model. At the same time, the differential privacy sensitivity and privacy budget are combined to ensure the effect of privacy protection. Finally, the aggregation result is restored to the global federation model parameters through the homomorphic decryption function to complete the update of the global federation model.

[0069] The global federated model parameters refer to the unified parameter values ​​generated by weighted averaging of the global federated models of each edge node in federated learning.

[0070] S4.4: Add Laplace noise to the gradient calculated by the edge node to generate the privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node. The expression is: ; in, Indicates The edge node is Noisy gradients during training epochs, Indicates the training round number, Indicates The edge node is The unadded noise of the wheel, represents the noise gradient, Indicates Dynamic gradient sensitivity of round training, represents the dynamic adjustment function, Indicates The federation weight of edge nodes, Indicates Real-time resource indicators of edge nodes, represents the resource weight coupling function value, Indicates based on training rounds Dynamic noise control function, Represents rounds over time Dynamically decaying privacy budget function, represents a skewed normal distribution, Represents positional parameters, represents the scale parameter, represents the skewness parameter.

[0071] The specific process includes adding Laplace noise to the gradient calculated by the edge node in each round of federated learning training to generate privacy-preserving gradients. This process determines the appropriate amount of noise through a dynamic adjustment function and a dynamic noise control function based on training rounds to ensure that the performance of the global federated model is not affected while protecting data privacy.

[0072] Furthermore, the original gradient of each edge node is weighted according to the federation weight, real-time resource indicators, and resource weight coupling function, and then an appropriate amount of Laplace noise is added to make it difficult for the data contribution of a single node to be inferred by reverse engineering. To further enhance security, the noisy gradient is encrypted using the Paillier encryption algorithm. The encrypted gradient is then uploaded to the cloud aggregation node, where it can be safely aggregated without leaking any private information. This method not only effectively protects the data privacy of all parties involved, but also promotes efficient federated model training under multi-party collaboration.

[0073] S4.5: Perform weighted summation based on the encrypted gradients of each edge node and update the global federated model parameters through natural gradient descent. The expression is: ; in, Indicates The global federated model parameters after round of updates, Indicates The global federated model parameters during round training, represents the step size coefficient, Represents the total number of edge nodes, represents a homomorphic decryption operation, Indicates The noise gradient of the edge nodes, Indicates The federation weight of edge nodes, represents the homomorphic encryption private key, represents element-wise multiplication, Represents the Fisher information matrix The low-rank approximate inverse matrix of represents the Riemann manifold regularization term.

[0074] The specific process includes weighted summation based on the encrypted gradients of each edge node in each round of training, and updating the global federated model parameters through natural gradient descent. First, the encrypted and noise-added gradients are obtained from each edge node, and the encrypted gradients are restored to a processable form using homomorphic decryption operations. Then, the encrypted gradients are weighted summed according to the federated weight of each node to ensure that nodes with greater contributions have higher influence during the update process.

[0075] Next, the natural gradient descent method is used, combined with the step size coefficient to adjust the update amplitude, and the low-rank approximate inverse matrix of the Fisher information matrix is ​​used to optimize the update direction. At the same time, the Riemann manifold regularization term is used to maintain the good geometric characteristics of the global federated model parameter space. In this way, not only the effective update of the global federated model parameters is achieved, but also the data privacy protection and efficient global federated model parameter optimization under multi-party collaboration are guaranteed. This process comprehensively considers the data contribution and real-time resource status of different nodes, ensuring that the global federated model parameters can continuously improve performance while protecting privacy.

[0076] S5: Use global federated model parameters to trigger dynamic navigation and risk warning, and synchronize the optimized virtual and real scene bidirectional mapping protocol to multiple scenic spot digital twins through the Metaverse protocol to build a cross-scene virtual cultural tourism platform.

[0077] S5.1: Through the global federated model parameters, the tourist location, scenic area capacity and environmental data are analyzed in real time, the guide path is dynamically generated, and the dynamic correlation parameters of the scenic area tourist distribution and capacity risk are calculated in real time. The expression is: ; in, Indicates The dynamic correlation parameters of the scenic spot tourist distribution and capacity risk of the round, represents the smooth positive activation function, represents the mathematical expectation, represents the probability density function, Represents the spatial coordinate vector within the scenic area, Indicated in The global federated model parameters updated after the second global aggregation, represents the scenic area capacity threshold matrix, represents the path gradient penalty coefficient, Indicates Wheel Guided Path The spatial gradient of Represents the element-wise multiplication algorithm of two vectors.

[0078] The specific process includes using the global federated model parameters updated by federated aggregation to evaluate the current distribution of tourists in the scenic area and its impact on the scenic area capacity. The mathematical expectation and probability density function are processed by smooth positive activation functions, and the tourist density and potential capacity risk of each area are accurately calculated in combination with the spatial coordinate vector of the scenic area. This process also takes into account the scenic area capacity threshold matrix and path gradient penalty coefficient to ensure that the generated guide path can guide tourists to avoid crowded areas and optimize the overall tour experience.

[0079] Furthermore, the path gradient penalty coefficient is used to adjust the spatial gradient of the guided path to avoid the guided path being too concentrated or dispersed, thereby effectively managing the distribution of people flow in the scenic area and reducing the risk of capacity overload. Finally, through comprehensive analysis and calculation, not only the goal of dynamically generating the optimal guided path is achieved, but also the distribution of tourists and capacity risks in the scenic area can be monitored and adjusted in real time.

[0080] S5.2 is based on the navigation path and dynamic association parameters encoded into bidirectional mapping parameters through the Metaverse protocol, and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus.

[0081] The specific process includes structurally integrating the generated guide path with dynamic association parameters such as the distribution of scenic spot tourists and capacity risks to ensure that the mapping relationship between virtual and real scenes is clear and consistent. The digital twin nodes of multiple scenic spots are formed by combining the actual needs of physical scenic spots, using blockchain technology to ensure data consistency and security, and using the Metaverse protocol to achieve standardized processing of complex information. They form a distributed network, allowing each scenic spot to not only optimize its own operations locally, but also collaborate efficiently with other scenic spots to jointly improve the overall service quality. Subsequently, the standardized encoding rules of the Metaverse protocol are used to convert the dynamic association parameters into a unified format that is suitable for the digital twins of multiple scenic spots, so as to achieve seamless docking and collaborative management between different scenic spots. On this basis, the encoded bidirectional mapping parameters are verified and synchronized through the blockchain consensus mechanism to ensure that the digital twin nodes of each scenic spot can safely and reliably obtain and update data, thereby supporting real-time interaction and resource optimization configuration across scenes, and laying the foundation for building a highly interconnected virtual cultural tourism platform.

[0082] S5.2: Based on the synchronized digital twin node parameters, a virtual cultural tourism platform with cross-scenario virtual-real interaction is constructed.

[0083] Furthermore, the synchronized parameters are used to update the digital twins of each scenic spot to ensure the high consistency and dynamic linkage between the virtual scene and the real environment. Subsequently, the digital twin node parameters of different scenic spots are associated and integrated to form a unified cross-scene data framework to support tourists' seamless switching and interaction between multiple scenic spots. On this basis, combined with the virtual-reality mapping protocol and interaction logic, real-time synchronization of virtual characters, guide paths, and physical equipment control signals is achieved to provide tourists with an immersive tour experience. In this way, not only the efficient coordination and optimal configuration of multi-scenic area resources are achieved, but also the coherence and sense of participation of tourists in cross-scene tours are improved, creating a highly intelligent and interactive virtual cultural tourism platform.

[0084] This embodiment also provides a computer device, which is suitable for the method of constructing a virtual cultural and tourism platform based on digital twins, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method of constructing a virtual cultural and tourism platform based on digital twins as proposed in the above embodiment.

[0085] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0086] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by the processor, the method for constructing a virtual cultural and tourism platform based on digital twins as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0087] In summary, the present invention achieves seamless connection between the virtual environment and the real world by: dynamically allocating precision weights of the federated model, triggering the target detection algorithm to perform semantic segmentation on key areas in combination with frame rate fluctuations of the real-time video stream, calculating geometric alignment error values ​​to divide precision levels, and dynamically switching the complexity of NPC behavior logic based on precision grading parameters, which not only improves the realism and interactivity of the virtual characters, but also reduces computing costs by optimizing resource usage.

[0088] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a virtual cultural tourism platform based on digital twins, characterized in that: include, Collect real-time data streams from the entire cultural and tourism industry, initialize the federated learning framework at the edge point, and generate a federated learning initialization instruction set; Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework. At the same time, the video twin engine is used to calibrate the real-time video stream with the 3D model to generate a bidirectional mapping protocol of virtual and real scenes containing accuracy grading parameters and NPC control instructions. Based on the bidirectional mapping protocol of virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection and the mixed reality collision monitoring algorithm is executed to generate a closed-loop interaction data set containing NPC behavior instructions and physical device control signals; Based on the closed-loop interactive data set, the federated model training is performed on each edge node, the federated model gradient is calculated and uploaded to the federated master node, and the master node aggregates the gradient and updates the global federated model parameters; The global federated model parameters are used to trigger dynamic navigation and risk warnings, and the optimized virtual-real scene bidirectional mapping protocol is synchronized to the digital twins of multiple scenic spots through the Metaverse protocol to build a cross-scene virtual cultural and tourism platform.

2. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 1, characterized in that: The real-time data stream of the entire cultural and tourism area includes thermal distribution data of tourist behavior, environmental parameters, cultural relics status data and real-time video stream data.

3. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 2, characterized in that: Initializing the federated learning framework at the edge point and generating a federated learning initialization instruction set, the specific steps are as follows: Initialize the federated learning framework on the edge node, deploy a lightweight federated learning client, load the real-time data stream of the entire cultural and tourism domain, and output a dimensionally aligned local training data set; Based on the local training data set, through multimodal learning, a federated learning initialization instruction set containing a federated model architecture template, aggregation weight rules, and communication protocol parameters is dynamically generated.

4. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 3, characterized in that: Based on the federated learning initialization instruction set, the federated model accuracy weight is dynamically allocated through the federated learning framework, and the real-time video stream is calibrated with the three-dimensional model using the video twin engine to generate a virtual-real scene bidirectional mapping protocol containing accuracy grading parameters and NPC control instructions. The specific steps are as follows: Based on the federated learning initialization instruction set, the performance parameters and data reliability indicators of edge nodes are extracted, and the accuracy weight of the federated model is dynamically allocated; When the video twin engine detects that the frame rate fluctuation of the real-time video stream exceeds the stability threshold, it triggers the target detection algorithm to perform semantic segmentation on the key areas and extract the bounding box coordinates of the moving objects; Based on the bounding box coordinates, the geometric alignment error value is calculated and the accuracy level is divided to generate the accuracy grading parameters including the grid density and algorithm selection rules; Based on the accuracy grading parameters, the geometric alignment error value of the current scene and the federated model accuracy weight are evaluated through scene consistency analysis to dynamically switch the NPC behavior engine containing logical complexity; The federated model precision weights, geometric alignment error values, and NPC behavior engine are encapsulated into structured data packets, synchronized to edge nodes and cloud master nodes through a unified spatiotemporal interface, and a bidirectional mapping protocol for virtual and real scenes containing precision grading parameters and NPC control instructions is generated.

5. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 4, characterized in that: The virtual-real scene bidirectional mapping protocol is based on scanning the three-dimensional geometric structure of the physical space through ray detection and executing the mixed reality collision monitoring algorithm to generate a closed-loop interaction data set containing NPC behavior instructions and physical device control signals. The specific steps are as follows: Based on the bidirectional mapping protocol of virtual and real scenes, the three-dimensional geometric structure of the physical space is scanned through ray detection, and the surface point cloud data of the physical space objects are extracted; Align the surface point cloud data with the 3D model coordinate system and execute the mixed reality collision detection algorithm to calculate the collision probability; According to the collision probability, the virtual NPC's path planning is dynamically adjusted, and an instruction set including obstacle avoidance priority and movement speed is generated; The instruction set of obstacle avoidance priority and moving speed is converted into driving control signals of physical devices and encapsulated into a closed-loop interaction data set containing NPC behavior instructions and physical device control signals.

6. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 5, characterized in that: Based on the closed-loop interaction dataset, the federated model training is performed on each edge node. The specific steps are as follows: By parsing the closed-loop interaction dataset, NPC behavior instructions, physical device control signals, and posture verification data are extracted, and standardized training samples are generated; Perform federated model training based on standardized training samples and add dynamic differential noise to the trained federated model; The federated model with added noise is homomorphically encrypted to generate an encrypted parameter package and upload it to the cloud aggregation node. The aggregation node performs weighted average of the parameters of each node and generates the global federated model parameters.

7. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 6, characterized in that: The federated model gradient is calculated and uploaded to the federated master node. The master node aggregates the gradient and updates the global federated model parameters. The specific steps are as follows: Add Laplace noise to the gradient calculated by the edge node to generate privacy-preserving gradient, and use Paillier to encrypt the gradient of each edge node; A weighted sum is performed based on the encrypted gradients of each edge node, and the global federated model parameters are updated through natural gradient descent.

8. The method for constructing a virtual cultural tourism platform based on digital twins according to claim 7, characterized in that: The global federated model parameters are used to trigger dynamic navigation and risk warning, and the optimized virtual-real scene bidirectional mapping protocol is synchronized to multiple scenic spot digital twins through the Metaverse protocol to build a cross-scene virtual cultural tourism platform. The specific steps are as follows: Through the global federated model parameters, the tourist location, scenic area capacity and environmental data are analyzed in real time, the guide path is dynamically generated, and the dynamic correlation parameters of the scenic area tourist distribution and capacity risk are calculated in real time; Based on the guided path and dynamic association parameters, they are encoded into bidirectional mapping parameters through the Metaverse protocol and synchronized to multiple scenic spot digital twin nodes based on blockchain consensus; Based on the synchronized digital twin nodes, a virtual cultural and tourism platform with cross-scene virtual and real interaction is built.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for constructing a virtual cultural and tourism platform based on digital twins as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a virtual cultural and tourism platform based on digital twins as described in any one of claims 1 to 8 are implemented.

Images