Element universe and digital twinborn technology-based relics restoration method

Abstract

The invention relates to a relics restoration method based on a meta-universe and a digital twinborn technology, and aims to solve the problems of relics disappearance and historical memory deficiency in urban construction, and utilizes the meta-universe and the digital twinborn technology to build a relics 3D model on a meta-universe platform to realize scene restoration. A virtual scene is constructed through software such as 3DMAX and Unity 3D and related technologies, and the characteristics of identity, social contact, immersion and the like are achieved; the method has obvious advantages, the cost can be reduced, the efficiency can be increased, the real scene can be stably simulated, and multi-place policy support can be obtained. The market potential is large, and the method faces college teachers and students, culture enthusiasts and other groups. Popularization strategies such as social media propaganda and SEO are adopted, and the product has unique advantages and innovation and is flexible in pricing; in addition, the method has both campus and social public welfare, provides an innovative scheme for relics recovery, and is of great significance in the fields of cultural inheritance, education and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality (VR), augmented reality (AR) and metaverse, in particular to a system and method for realizing visual restoration of ruins by using the above technologies. BACKGROUND

[0002] Under the background of continuous advancement of urban construction, reconstruction and rural urbanization, a large number of historical and cultural blocks, buildings and villages are gradually disappearing, and people's historical memories are also disappearing. While new buildings are developed and constructed in batches, the cultural ruins are also decreasing in batches.

[0003] Traditional physical restoration methods have many limitations, and the emergence of the metaverse concept brings new opportunities for 3D restoration of ancient ruins. Compared with traditional methods, digital twin technology in the metaverse has the characteristics of dynamic, real-time, two-way and full life cycle, which can more effectively realize the scene restoration of destroyed ruins. Although there are some companies on the market that provide VR / AR visual restoration of ruins, they generally have high hardware and software requirements, high costs, slow content update speed, small scale, lack of distinctiveness, and large deviation from history and traditional culture. In the domestic market, such enterprises are relatively rare, and cannot meet the market demand for high-quality restoration of ruins. SUMMARY

[0004] 1. Invention purpose: The present application aims to provide a Tianyan-VR / AR visual restoration system and method for ruins, which constructs a metaverse space close to reality through digital twin technology, realizes high-precision visual restoration of ruins, meets the needs of residents to understand historical ruins, and promotes the development of related public undertakings.

[0005] 2. Technical solution

[0006] System architecture: The system includes a data collection module, a model construction module, a virtual environment construction module and a system function implementation module. The data collection module is responsible for collecting various data related to ruins, such as building basic data, plant data, indoor furnishings data and audio and video materials, etc. The model construction module uses 3DMAX software to perform geometric modeling on buildings, and uses Billboard technology to establish plant models, and optimizes the models to produce LOD three-level models. The virtual environment construction module builds a virtual environment based on Unity3D, realizes interactive functions through Java and C# languages, and users can access it through a browser. The system function implementation module establishes a 3D model of ruins on a metaverse platform based on digital twin technology, realizes the scene restoration of destroyed ruins, and has multiple functions such as identity, socialization, immersion, low latency, diversity, ubiquity, economy and civilization.

[0007] Methodology: The metaverse-based method for relic restoration includes data collection, model building, virtual scene construction, and system function implementation. The data collection step gathers various types of data related to the relics. The model building step utilizes 3DMAX software and billboard materials to create and optimize the model. The virtual scene construction step uses Unity3D and related programming languages ​​to build a virtual environment and implement interactive functions. The system function implementation step builds a 3D model of the relics on the metaverse platform based on digital twin technology, realizing various system functions. Furthermore, it includes steps for conducting virtual relic construction training through online and offline teaching platforms, as well as conducting related commercial activities through cultural and creative e-commerce platforms.

[0008] Technical Applications: This invention utilizes a variety of technologies in the process of reconstructing historical sites, including cloud computing and network technology, human-computer interaction technology, artificial intelligence technology, blockchain and NFT technology, and Internet of Things technology. These technologies work together to provide the system with a high-speed, low-latency transmission channel, enhancing the user's immersive experience and interactive capabilities, achieving a hybrid virtual-real experience, protecting the system's economic structure and ownership of virtual items, and enabling the connection and symbiosis of everything within the system.

[0009] 3. Beneficial Effects: The Tianyan-VR / AR Visual Relics Restoration System and Method of this invention have significant advantages. Most similar software on the market is still in the conceptual stage, while this project has been largely completed and perfected. Recreating ancient Chinese sites through technological means is of great significance in promoting Chinese culture. At the same time, the system and method provided by this invention can bring users a more vivid and engaging experience of visiting historical sites, and has broad application prospects in cultural tourism, education and training, and cultural arts. Furthermore, this project also has campus and social public welfare aspects, helping to encourage universities to focus on related technology research and public welfare undertakings, and supporting existing relics protection work. Attached Figure Description

[0010] Figure 1 System architecture diagram

[0011] The diagram illustrates a layered architecture with subdivided modules. From top to bottom, the layers are: User Layer, Application Layer, Business Logic Layer, Data Processing Layer, Data Storage Layer, and Hardware Interaction Layer. The User Layer includes different user groups such as university faculty and students; the Application Layer includes VR / AR visualization applications for historical site restoration and online teaching platforms; the Business Logic Layer covers multiple modules such as historical site scene restoration logic; the Data Processing Layer includes functional modules such as data acquisition and preprocessing; and the Data Storage Layer uses a database (Oracle).

[0012] It consists of file storage; the hardware interaction layer is connected to VR / AR devices and server hardware (including 5G network).

[0013] Technical significance: This diagram clearly presents the overall system architecture, clarifies the functional positioning and interrelationships of each layer and module, provides a macro framework for understanding the system's workflow and implementation principles, helps technical personnel in system development, maintenance and optimization, and also facilitates relevant personnel in understanding how the various parts of the system work together to achieve functions such as heritage restoration, teaching, and cultural and creative product transactions.

[0014] Figure 2 Data Acquisition Flowchart

[0015] The diagram illustrates (hypothetically) the data collection sequence using process nodes and arrows. Starting with the data collection initiation, it branches into multiple sub-branches, including building data collection and plant data collection. Each sub-branch is further divided into specific steps such as field measurement, photography, and literature review. After collection, the data is aggregated in the data preprocessing stage and finally enters the storage stage.

[0016] Technical significance: (Hypothetical) This diagram illustrates the data acquisition process in detail, clearly defining the steps for obtaining data from different sources, as well as the subsequent processing and storage paths. This ensures the accuracy, completeness, and standardization of data acquisition, providing a reliable data foundation for the system and is a key step in building a high-quality heritage restoration model.

[0017] Figure 3 Model building flowchart (assuming it exists)

[0018] The illustration shows: (hypothetically) starting from the input data collection, entering the building model construction stage, demonstrating the geometric modeling, model optimization and other steps in 3DMAX software; next is the construction of the plant model, demonstrating the application of billboard material technology; then comes model integration and testing, through which the decision is made whether to return for modification, and finally the complete model is output.

[0019] Technical significance: (Hypothetical) This diagram illustrates the detailed process of model building, highlighting key technologies and iterative optimization processes, providing an operational guide for technicians to build high-precision heritage models, and ensuring model quality and system performance.

[0020] Figure 4 Virtual environment interaction intent

[0021] The diagram illustrates the interaction process between a user and a virtual historical site using a VR / AR device. The user interacts with the site through head movements, hand gestures, and voice commands. The head-mounted device's gyroscope, controller sensors, and microphone capture this information and transmit it to the interactive system. After analyzing the data, the virtual scene responds by adjusting the viewing angle, executing interactive actions, and performing specific functions. Finally, the head-mounted device's display, controllers or other devices, and speakers provide visual, tactile, and auditory feedback to the user, respectively.

[0022] Technical significance: Multimodal interaction integrates various feedback methods, allowing users to deeply immerse themselves in virtual heritage scenes and enhance their understanding and experience of the heritage; it accurately captures and analyzes user operations, enabling real-time responses from the virtual scene and enhancing the realism of the interaction; it provides a technological foundation for the application of VR / AR in multiple fields, promoting innovation in industries such as cultural heritage protection, education, and tourism; and it drives the upgrading of VR / AR device hardware and the optimization of interaction algorithms, thus promoting the development of virtual reality technology.

[0023] Figure 5 System functional architecture diagram

[0024] The illustration is presented in a radial pattern. The center represents the user, surrounded by functional modules such as Identity, Social, Immersive Experience, Low Latency, Diversity, Usability, Economy, and Civilization. Each module is represented by a circular icon, with simple graphics or text inside the icon illustrating its characteristics; for example, the Social module uses a speech bubble, and the Economy module uses a coin icon. The modules are connected by lines to represent their interrelationships and collaborative relationships.

[0025] Technical Significance: This diagram clearly illustrates the various functions of the system and their logical relationships. It provides the development team with an intuitive perspective to understand the system's functional architecture, facilitating accurate understanding of the positioning and role of each functional module and their collaborative mechanisms during system development and upgrades. This ensures the integrity and consistency of the system's functions, thereby providing users with a comprehensive and high-quality service experience. Detailed Implementation

[0026] Data collection

[0027] 1. Building Data Acquisition: High-precision laser rangefinders and total stations were used to conduct on-site measurements of the length, width, and height of the ruins, as well as the relative positions and angles between various structures. Every data point was precisely recorded to ensure errors were kept within a minimal range. Professional perspective-free photography equipment was used to photograph the building's exterior from multiple angles, including its overall appearance, detailed decorations, and door and window styles, ensuring that the photographs clearly presented every feature of the building. Regarding the internal structure, on-site surveys were conducted, ensuring safety, to record information such as the internal spatial layout and beam-column structure. Simultaneously, historical documents and archives were consulted to obtain background information such as the building's original design drawings, construction date, and repair records, providing comprehensive data support for subsequent model construction.

[0028] 2. Plant Data Collection: Around the ruins, professional cameras will be used to photograph various plants, including trees and flowers, from different angles and distances. Sufficient lighting will be ensured during photography to clearly display the plant's shape, texture, and color. Information such as plant species, location, height, and crown width will be recorded. For rare plants or those with special historical and cultural significance, detailed records of their characteristics and background information will be documented. Furthermore, satellite remote sensing imagery and Geographic Information System (GIS) data can be used to obtain information on plant distribution and growing environments, providing a more accurate basis for plant placement in the virtual scene.

[0029] 3. Data Collection on Interior Furnishings: If the site contains interior spaces, conduct an on-site investigation and record the types, quantities, locations, and styles of furniture, decorations, and artifacts. For important items, take detailed measurements and photographs, and if necessary, use a 3D laser scanner to obtain accurate 3D models. Consult relevant historical materials and archaeological reports to understand the historical background, cultural connotations, and functions of the interior furnishings, so as to accurately recreate them in the virtual environment.

[0030] 4. Audio and Video Material Acquisition: At the site, professional audio recording equipment will be used to record environmental sound effects, such as wind, rain, birdsong, and any unique sounds that may exist around the site, such as the chimes of ancient bells and the sound of flowing water, to add realism to the virtual scene. Video footage of the site will be captured, including panoramic views, close-ups, and changes in light and shadow at different times of day, showcasing the site's appearance from multiple angles. Simultaneously, historical audio and video materials related to the site will be collected, such as historical documentaries and expert commentary audio, to enrich the content of the virtual scene.

[0031] 5. Data Preprocessing: All collected data is transferred to a computer. Data processing software, such as Adobe Photoshop and Autodesk ReCap, is used to denoise, crop, and adjust colors in the image data to enhance clarity and contrast, making it more suitable for model construction. For measurement data, error analysis and correction are performed to ensure accuracy. Data in different formats is converted and organized into formats easy to use later, such as converting image data to JPEG and PNG formats, and organizing measurement data into CSV and XML formats. A data index is then created for easy and quick retrieval.

[0032] Model building

[0033] 1. Building Model Construction: Open 3ds Max software and, based on the collected building measurement data and photographs, use geometric modeling methods to gradually build the building model, starting from the basic structure. First, create the main framework of the building, determining the position and dimensions of walls, beams, and columns, then add details such as doors, windows, and decorations. During the modeling process, use techniques such as polygon modeling and spline modeling appropriately to improve the accuracy and quality of the model. For example, for complex architectural decorations, such as hollowed-out designs and reliefs, refine their shapes by editing the vertices, edges, and faces of polygons; for regular building structures, use splines to draw the outline, and then generate the model through extrusion and lofting operations. To address the issue of large amounts of building model data, optimization is performed. Simplify the model structure, removing details that have little impact on the overall effect, such as small protrusions and depressions. Appropriately set the model's LOD (Level of Detail): use a low-detail model when viewing from a distance to reduce rendering computation; switch to a high-detail model when viewing up close to ensure the model's realism. UV unwrapping of the model breaks down the surface of the 3D model into a 2D plane, which is then used for texture mapping to make the model more realistic.

[0034] 2. Plant Model Construction: Plant models are constructed using a multi-image-based billboard material technique. In 3ds Max, billboard materials are created based on photographs of the plants, mapping the plant images onto a plane and setting parameters such as transparency and lighting effects to create a three-dimensional effect. The size, angle, and position of the billboard are adjusted to simulate the plant's growth posture. For tall trees, multiple billboard materials can be combined to increase the model's depth and realism. Collision detection is also added to the plant models, allowing characters in the virtual scene to interact naturally with the plants, such as colliding with them while walking or the plants swaying in the wind.

[0035] 3. Model Optimization and Integration: After constructing the building and plant models, perform overall model optimization. Check the model's topology to ensure reasonable wiring and avoid issues such as overlapping surfaces and non-manifold geometry, improving rendering efficiency and stability. Optimize the model's materials and textures, compressing texture image sizes to reduce memory usage while maintaining texture clarity and quality. Integrate the building, plant, and interior furnishing models into a single scene, adjusting their positions, proportions, and lighting relationships to create a more harmonious and realistic scene. Perform test rendering on the integrated scene to check for missing models, texture errors, abnormal lighting, and other issues, and promptly repair and adjust them.

[0036] Virtual scene building

[0037] 1. Unity3D Platform Setup: Open Unity3D software, create a new project, and set the project's basic parameters, such as scene size, rendering quality, and target platform. Export the completed model from 3ds Max as an FBX file and import it into the Assets folder of the Unity3D project. In the Unity3D Hierarchy panel, organize and manage the imported model, creating corresponding folders and hierarchical structures for easy searching and editing. Based on the actual layout of the ruins, place the model in the Scene panel, adjusting its position, rotation, and scale to construct a complete virtual scene.

[0038] 2. Interactive Functionality Implementation: Interactive scripts will be written using Java and C# to implement user interaction with the virtual environment. For example, scripts will be written to implement user actions such as walking, running, and jumping, by acquiring the user's input device (e.g., keyboard, mouse, gamepad).

[0039] Information controls the movement of virtual characters within the scene. It enables user interaction with virtual objects, such as clicking, picking up, and placing items. Using raycasting technology, it detects the user's click location, determines whether interaction has occurred, and executes the corresponding action. Camera control functionality is added, allowing users to freely switch between first-person and third-person perspectives, as well as different viewing angles, enhancing user immersion and the overall user experience.

[0040] 3. Scene Optimization and Performance Enhancement: The virtual scene is optimized to improve its performance. Lighting effects are optimized by using baked lighting technology to pre-calculate the light distribution in the scene, reducing real-time computation overhead and making the lighting effects more natural. The scene's occlusion culling function is optimized by setting the relationship between occluders and occluded objects to avoid rendering occluded objects, thus improving rendering efficiency. The particle system and special effects in the scene are optimized to reduce the number of particles and the complexity of effects, preventing stuttering. Scene testing is conducted on different hardware devices to collect performance data. Based on the test results, scene parameters and settings are further adjusted to ensure smooth operation on various devices.

[0041] 4. Cloud Computing and Network Deployment: Virtual scenes are stored on cloud servers, leveraging the powerful computing capabilities and storage resources provided by cloud computing technology to achieve efficient scene loading and operation. A suitable cloud service provider, such as Alibaba Cloud or Tencent Cloud, is selected, and appropriate cloud server resources are configured according to the project's needs and scale. User requests are transmitted to the cloud server via network technology. The server processes the requests in real time and transmits the rendered images back to the user's device. The high-speed transmission capabilities of 5G networks reduce latency and improve data transmission speed, ensuring a smooth and real-time virtual experience for users. Simultaneously, CDN (Content Delivery Network) technology is used to cache the static resources of the virtual scene on the node closest to the user, further improving loading speed.

[0042] System Function Implementation

[0043] 1. Identity and social functions: Utilize blockchain technology to build a user identity system and generate a unique digital identity for each user.

[0044] During user registration, a public-private key pair is generated using an encryption algorithm. The public key serves as the user's digital identity, while the private key is securely kept by the user. Within the system, the user's digital identity is bound to their real-world identity to a certain extent, ensuring authenticity and security. Social functions are implemented, building a social platform where users can add friends, create groups, and send messages. In virtual scenarios, users can see other users' digital avatars and interact in real time, such as visiting historical sites together or exchanging experiences. These social functions promote communication and cooperation among users, enhancing their sense of participation and belonging.

[0045] 2. Immersive Experience and Low Latency: Advanced display technologies, such as high-resolution OLED displays and wide-angle optical lenses, enhance the user's visual experience, making virtual scenes clearer and more realistic. Combined with spatial audio technology, it provides users with 3D surround sound, allowing them to accurately determine the direction and distance of sound sources, enhancing immersion. Sensor technologies, such as gyroscopes and accelerometers, track the user's head and body movements in real time, enabling real-time switching of perspectives and precise feedback of actions, making user operations more natural and fluid. Optimized network transmission protocols and server architecture reduce system latency. Real-time communication technologies, such as WebRTC, enable real-time data transmission and synchronization, ensuring timely feedback on user actions in the virtual scene and maintaining synchronization with interactions with other users.

[0046] 3. Diversified and Ubiquitous Access: An open creation platform is built, supporting users from all walks of life to create content within the system. A wealth of creation tools and resources are provided, such as model editors, material libraries, and scripting languages, facilitating users' design of virtual scenes, creation of items, and story writing. A user incentive mechanism is established to reward outstanding creative content, encouraging active user participation and enriching the system's content ecosystem. To achieve ubiquitous access, multi-platform compatible applications are developed, including PC, mobile, and VR / AR device versions. Optimization and adaptation are performed for the characteristics and performance of different platforms, ensuring smooth system operation on various devices. Cloud computing technology enables cross-platform synchronization of user data, allowing users to access their personalized settings, creative content, and game progress information when logging into the system on different devices.

[0047] 4. Economic System and Realization of Digital Civilization: Construct an economic system linked to the real-world economic system, introducing virtual currencies and digital assets. Users can earn virtual currencies by completing tasks, creating content, and participating in activities, and use these virtual currencies to purchase virtual items, services, or digital assets. Digital assets are identified and traded using NFT (Non-Fungible Token) technology, ensuring their uniqueness and ownership. Through the economic system, user participation in its construction and development is incentivized, while also providing economic rewards for creators. As users engage in activities and interactions within the system, a unique digital civilization gradually forms. Establish rules and values ​​for this digital civilization, such as respecting intellectual property rights, protecting cultural heritage, and promoting cooperation and exchange. Disseminate digital civilization through cultural activities and competitions, enhancing users' cultural identity and sense of belonging, and promoting the development and inheritance of digital civilization.

[0048] From a technical perspective, this invention achieves highly realistic restoration of historical sites, with its constructed virtual scenes offering an extremely lifelike sensory experience in terms of sight and sound. Through optimized algorithms and model building techniques, the system runs smoothly, effectively avoiding lag and maintaining stable presentation even in complex scenes. In terms of interaction, the integration of multiple interaction methods makes user operation natural and convenient, such as gesture recognition to control virtual objects and voice interaction to obtain information, greatly enhancing user engagement. Moreover, the system possesses excellent scalability, easily integrating new historical site data and developing new functions to adapt to ever-changing market demands, setting a new benchmark in the field of historical site restoration and powerfully promoting the integration of cultural heritage protection and technology.

Claims

1. A method for relic restoration based on metaverse and digital twin technology, characterized in that, include: o Data acquisition module: used to collect basic building data, plant data, interior furnishing data, and audio-visual materials, including on-site measurement data, non-perspective photography data, etc. o Model building module: Using 3DMAX software, the building model is designed using geometric modeling methods, and the plant model is created using billboard material technology. The model is then processed with geometric optimization, UV unwrapping, texture mapping, etc., to create a Level 3 LOD model. o Virtual scene building module: Based on Unity3D, build virtual environments, design scene interactions, and implement interactive functions through Java and C# languages, so that users can access the virtual world through a browser; o System Function Modules: Features include identity setting, social interaction, immersive experience, low-latency synchronization, diversified content creation, ubiquitous access, integration with the economic system, and digital civilization derivatives.

2. The relic restoration method based on metaverse and digital twin technology according to claim 1, characterized in that, The data acquisition module is used for building data acquisition, covering the acquisition of data on building frames, decorative layers, hollows, reliefs, components, etc.

3. The relic restoration system based on metaverse and digital twin technology according to claim 1, characterized in that, In the model building module, when optimizing the building model, factors such as the number of components, material properties, and texture mapping are comprehensively considered to optimize the system while ensuring normal operation and virtual experience.

4. The relic restoration method based on metaverse and digital twin technology according to claim 1, characterized in that, In the virtual scene building module, scene interaction design is implemented using Unity3D. Interactive functions include, but are not limited to, users browsing, operating, and interacting with objects in the virtual scene.

5. The relic restoration method based on metaverse and digital twin technology according to claim 1, characterized in that, In the system function implementation module: The identity setting function provides each person with one or more unique digital identities in the system; o Social features enable users to communicate seamlessly with other people's digital identities, achieving a social experience equivalent to reality; o Immersive experience features use technology to give users a highly realistic immersive experience through their senses; The low-latency synchronization function ensures that everything happening in the system is synchronized with the timeline, basically achieving synchronous refresh. o Diversified content creation features allow users from all walks of life to create content, with UGC dominating the virtual space; The ubiquitous access feature allows users to connect to the system anytime, anywhere. The economic system linkage function enables the system to have its own economic system that is linked to existing real-world economic systems; The derivative functions of digital civilization have enabled humanity to form branches of digital civilization through the accumulation of time and consensus within the system.

6. A method for relic restoration using the relic restoration system based on metaverse and digital twin technology as described in any one of claims 1-5, characterized in that, Includes the following steps: o Data collection steps: Collect various types of heritage-related data according to the requirements of the data acquisition module; Model creation steps: Use the model building module method to create a 3D model of the ruins; o Virtual scene building steps: Build a virtual scene and implement interactive functions using the virtual scene building module; o System function implementation steps: Enable and improve the various functions in the system function implementation module to provide users with a complete heritage restoration experience.

7. The method for restoring relics according to claim 6, characterized in that, The data collection step also includes organizing, classifying, and preprocessing the collected data to meet the requirements of model building.

8. The method for restoring relics according to claim 6, characterized in that, In the model building process, when building the plant model, a large-scale plant reconstruction technique based on multiple images, namely the billboard material technique, is used to reduce the amount of model data.

9. The method for restoring relics according to claim 6, characterized in that, In the virtual scene construction process, cloud computing and network technologies are used to ensure that users have a real-time and smooth experience.

10. The method for restoring relics according to claim 6, characterized in that, In the process of implementing system functions, blockchain and NFT technologies are used to support the system's economic structure and realize functions such as financial transactions and digital copyright confirmation.

Images