Methods for Constructing the Metaverse of Immovable Cultural Relics

Abstract

This invention provides a method for constructing a metaverse for immovable cultural relics. The method includes: S101: acquiring an overall 3D model of the immovable cultural relic through oblique photogrammetry, and constructing a main 3D model and accessory 3D models of the immovable cultural relic based on point cloud or BIM data; S102: acquiring attribute data of the immovable cultural relic, overlaying the main 3D model with the overall 3D model, trimming the overall 3D model based on the overlap information, and moving the accessory 3D models to positions corresponding to the main 3D model to form a metaverse scene; S103: attaching the attribute data to the metaverse scene and rendering the metaverse scene. This invention expands cultural dissemination channels, facilitating more convenient and intuitive daily management of immovable cultural relics by administrators, while allowing for close-up viewing and appreciation of these relics, providing a better experience and promoting the dissemination and promotion of immovable cultural relic culture.

Description

Technical Field

[0001] This invention relates to the field of metaverse technology, and in particular to a method for constructing a metaverse of immovable cultural relics. Background Technology

[0002] Currently, according to statistics, there are approximately 350,000 registered cultural relics sites in China, and 12 million cultural relics are collected in various museums. Every year, a large number of cultural relics are unearthed. However, due to many limitations such as time, space, preservation conditions, and protection technology, a large number of cultural relics are immovable, such as ancient buildings (historical buildings), traditional settlements, ancient streets, and archaeological sites.

[0003] Furthermore, the practice of opening these immovable cultural relics to the public for display makes their daily management difficult and inconvenient. It also hinders close-up viewing and appreciation, resulting in a poor experiential aspect and impeding the dissemination and promotion of the culture associated with these relics. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes a method for constructing a metaverse of immovable cultural relics. This method involves acquiring relevant data about the immovable cultural relics, using this data to construct an overall 3D model, a main 3D model, and 3D models of accessories. A metaverse scene is created by combining these 3D models, and the attribute data of the immovable cultural relics is attached to this scene. This allows for the intuitive display of the immovable cultural relics within the metaverse, expanding cultural dissemination channels and facilitating more convenient and intuitive daily management of these relics by administrators. Simultaneously, it enhances the public's online and offline immersive interactive experience, allowing for close-up viewing and appreciation of these relics, providing a superior experience and promoting the dissemination and promotion of immovable cultural relics culture.

[0005] To address the aforementioned problems, the present invention employs the following technical solution: a method for constructing a metaverse of immovable cultural relics, comprising: S101: acquiring an overall three-dimensional model of the immovable cultural relic through oblique photogrammetry, and constructing a main three-dimensional model and ancillary three-dimensional models of the immovable cultural relic based on point cloud or BIM data; S102: acquiring attribute data of the immovable cultural relic, overlapping the main three-dimensional model with the overall three-dimensional model, trimming the overall three-dimensional model based on the overlap information, and moving the ancillary three-dimensional models to positions corresponding to the main three-dimensional model to form a metaverse scene; S103: attaching the attribute data to the metaverse scene and rendering the metaverse scene.

[0006] Furthermore, after the step of rendering the metaverse scene, the method further includes: receiving login information, and after confirming successful login through the login information, generating a virtual data persona for the user, and displaying the metaverse interactive scene through the virtual data persona.

[0007] Furthermore, the step of generating a virtual digital person for the user specifically includes: presetting various types of virtual digital persons, obtaining the user's facial information, selecting a matching virtual digital person based on the recognition result of the facial information, and modifying the virtual digital person based on the facial information; or obtaining the user's appearance information and generating the virtual digital person through three-dimensional reconstruction based on the appearance information.

[0008] Furthermore, the step of demonstrating metaverse interaction through virtual digital humans includes: obtaining the experience mode selected by the user, displaying experience content according to the experience mode, and executing the interaction between virtual digital humans according to the input instructions.

[0009] Furthermore, the step of displaying metaverse interaction through virtual data humans also includes: acquiring metaverse interaction information and experience feedback information, and iteratively optimizing the metaverse of immovable cultural relics based on the metaverse interaction information and experience feedback information.

[0010] Furthermore, the step of obtaining the overall three-dimensional model of the immovable cultural relic through oblique photogrammetry specifically includes: obtaining oblique photogrammetry data and near-ground data of the immovable cultural relic, and constructing the overall three-dimensional model of the immovable cultural relic based on the oblique photogrammetry data and near-ground data.

[0011] Furthermore, the step of obtaining the attribute data of immovable cultural relics further includes: converting the attribute data into a format corresponding to the type of the attribute data.

[0012] Furthermore, the step of overlapping the main three-dimensional model with the overall three-dimensional model specifically includes: loading the main three-dimensional model, the overall three-dimensional model and the appendage three-dimensional model, and overlapping the main three-dimensional model and the overall three-dimensional model with corresponding points in the main three-dimensional model and the overall three-dimensional model.

[0013] Furthermore, the step of moving the three-dimensional model of the appendage to a position corresponding to the three-dimensional model of the main body specifically includes: obtaining the spatial relationship between the appendage and the main body, and moving the three-dimensional model of the appendage according to the spatial relationship.

[0014] Furthermore, the step of attaching attribute data to the metaverse scene specifically includes: obtaining the identification code of each 3D model in the metaverse scene, and binding the 3D model with the attribute data according to the identification code.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: It acquires relevant data on immovable cultural relics, uses this data to construct an overall 3D model, a main 3D model, and 3D models of accessories for the immovable cultural relics, creates a metaverse scene through the combination of 3D models, and attaches the attribute data of the immovable cultural relics to the metaverse scene. This allows for the intuitive display of immovable cultural relics in the metaverse, expanding cultural dissemination channels and facilitating more convenient and intuitive daily management of immovable cultural relics by administrators. Simultaneously, it enhances the public's online and offline immersive interactive experience, allowing for close-up viewing and appreciation of these cultural relics, providing a superior experience and thus promoting the dissemination and promotion of immovable cultural relic culture. Attached Figure Description

[0016] Figure 1 This is a flowchart of an embodiment of the method for constructing the metaverse of immovable cultural relics according to the present invention;

[0017] Figure 2 This is a flowchart of another embodiment of the method for constructing the metaverse of immovable cultural relics according to the present invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that the various embodiments of this disclosure described and shown in the accompanying drawings can be combined with each other without conflict, and the structural components or functional modules can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0019] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular terms “a,” “said,” and “the” as used in this disclosure and the appended claims are also intended to include the plural terms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0020] Please see Figure 1-2 ,in, Figure 1 This is a flowchart of an embodiment of the method for constructing the metaverse of immovable cultural relics according to the present invention; Figure 2 This is a flowchart illustrating another embodiment of the method for constructing the metaverse of immovable cultural relics according to the present invention. (In conjunction with...) Figure 1-2 The present invention describes the method for constructing the metaverse of immovable cultural relics.

[0021] In this embodiment, the device executing the method for constructing the metaverse of immovable cultural relics can be a laptop computer, desktop computer, cloud platform, or other smart terminal capable of constructing 3D models and metaverses. The method for constructing the metaverse of immovable cultural relics executed by the smart terminal includes:

[0022] S101: Obtain the overall three-dimensional model of the immovable cultural relic through oblique photogrammetry, and construct the main three-dimensional model and the three-dimensional model of the appendages of the immovable cultural relic based on point cloud or BIM data.

[0023] In this embodiment, the steps of obtaining the overall three-dimensional model of an immovable cultural relic through oblique photogrammetry specifically include: acquiring oblique photogrammetry data and near-ground data of the immovable cultural relic; and constructing the overall three-dimensional model of the immovable cultural relic based on the oblique photogrammetry data and near-ground data. By creating the overall three-dimensional model, the entire landscape of the area where the immovable cultural relic is located can be represented, facilitating a more intuitive and comprehensive understanding of the environment surrounding the immovable cultural relic by the public.

[0024] The oblique photography data includes oblique photography data obtained by aerial photography through drones, helicopters or other devices, as well as near-ground data. The near-ground data is the photographic data of the immovable cultural relics near the ground. The accuracy of the overall three-dimensional model is increased by adding near-ground data.

[0025] In one specific embodiment, the smart terminal runs modeling software such as Smart3D to acquire oblique photogrammetry data and near-ground data. Then, it performs aerial triangulation on the oblique photogrammetry data and near-ground data using the modeling software. The aerial triangulation results of the two are matched by connecting corresponding points. Based on the matched data, an overall 3D model is built to eliminate the distortion of the 3D model and improve the model browsing experience.

[0026] In this embodiment, point cloud data of the immovable cultural relic and its appendages are acquired using 3D scanning technologies such as laser scanning. The smart terminal can process the data using modeling software such as 3ds Max, Blender, and Cinema 4D to obtain 3D models of the main body and appendages, or it can generate these models using BIM technology. The main body 3D model is the 3D framework structure model of the immovable cultural relic itself, while the appendage 3D models are small models of items such as tables, chairs, beds, plaques, lamps, trash cans, and others within the immovable cultural relic. Subsequent metaverse interaction is achieved through these main body and appendage 3D models.

[0027] S102: Obtain the attribute data of immovable cultural relics, overlay the main 3D model with the overall 3D model, trim the overall 3D model according to the overlap information, and move the accessory 3D model to the position corresponding to the main 3D model to form a metaverse scene.

[0028] In this embodiment, the attribute data of immovable cultural relics includes text related to the immovable cultural relics (such as a brief introduction of the immovable cultural relics), images (photos of the cultural relics), audio and video, documents, and other data that can disseminate relevant knowledge and culture about the immovable cultural relics. If the data is recorded in a paper document, it can also be digitized through methods such as keyboard input and scanning to be collected into a smart terminal.

[0029] The steps following the acquisition of attribute data for immovable cultural relics also include: converting the attribute data into a format corresponding to its type.

[0030] In one specific embodiment, after collecting attribute data, the text, images, audio, and video in the attribute data are converted into doc, png, mp3, and mp4 formats, respectively.

[0031] In this embodiment, the step of overlapping the main three-dimensional model with the overall three-dimensional model specifically includes: loading the main three-dimensional model, the overall three-dimensional model and the appendage three-dimensional model, and overlapping the main three-dimensional model and the overall three-dimensional model with corresponding points in the main three-dimensional model and the overall three-dimensional model.

[0032] Before constructing the metaverse by loading the 3D model through the preset game engine, the overall 3D model, the main 3D model, and the accessory 3D models are converted into a data format supported by the game engine. This data format can be a common format such as fbx or obj.

[0033] In a specific embodiment, the steps of overlapping the main 3D model and the overall 3D model include: 1) loading the overall 3D model and the main 3D model into the specified game engine simultaneously; 2) obtaining the bounding box of the main 3D model, and generating a BOX of the same size as the bounding box, setting the material of the BOX to transparent (so that the BOX is not visible), and placing it in the same group as the main 3D model (to facilitate the subsequent movement of the main body and the BOX together); 3) setting the material of the overall 3D model to a mask type (to facilitate subsequent clipping), and binding the position and size of the BOX as parameters to the material of the overall 3D model in real time, and adjusting the position of the main body and the BOX by moving them so that their corresponding points coincide.

[0034] In this embodiment, the overlapping part between the overall 3D model and the main body is obtained through overlap information, and this part is cropped and removed from the overall 3D model.

[0035] In this embodiment, the step of moving the 3D model of the appendage to a position corresponding to the 3D model of the main body specifically includes: obtaining the spatial relationship between the appendage and the main body, and moving the 3D model of the appendage according to the spatial relationship. Specifically, the spatial relationship between the appendage and the main body can be determined by on-site photographs of the immovable cultural relic, and then the 3D model of the appendage can be moved according to the spatial relationship so that the relative position of the 3D model of the appendage and the 3D model of the main body conforms to the spatial relationship.

[0036] In other embodiments, the appendage 3D model can be moved based on the spatial relationship information between the input main 3D model and the appendage 3D model. It can also be determined whether the overall 3D model includes the appendage model; if so, the appendage 3D model is moved to coincide with the corresponding appendage in the overall 3D model.

[0037] S103: Attach the attribute data to the metaverse scene and render the metaverse scene.

[0038] In this embodiment, the step of attaching attribute data to the metaverse scene specifically includes: obtaining the identification code of each 3D model in the metaverse scene, and binding the 3D model with the attribute data according to the identification code.

[0039] In the 3D model, each component of the immovable cultural relic and its appendages has a unique identifier. Attribute data is bound together by linking this identifier to the corresponding attribute data. Once bound, the attribute data linked to the identifier of the 3D model is displayed based on the user's selection or manipulation of the model.

[0040] In one embodiment, to simplify operation and avoid lag and poor smoothness, after constructing the metaverse scene, a cloud rendering platform is used to render the metaverse scene's package to generate the metaverse.

[0041] In this embodiment, after the step of rendering the metaverse scene, the method further includes: receiving login information, and after confirming successful login through the login information, generating a virtual data persona for the user, and displaying the metaverse interactive scene through the virtual data persona.

[0042] In this embodiment, the login information can be the user's fingerprint, account password, palm print, and other information that can be used for registration and user identification.

[0043] The steps for generating a virtual digital person for a user specifically include: pre-setting various types of virtual digital persons, obtaining the user's facial information, selecting a matching virtual digital person based on the facial information recognition results, and modifying the virtual digital person based on the facial information; or obtaining the user's appearance information and generating a virtual digital person through three-dimensional reconstruction based on the appearance information.

[0044] In one specific embodiment, virtual digital humans of different genders and age groups are constructed using modeling software, and their information is stored. Facial information of the user is acquired through laser scanning technology, and the user's gender, age, and other information are determined based on facial AI recognition technology. This information is then used to match a virtual digital human that is similar to the user, and the facial information acquired through laser scanning technology is used as the facial feature model of the virtual digital human to make it more realistic (the collected facial point cloud data is used to model a detailed facial model, and the facial model is then replaced with the pre-built virtual human facial model).

[0045] In another specific embodiment, a user's character model is constructed using laser scanning technology and 3D reconstruction technology. Since the virtual digital human created in this way is a real-time 3D reconstruction based on the user's real appearance (collecting the user's appearance and performing 3D reconstruction based on that appearance, the same method as the overall 3D model is used, but only point cloud data is used for 3D reconstruction), no additional matching is required. The reconstruction time is slow, but it is more realistic. At the same time, after the visit and experience is completed, the digital assets of the user's own virtual digital human (the digital assets are the 3D reconstruction results obtained above, the 3D model) and its 3D printed physical model can be used as souvenirs.

[0046] The steps for demonstrating metaverse interaction through virtual digital humans include: obtaining the experience mode selected by the user, displaying experience content according to the experience mode, and executing interactions between virtual digital humans according to the input instructions.

[0047] In this embodiment, the experience modes include a free experience mode and a game experience mode. The free experience mode utilizes XR technology for interactive experiences. Users can choose between first-person or third-person perspectives for free roaming and interaction. During roaming, users can click to query the main body of the immovable cultural relic and its accessories. The retrieved information is displayed in pop-up windows, which can contain text, images, videos, etc. It can also simulate weather systems such as rain, snow, sunshine, and thunderstorms. The materials of the immovable cultural relic switch according to different weather conditions to achieve a realistic effect. Real-life scenes can be simulated using sound, light, and electricity technologies to mimic the corresponding weather in the immovable cultural relic's metaverse, allowing users to experience an immersive feeling. The game experience mode sets up a series of game levels related to the culture of immovable cultural relics based on compiled information. Using XR technology, users can interact with the immovable cultural relics in the game. The game mode also allows for scoring and ranking based on interaction performance. Top-ranked users receive souvenirs related to immovable cultural relics, encouraging a deeper understanding of the cultural context behind the relics and promoting cultural dissemination.

[0048] In this embodiment, the steps of presenting the metaverse interaction through the virtual digital human further include: obtaining the metaverse interaction information and experience feedback information, and iteratively optimizing the immovable cultural relic metaverse according to the metaverse interaction information and experience feedback information.

[0049] In one embodiment, the metaverse interaction information includes the interaction with the immovable cultural relics in the metaverse and the interaction of virtual digital humans in the same scene in the metaverse. Among them, the interaction between virtual digital humans includes virtual room creation, voice chat, perspective following, etc.

[0050] In a specific embodiment, in virtual room creation, one person creates a virtual room (or some default public virtual rooms can also be selected), and others can enter the same virtual room by searching for the room number. There is a virtual commentator in each virtual room responsible for explaining the immovable cultural relics. During voice chat, the host can invite others, or others can apply to enter the same virtual room. In the same virtual room, virtual digital humans can interact through voice, just like interacting in real time in a real scene, and discuss and share the experience of visiting immovable cultural relics together. Moreover, in the same virtual room, the perspective of the virtual digital human can be directly controlled to follow the perspective of the virtual commentator, or the perspective can be followed by clicking on a certain virtual digital human.

[0051] In a specific embodiment, the experience feedback information is obtained by means of hotspot analysis. Specifically, by calculating the residence time of the virtual digital human within a certain range of a certain location, and classifying it into three levels A, B, and C according to the length of time. Less than 1 minute is C, 1 - 5 minutes is B, and more than 5 minutes is A. The longer the time, the more popular it is. Then, combined with the content that can be seen within a certain range of this point for analysis, so as to obtain what attracts it, whether it is the cultural relic itself or the visual expression of its interaction in the metaverse. If the cultural relic itself attracts it, it is determined that more attention needs to be paid to mining its own cultural information for subsequent optimization and iteration; if the visual expression of the interaction attracts it, the application of this type of visual expression in the metaverse can be expanded to promote the comprehensive promotion of the culture related to immovable cultural relics.

[0052] Furthermore, footprint sharing can also be carried out according to the user's instructions. The user can generate a trajectory line of their tour footprint in the immovable cultural relic metaverse, save it and share it with other friends, so that friends can follow your footprint for a tour next time. And, obtain the user's experience feedback according to the user's footprint sharing information (such as the time of browsing each object and the walking route through footprint sharing), and perform iterative optimization through this experience feedback.

[0053] In this embodiment, the feedback information also includes questionnaire survey information. Feedback is obtained through a combination of online and offline questionnaire surveys to facilitate the optimization and iteration of the immovable cultural relic metaverse. For example, based on the questionnaire survey results and the actual situation, shortcomings in the metaverse can be modified, and then the original metaverse scene can be replaced to achieve an upgrade and optimization.

[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0055] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing an immovable cultural relic meta-universe, characterized in that, The method for constructing the immovable cultural relic metaverse comprises the following steps: An overall three-dimensional model of the immovable cultural relic is obtained by means of oblique photogrammetry, and a main three-dimensional model and an accessory three-dimensional model of the immovable cultural relic are constructed according to point cloud or BIM data; Attribute data of the immovable cultural relic is obtained, the main three-dimensional model and the overall three-dimensional model are overlapped together, the overall three-dimensional model is trimmed according to the overlapping information, and the accessory three-dimensional model is moved to a position corresponding to the main three-dimensional model to form a metaverse scene; The attribute data is connected to the metaverse scene, and the metaverse scene is rendered; Login information is received, and after it is determined that the login is successful through the login information, a virtual digital person of the user is generated, and the metaverse interactive scene is displayed through the virtual digital person; Metaverse interactive information and experience feedback information are obtained, and the immovable cultural relic metaverse is iteratively optimized according to the metaverse interactive information and the experience feedback information, wherein the experience feedback information is obtained by means of hotspot analysis, including: the virtual digital person stays in a certain range of a place for a certain time, and is divided into three levels according to the time length, the longer the time, the more popular it is, and then the content that can be seen in the certain range of the place is analyzed to determine whether the immovable cultural relic itself or the interactive visualization expression in the metaverse attracts the virtual digital person; when the immovable cultural relic itself attracts the virtual digital person, it is determined that the cultural information of the immovable cultural relic itself needs to be excavated for subsequent optimization and iteration; when the interactive visualization expression attracts the virtual digital person, the application of the interactive visualization expression in the metaverse is expanded.

2. The method of claim 1, wherein, The step of generating the virtual digital person of the user specifically comprises the following steps: Various types of virtual digital persons are preset, facial information of the user is obtained, a matching virtual digital person is selected according to the recognition result of the facial information, and the virtual digital person is modified according to the facial information; Or the appearance information of the user is obtained, and the virtual digital person is generated by three-dimensional reconstruction according to the appearance information.

3. The method of claim 1, wherein the method further comprises: The step of displaying the metaverse interactive scene through the virtual digital person comprises the following steps: An experience mode selected by the user is obtained, experience content is displayed according to the experience mode, and interaction between virtual digital persons is performed according to an input instruction.

4. The method of claim 1, wherein the method is characterized by, The step of obtaining the overall three-dimensional model of the immovable cultural relic by means of oblique photogrammetry specifically comprises the following steps: Oblique photogrammetry data and near-ground data of the immovable cultural relic are obtained, and the overall three-dimensional model of the immovable cultural relic is constructed according to the oblique photogrammetry data and the near-ground data.

5. The method of claim 1, wherein the method further comprises: The step of obtaining the attribute data of the immovable cultural relic further comprises the following steps: According to the type of the attribute data, the attribute data is converted into a format corresponding to the type.

6. The method of claim 1, wherein the method further comprises: The step of overlapping the main three-dimensional model and the overall three-dimensional model specifically comprises the following steps: The main three-dimensional model, the overall three-dimensional model and the accessory three-dimensional model are loaded, and the main three-dimensional model and the overall three-dimensional model are overlapped together according to the same-named points in the main three-dimensional model and the overall three-dimensional model.

7. The method of claim 1, wherein the method further comprises: The step of moving the accessory three-dimensional model to a position corresponding to the main three-dimensional model specifically comprises the following steps: ​ Obtaining a spatial relationship between the accessory and the main body, and moving the three-dimensional model of the accessory according to the spatial relationship.

8. The method of claim 1, wherein the method further comprises: The step of attaching the attribute data to the metaverse scene specifically comprises: obtaining an identification code of each three-dimensional model in the metaverse scene, and binding the three-dimensional model and the attribute data together according to the identification code. ​

Images