3D Model Reconstruction Method, Device and Storage Medium of Ancient Building Neural Sparse Voxel Field Based on Ancient Chinese Text Information Mining

Abstract

The present invention discloses a method, device and storage medium for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining, belonging to the technical field of visual 3D reconstruction. The present invention uses a large language model to understand ancient texts, collects ancient building pictures to train the LoRA model, constructs a parameterized three-dimensional Mesh basic model, extracts the contour lines of ancient building images, generates a video set, then extracts frames to generate a frame number set, and then obtains sequence frame images. The camera pose set C1 is determined according to the sequence frame images. The camera pose set C1 and the sparse three-dimensional point cloud are input into the neural sparse voxel field model to generate a 3D model of the ancient building neural sparse voxel field; and optimization is carried out. The present invention can achieve high-precision 3D model reconstruction of historical buildings in a short time.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual three-dimensional reconstruction, and particularly relates to a method for reconstructing a 3D model of an ancient building neural sparse voxel field based on the mining of ancient text information. Background Art

[0002] With the damage, weathering or disappearance of historical buildings, the digital reconstruction of ancient buildings based on the remaining materials has become one of the core needs of researchers and related industries. Traditional building reconstruction methods usually rely on archaeological excavation, on-site measurement and the study of relics. However, these methods are costly, time-consuming, and it is difficult to obtain complete information in the case of severely damaged buildings.

[0003] Ancient texts (such as ancient literature, drawings, inscriptions, poems, etc.) often record information such as the structure, layout, decoration and use of buildings. By analyzing these text materials, important references can be provided for the reconstruction of ancient buildings. In recent years, with the development of natural language processing (NLP), computer vision and generative model technologies, building reconstruction technologies based on the segmentation and understanding of ancient text information have gradually emerged. This technology combines ancient text information with the existing building model database to generate a rough building model that conforms to historical descriptions, thereby realizing the virtual reconstruction of destroyed or incomplete buildings.

[0004] Although the building reconstruction technology based on ancient text information has certain application values in cultural heritage protection and historical research, the existing technologies still have many disadvantages and deficiencies, which are specifically analyzed as follows:

[0005] Ambiguity and vagueness of ancient texts. Ancient texts usually carry a large amount of ambiguous and vague descriptions. The lack of precision in building details. The building descriptions in ancient texts usually lack precise dimension and detail information, especially in terms of building materials, decorative details, proportions, etc., and it is difficult to obtain sufficient data. This incompleteness will result in the generated model lacking details or not being realistic enough.

[0006] Insufficient understanding of cultural background and style. Ancient buildings often have their specific cultural and historical backgrounds, and there are significant differences in architectural styles in different historical periods and regions. The existing models lack a deep understanding of these historical and cultural backgrounds and are difficult to correctly present the architectural styles of specific periods.

[0007] Strong data dependence and lack of high-quality data sets. The text-based building reconstruction method requires a large number of high-quality training data sets, including text descriptions and corresponding three-dimensional models of ancient buildings. However, the currently available data sets are relatively scarce, and most of them are modern buildings or simplified models, which cannot meet the needs of historical building reconstruction. Due to the lack of real three-dimensional data of ancient buildings and high-quality text description comparison data, it is difficult for the model to achieve a high-precision reconstruction effect during the training process.

[0008] Regarding the quality and computational complexity of generative models, generating 3D building models requires strong computational power, especially when high resolution and complex structures are involved. Existing generative models often struggle to balance quality and computational efficiency. Generating high-quality building models not only requires a large amount of computational resources but also takes a long time to generate, making it difficult to meet the needs of real-time applications or large-scale reconstruction. Summary of the Invention

[0009] To solve the technical problems existing in the prior art, the present invention provides a method for reconstructing a 3D model of an ancient building's neural sparse voxel field based on the mining of ancient text information. By using technical means such as natural language processing, targeted LoRA model training, AIGC ancient building video generation, and NSVF (Neural Sparse Voxel Fields) 3D reconstruction, it realizes the high-precision 3D model reconstruction of historical buildings, aiming to meet the needs of digital protection and virtual reproduction of cultural heritage.

[0010] The present invention provides a method for reconstructing a 3D model of an ancient building's neural sparse voxel field based on the mining of ancient text information, including the following steps:

[0011] Step S1: Adjust the large language model according to ancient text corpora, evaluate the accuracy of the adjusted large language model's understanding of ancient texts, and construct a parameterized 3D Mesh basic model A1 based on the adjusted large language model;

[0012] Step S2: Collect pictures of the same building type from the relevant dynasties of the target ancient building to be reproduced to form an ancient building image set X1; the pixel of the pictures is higher than the preset pixel;

[0013] Step S3: Label the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description labels, generate an ancient building image style data set X2 including images and their labels, and input the ancient building image style data set X2 into the Flux model base for training to generate a historical style LoRA model;

[0014] Step S4: Based on the parameterized 3D Mesh basic model A1, use the ControlNet model to extract the contour lines of ancient building images from different angles, and input the extracted contour lines of ancient building images from different angles into the historical style LoRA model to generate the optimal ancient building images from different angles;

[0015] Step S5: Input the optimal ancient building images from different angles into the locally deployed AI video generation model and input preset keywords to output N multi-view AIGC ancient building videos to form a video set VN;

[0016] Step S6: Extract frames from the generated video set VN using OpenCV, control the frame rate to obtain images at specified intervals, and obtain the frame number set X3;

[0017] Step S7: Preprocess the images in the frame number set X3 to obtain a sequence of frame images;

[0018] Step S8: Extract the features of the sequence of frame images through the COLMAP tool and perform feature matching to determine the camera pose of each image, and obtain the camera pose set C1;

[0019] Step S9: Input the camera pose set C1 and the image features matching the camera pose into COLMAP to generate a sparse three-dimensional point cloud, and input the sparse three-dimensional point cloud and the camera pose set C1 into the neural sparse voxel field model to generate an ancient building neural sparse voxel field 3D model;

[0020] Step S10: Optimize the parameters of the ancient building neural sparse voxel field 3D model by optimizing the rendering loss of the ancient building neural sparse voxel field 3D model, and gradually generate an ancient building neural sparse voxel field 3D model that meets the preset accuracy.

[0021] Preferably, step S1 specifically includes the following steps:

[0022] Step S101: Collect ancient Chinese texts;

[0023] Step S102: Organize the collected ancient Chinese texts to obtain an ancient Chinese text training data set corresponding to a specific dynasty;

[0024] Step S103: Organize a set of question-and-answer pairs from ancient Chinese literature, use the ancient Chinese text training data set and the set of question-and-answer pairs as inputs, and perform adjusted training on the large language model based on Qwen2.5 - 72B to obtain an adjusted large language model;

[0025] Step S104: Establish a text segmentation and understanding model based on the adjusted large language model;

[0026] Step S105: Use the ancient Chinese text data set to evaluate the accuracy of ancient Chinese understanding of the text segmentation and understanding model, and digitize the ancient building information in the ancient Chinese text through the text segmentation and understanding model to obtain ancient building structure data;

[0027] Step S106: Input the ancient building structure data into GH or Blender to construct a parametric Mesh basic model A1.

[0028] Preferably, step S3 specifically includes the following steps:

[0029] Step S301: Annotate the ancient building image set X1 according to historical style, integrate the ancient building image set X1 and the image description annotations to generate an ancient building image style dataset X2 including images and their annotations;

[0030] Step S302: input the ancient building image style dataset X2 into the Flux model base for LoRA model training to generate a historical style LoRA model;

[0031] Step S303: using the historical style LoRA model, inputting a preset prompt word, generating multiple ancient building images, and selecting the one with the best effect;

[0032] Step S304: Select the LoRA model corresponding to the ancient building image with the best effect as the LoRA model for subsequent use. Preferably, in step S6, the frame rate is controlled by the following formula to control the extracted frame number set X3:

[0033] X3=V S ×FPS,X3∈V N ;

[0034] Among them, X3 is the frame number set, V S The duration of the video generated by AI, V N A collection of N multi-perspective ancient building videos.

[0035] Preferably, the pre-processing in step S7 includes improving the clarity of each frame of image by denoising, brightness equalization and contrast enhancement, and the brightness equalization and contrast enhancement are achieved by histogram equalization.

[0036] Preferably, in step S8, the camera pose set C1 is determined based on the processed sequence frame images by COLMAP, and the camera extrinsic parameter matrix T is obtained and expressed as follows:

[0037] T = [R|t];

[0038] Where R is the rotation matrix and t is the translation vector.

[0039] Preferably, step S9 specifically includes the following steps:

[0040] Step S901: input the camera pose set C1 and the image features matching the camera pose into COLMAP to generate a sparse 3D point cloud;

[0041] Step S902: Input the sparse 3D point cloud and camera pose set C1 into the neural sparse voxel field model, and use the sparse voxel grid to perform 3D space division on each input frame image, dividing each frame image into a uniform 3D voxel grid;

[0042] Step S903: using the intersection of the camera ray and the 3D voxel grid to determine which 3D voxels are valid voxels, and eliminating invalid voxels. Valid voxels are 3D voxels that the camera ray passes through and interacts with.

[0043] Step S904: assigning an initial learnable voxel feature vector to each valid voxel;

[0044] Step S905: Using the neural sparse voxel field grid to learn the density, structure and texture of the image through sparse voxels and volume rendering formulas, a neural sparse voxel field 3D model of the ancient building is generated.

[0045] Preferably, in a neural sparse voxel field, high-resolution voxels are used in detail-rich areas, while low-resolution voxels are used in structure-simple areas.

[0046] Preferably, step S905 is specifically as follows: using a sparse convolutional network Sparse ConvNet to extract the structure and texture of each frame image from the neural sparse voxel field grid, a volume rendering module, each ray is emitted from a determined camera, interacts with the neural sparse voxel field grid along the path, calculates the color based on the volume rendering formula, and generates a neural sparse voxel field 3D model of the ancient building.

[0047] Preferably, step S10 specifically includes the following steps:

[0048] Step S1001: training the neural sparse voxel field 3D model of the ancient building by minimizing the difference between the rendered image and the real image to reduce rendering loss;

[0049] Step S1002: During the training process, the ancient architecture neural sparse voxel field 3D model will continuously optimize the sparsification of three-dimensional voxels and the structure of the neural sparse voxel grid, so that the ancient architecture neural sparse voxel field 3D model is closest to the sparse three-dimensional point cloud;

[0050] Step S1003: Automatically label the material and component information of the ancient building in the neural sparse voxel field 3D model of the ancient building, and store them in layers according to the building structure.

[0051] The present invention provides a device for reconstructing a 3D model of an ancient building using a neural sparse voxel field based on ancient text information mining, comprising a processor capable of executing the above-mentioned method for reconstructing a 3D model of an ancient building using a neural sparse voxel field based on ancient text information mining.

[0052] The present invention provides a storage medium having a computer program stored thereon. When the computer program is executed, the method for reconstructing a 3D model of an ancient building using a neural sparse voxel field based on ancient text information mining can be performed.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The present invention can automatically extract information such as the structure, style, and material of buildings from ancient text, and generate a highly realistic 3D model based on the neural coefficient voxel field technology to achieve a detailed reproduction of historical buildings. Through multi-scale detail enhancement and volume rendering technology, the authenticity and detail expressiveness of the model are ensured.

[0055] 2. Traditional building reconstruction methods rely on archaeological sites or picture materials, while the present invention utilizes the descriptive information in ancient documents to provide a new data source for building restoration. Even in the absence of physical remains or image materials, the reconstruction of ancient buildings can still be achieved, greatly expanding the scope of reconstruction applications.

[0056] 3. The present invention adopts a technical solution that combines a sparse voxel grid with a neural sparse voxel field representation, reducing the computational and storage costs of the 3D model. The neural sparse voxel field uses high-resolution voxels in areas with rich details and low-resolution voxels in areas with simple structures, adaptively adjusting the voxel resolution and improving the efficiency of the reconstruction process.

[0057] 4. The present invention achieves high-quality rendering effects from multiple perspectives through volume rendering, enabling the generated 3D model to present consistency and realism under different perspectives and lighting conditions, facilitating multi-angle observation in virtual display, cultural heritage protection, and academic research. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of a method for reconstructing a 3D model of an ancient building's neural sparse voxel field based on mining ancient text information according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] The present invention provides a method for reconstructing a 3D model of an ancient building's neural sparse voxel field based on mining ancient text information, including the following steps:

[0061] Step S1, adjust the large language model according to ancient text corpus, evaluate the accuracy of the adjusted large language model's understanding of ancient text, and construct a parameterized 3D Mesh basic model A1 based on the adjusted large language model;

[0062] Step S2, collect pictures of the same building type from the relevant dynasties of the target reproduced ancient buildings to form an ancient building image set X1; the pixel of the pictures is higher than the preset pixel;

[0063] Step S3, annotate the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description annotation, generate an ancient building image style data set X2 including the images and their annotations, and input the ancient building image style data set X2 into the Flux model base for training to generate a historical style LoRA model;

[0064] Step S4, based on the parametric 3D Mesh basic model A1, use the ControlNet model to extract the contour lines of ancient building images at different angles, and input the extracted contour lines of ancient building images at different angles into the historical style LoRA model to generate the optimal ancient building images at different angles;

[0065] Step S5, input the optimal ancient building images at different angles into the locally deployed AI video generation model, and input the preset keywords to output N multi-view AIGC ancient building videos to form a video set VN;

[0066] Step S6, use OpenCV to extract frames from the generated video set VN, control the frame rate to obtain images at specified intervals, and obtain a frame number set X3;

[0067] Step S7, preprocess the images in the frame number set X3 to obtain sequence frame images;

[0068] Step S8, extract the features of the sequence frame images through the COLMAP tool and perform feature matching to determine the camera pose of each image, and obtain a camera pose set C1;

[0069] Step S9, input the camera pose set C1 and the image features matching the camera pose into COLMAP to generate a sparse 3D point cloud, and input the sparse 3D point cloud and the camera pose set C1 into the neural sparse voxel field model to generate an ancient building neural sparse voxel field 3D model;

[0070] Step S10, optimize the parameters of the ancient building neural sparse voxel field 3D model by optimizing the rendering loss of the ancient building neural sparse voxel field 3D model, and gradually generate an ancient building neural sparse voxel field 3D model that meets the preset accuracy.

[0071] According to a specific embodiment of the present invention, step S1 specifically includes the following steps:

[0072] Step S101: Collect ancient text;

[0073] Step S102: Sort out the collected ancient text to obtain an ancient text training data set corresponding to the dynasty;

[0074] Step S103: sorting out a set of question-answer pairs from ancient Chinese literature, taking the ancient Chinese text training dataset and the set of question-answer pairs as input, and adjusting and training the large language model based on Qwen2.5-72B to obtain an adjusted large language model;

[0075] Step S104: establishing a text segmentation and understanding model based on the adjusted large language model;

[0076] Step S105: using the ancient text data set to evaluate the accuracy of the text segmentation and understanding model in understanding the ancient text, and digitizing the ancient building information in the ancient text through the text segmentation and understanding model to obtain the ancient building structure data;

[0077] Step S106: Input the ancient building structure data into GH or Blender to construct a parametric Mesh basic model A1.

[0078] According to a specific embodiment of the present invention, step S3 specifically includes the following steps:

[0079] Step S301: Annotate the ancient building image set X1 according to historical style, integrate the ancient building image set X1 and the image description annotations to generate an ancient building image style dataset X2 including images and their annotations;

[0080] Step S302: input the ancient building image style dataset X2 into the Flux model base for LoRA model training to generate a historical style LoRA model;

[0081] Step S303: using the historical style LoRA model, inputting a preset prompt word, generating multiple ancient building images, and selecting the one with the best effect;

[0082] Step S304: Select the LoRA model corresponding to the ancient building image with the best effect as the LoRA model for subsequent use.

[0083] According to a specific embodiment of the present invention, in step S6, the frame rate is controlled by the following formula to control the extracted frame set X3:

[0084] X3=V S ×FPS,X3∈V N ;

[0085] Among them, X3 is the frame number set, V S The length of the video generated by AI, V N It is a collection of N multi-perspective ancient building videos.

[0086] According to a specific embodiment of the present invention, the pre-processing in step S7 includes improving the clarity of each frame of image by denoising, brightness equalization and contrast enhancement, and the brightness equalization and contrast enhancement are achieved by histogram equalization.

[0087] According to a specific embodiment of the present invention, in step S8, the camera pose set C1 is determined based on the processed sequence frame images by COLMAP, and the camera extrinsic parameter matrix T is obtained and expressed as follows:

[0088] T = [R|t];

[0089] Where R is the rotation matrix and t is the translation vector.

[0090] According to a specific embodiment of the present invention, step S9 specifically includes the following steps:

[0091] Step S901: input the camera pose set C1 and the image features matching the camera pose into COLMAP to generate a sparse 3D point cloud;

[0092] Step S902: Input the sparse 3D point cloud and camera pose set C1 into the neural sparse voxel field model, and use the sparse voxel grid to perform 3D space division on each input frame image, dividing each frame image into a uniform 3D voxel grid;

[0093] Step S903: using the intersection of the camera ray and the 3D voxel grid to determine which 3D voxels are valid voxels, and eliminating invalid voxels. Valid voxels are 3D voxels that the camera ray passes through and interacts with.

[0094] Step S904: assigning an initial learnable voxel feature vector to each valid voxel;

[0095] Step S905: Using the neural sparse voxel field grid to learn the density, structure and texture of the image through sparse voxels and volume rendering formulas, a neural sparse voxel field 3D model of the ancient building is generated.

[0096] According to a specific embodiment of the present invention, in a neural sparse voxel field, high-resolution voxels are used in detail-rich areas, while low-resolution voxels are used in structurally simple areas.

[0097] According to a specific embodiment of the present invention, step S905 is specifically as follows: using a sparse convolutional network SparseConvNet to extract the structure and texture of each frame image from the neural sparse voxel field grid, a volume rendering module, each ray is emitted from a determined camera, interacts with the neural sparse voxel field grid along the path, calculates the color based on the volume rendering formula, and generates a neural sparse voxel field 3D model of the ancient building.

[0098] According to a specific embodiment of the present invention, step S10 specifically includes the following steps:

[0099] Step S1001: Train the 3D model of the ancient building neural sparse voxel field by minimizing the difference between the rendered image and the real image to reduce the rendering loss;

[0100] Step S1002: During the training process, the 3D model of the ancient building neural sparse voxel field will continuously optimize the sparsification of the three-dimensional voxels and the structure of the neural sparse voxel grid to make the 3D model of the ancient building neural sparse voxel field closest to the sparse three-dimensional point cloud;

[0101] Step S1003: Automatically annotate the materials and component information of the ancient buildings in the 3D model of the ancient building neural sparse voxel field and store them in layers according to the building structure.

[0102] The present invention provides a device for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining, including a processor, and the processor can execute the above-mentioned method for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining.

[0103] The present invention provides a storage medium, on which a computer program is stored, and when the computer program is executed, it can execute the above-mentioned method for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining.

[0104] Embodiment 1

[0105] The following will detail the method for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining of the present invention.

[0106] As Figure 1 , the present invention provides a method for reconstructing a 3D model of an ancient building neural sparse voxel field based on ancient text information mining, including the following steps:

[0107] Step S1: Adjust the large language model according to the ancient text corpus, evaluate the accuracy of the adjusted large language model in understanding ancient texts, and construct a parameterized three-dimensional Mesh basic model A1 based on the adjusted large language model;

[0108] Step S2: Collect pictures of the same building type of the target ancient building to be reproduced in the relevant dynasties to form an ancient building image set X1; the pixel of the picture is higher than the preset pixel;

[0109] Step S3, label the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description labels, generate the ancient building image style dataset X2 including the images and their labels, and input the ancient building image style dataset X2 into the Flux model base for training to generate the historical style LoRA model;

[0110] Step S4, based on the parametric 3D Mesh basic model A1, use the ControlNet model to extract the contour lines of the ancient building images at different angles, and input the extracted contour lines of the ancient building images at different angles into the historical style LoRA model to generate the optimal ancient building images at different angles;

[0111] Step S5, input the optimal ancient building images at different angles into the locally deployed AI video generation model, and input the preset keywords to output N multi-view AIGC ancient building videos, forming the video set VN;

[0112] Step S6, use OpenCV to extract frames from the generated video set VN, control the frame rate to obtain images at specified intervals, and obtain the frame number set X3;

[0113] Step S7, preprocess the images in the frame number set X3 to obtain sequential frame images;

[0114] Step S8, extract the features of the sequential frame images through the COLMAP tool and perform feature matching to determine the camera poses of each image, obtaining the camera pose set C1;

[0115] Step S9, input the camera pose set C1 and the image features matching the camera poses into COLMAP to generate a sparse 3D point cloud, and input the sparse 3D point cloud and the camera pose set C1 into the neural sparse voxel field model to generate the ancient building neural sparse voxel field 3D model;

[0116] Step S10, optimize the parameters of the ancient building neural sparse voxel field 3D model by optimizing the rendering loss of the ancient building neural sparse voxel field 3D model, and gradually generate the ancient building neural sparse voxel field 3D model that meets the preset accuracy.

[0117] Embodiment 2

[0118] The following details the method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the mining of ancient text information of the present invention.

[0119] Step S1, adjust the large language model according to the ancient text corpus, evaluate the accuracy of the adjusted large language model in understanding ancient texts, and construct the parametric 3D Mesh basic model A1 based on the adjusted large language model;

[0120] Sort out a large amount of ancient Chinese language corpora corresponding to dynasties to form a dataset, input it into the large language model, and sort out a large number of Q&A pairs from ancient Chinese literature and input them into the large language model at the same time to adjust the large language model and build a large language model specifically for ancient Chinese, making it more convenient and fast to extract information from ancient Chinese.

[0121] In this embodiment, the method for evaluating the understanding ability: sort out an empty field table and an ancient Chinese literature, import them into the large language model, let the large language model fill in the table data, and compare it with the content of the corresponding ancient Chinese literature to see the matching degree between the two, and evaluate the understanding ability of the adjusted large language model for ancient Chinese.

[0122] In this embodiment, after evaluation, test the accuracy of the adjusted large language model, and export the data Json file required for creating the Mesh model in the next step.

[0123] In this embodiment, step S1 specifically includes the following steps:

[0124] Step S101: Collect ancient Chinese texts;

[0125] Step S102: Sort out the collected ancient Chinese texts to obtain an ancient Chinese text training dataset corresponding to the dynasty;

[0126] Step S103: Sort out a set of Q&A pairs from ancient Chinese literature, use the ancient Chinese text training dataset and the set of Q&A pairs as inputs, and adjust and train the large language model based on Qwen2.5-72B to obtain the adjusted large language model;

[0127] Step S104: Establish a text segmentation understanding model based on the adjusted large language model;

[0128] Step S105: Use the ancient Chinese text dataset to evaluate the accuracy of ancient Chinese understanding of the text segmentation understanding model, and digitize the ancient building information in the ancient Chinese text through the text segmentation understanding model to obtain ancient building structure data;

[0129] Step S106: Input the ancient building structure data into GH or Blender to construct a parametric Mesh basic model A1. Export the Json file of the ancient building structure data through the large language model and input it into GH or Blender to construct the Mesh basic model A1.

[0130] The parameterization here means using the large language model to extract specific descriptions of the target building dimensions from ancient Chinese texts, perform numerical processing, generate a table filled in according to the input empty field table as a standard, and synthesize the text and corresponding data in the table into a Json file and import it into the modeling software to create a Mesh model.

[0131] Step S2, collect pictures of the same type of ancient buildings in the relevant dynasties of the target reproduction of ancient buildings to form an ancient building image set X1; the pixels of the pictures are higher than the preset pixels;

[0132] Step S3, annotate the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description annotation, generate an ancient building image style data set X2 including the images and their annotations, and input the ancient building image style data set X2 into the Flux model base for training to generate a historical style LoRA model;

[0133] In this embodiment, step S3 specifically includes the following steps:

[0134] Step S301: Annotate the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description annotation, and generate an ancient building image style data set X2 including the images and their annotations;

[0135] Step S302: Input the ancient building image style data set X2 into the Flux model base for LoRA model training to generate a historical style LoRA model;

[0136] Step S303: Use the historical style LoRA model, input the preset prompt words, generate multiple ancient building images, and select an ancient building image with the best effect;

[0137] Step S304: Select the LoRA model corresponding to the ancient building image with the best effect as the LoRA model for subsequent use.

[0138] In this embodiment, the historical style is a.txt file formed by the descriptive annotation of the pictures. Through an ancient building picture corresponding to a descriptive.txt file, a comprehensive ancient building image style data set X2 of images plus texts is formed. The purpose of sorting out the ancient building image style data set X2 is to help train the preset LoRA model of the ancient building historical style.

[0139] Step S4, based on the parametric 3D Mesh basic model A1, use the ControlNet model to extract the contour lines of ancient building images at different angles, and input the extracted contour lines of ancient building images at different angles into the historical style LoRA model to generate the optimal ancient building images at different angles;

[0140] Output multiple pictures of the ancient building model at different angles in the software of the constructed parametric Mesh model, input the pictures of the ancient building at different angles into the ControlNet model, extract the contour lines of the accurate ancient building images, combine with the historical style LoRA model to form a workflow, input the preset keywords, generate a large number of target images for screening out the optimal images as the input for the next video generation.

[0141] ControlNet cannot be directly combined with the parametric Mesh model. First, export multiple images at different angles from the Mesh model and input them into ControlNet for the extraction and control of the ancient building contour.

[0142] Step S5, input the optimal ancient building images at different angles into the AI video generation model deployed locally, and input the preset keywords to output N multi-view AIGC ancient building videos, forming a video set VN.

[0143] Step S6, use OpenCV to extract frames from the generated video set VN, control the frame rate to obtain images at specified intervals, and get a frame number set X3. In this embodiment, the frame rate is controlled by the following formula to control the extracted frame number set X3:

[0144] X3 = V S ×FPS, X3 ∈ V N ;

[0145] Where, X3 is the frame number set, V S is the duration of the video generated by the AI, and V N is the set of N multi-view ancient building videos.

[0146] Step S7, preprocess the images in the frame number set X3 to obtain sequential frame images. The preprocessing in this embodiment includes improving the clarity of each frame image through denoising, brightness equalization, and contrast enhancement, and the brightness equalization and contrast enhancement are achieved through histogram equalization.

[0147] Step S8, extract the features of the sequential frame images through the COLMAP tool and perform feature matching to determine the camera pose of each image, obtaining a camera pose set C1. In this embodiment, the COLMAP is used to determine the camera pose set C1 according to the processed sequential frame images, and the external parameter matrix T of the camera is expressed by the following formula:

[0148] T = [R|t];

[0149] Where, R is the rotation matrix and t is the translation vector.

[0150] Step S9: Input the camera pose set C1 and the image features matching the camera poses into COLMAP to generate a sparse 3D point cloud, and then input the sparse 3D point cloud and the camera pose set C1 into the neural sparse voxel field model to generate an ancient building neural sparse voxel field 3D model;

[0151] In this embodiment, step S9 specifically includes the following steps:

[0152] Step S901: Input the camera pose set C1 and the image features matching the camera poses into COLMAP to generate a sparse 3D point cloud;

[0153] Step S902: Input the sparse 3D point cloud and the camera pose set C1 into the neural sparse voxel field model, use a sparse voxel grid to divide the 3D space of each input frame image, and divide each frame image into a uniform 3D voxel grid;

[0154] Step S903: Use the intersections of the camera rays and the 3D voxel grid to determine which 3D voxels are valid voxels, and remove the invalid voxels. A valid voxel refers to a 3D voxel that the camera ray passes through and interacts with;

[0155] Step S904: Assign an initial learnable voxel feature vector to each valid voxel;

[0156] Step S905: Use the neural sparse voxel field grid to learn the density, structure, and texture of the image through the sparse voxel and volume rendering formula, and generate an ancient building neural sparse voxel field 3D model.

[0157] Among them, step S905 is specifically: Use the Sparse ConvNet to extract the structure and texture of each frame image from the neural sparse voxel field grid. For the volume rendering module, each ray is emitted from the determined camera, interacts with the neural sparse voxel field grid along the path, calculates the color based on the volume rendering formula, and generates an ancient building neural sparse voxel field 3D model.

[0158] Step S10: Optimize the parameters of the ancient building neural sparse voxel field 3D model by optimizing the rendering loss of the ancient building neural sparse voxel field 3D model, and gradually generate an ancient building neural sparse voxel field 3D model that meets the preset accuracy.

[0159] In this embodiment, step S10 specifically includes the following steps:

[0160] Step S1001: Train the ancient building neural sparse voxel field 3D model by minimizing the difference between the rendered image and the real image to reduce the rendering loss;

[0161] Step S1002: During the training process, the ancient architecture neural sparse voxel field 3D model continuously optimizes the sparsification of three-dimensional voxels and the structure of the neural sparse voxel grid to make the ancient architecture neural sparse voxel field 3D model closest to the sparse three-dimensional point cloud;

[0162] Step S1003: Automatically annotate the materials and component information of the ancient architecture in the ancient architecture neural sparse voxel field 3D model and store them in layers according to the building structure.

[0163] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for reconstructing a 3D model of an ancient building's neural sparse voxel field based on the mining of ancient text information, characterized in that, It includes the following steps: Step S1: Adjust the large language model according to ancient Chinese corpora, evaluate the accuracy of the adjusted large language model in understanding ancient Chinese, and construct a parametric 3D Mesh basic model A1 based on the adjusted large language model; Step S2: Collect pictures of the same building type in the relevant dynasties of the target reproduced ancient buildings to form an ancient building image set X1; the pixel of the pictures is higher than the preset pixel; Step S3: Label the ancient building image set X1 according to the historical style, integrate the ancient building image set X1 and the image description labels, generate an ancient building image style data set X2 including images and their labels, and input the ancient building image style data set X2 into the Flux model base for training to generate a historical style LoRA model; Step S4: Based on the parametric 3D Mesh basic model A1, use the ControlNet model to extract the contour lines of ancient building images from different angles, and input the extracted contour lines of ancient building images from different angles into the historical style LoRA model to generate the optimal ancient building images from different angles; Step S5: Input the optimal ancient building images from different angles into the locally deployed AI video generation model, and input the preset keywords to output N multi-view AIGC ancient building videos to form a video set VN; Step S6: Use OpenCV to extract frames from the generated video set VN, control the frame rate to obtain images at specified intervals, and obtain a frame number set X3; Step S7: Preprocess the images in the frame number set X3 to obtain sequential frame images; Step S8: Extract the features of the sequential frame images through the COLMAP tool and perform feature matching to determine the camera poses of each image, and obtain a camera pose set C1; Step S9: Input the camera pose set C1 and the image features matching the camera poses into COLMAP to generate a sparse 3D point cloud, and input the sparse 3D point cloud and the camera pose set C1 into the neural sparse voxel field model to generate an ancient building neural sparse voxel field 3D model; Step S10: Optimize the parameters of the ancient building neural sparse voxel field 3D model by optimizing the rendering loss of the ancient building neural sparse voxel field 3D model, and gradually generate an ancient building neural sparse voxel field 3D model that meets the preset accuracy.

2. The method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, wherein Step S1 specifically includes the following steps: Step S101: Collect ancient Chinese texts; Step S102: Organize the collected ancient Chinese texts to obtain an ancient Chinese text training data set corresponding to the relevant dynasty; Step S103: Sort out a set of question and answer pairs from ancient Chinese literature, use the ancient Chinese text training data set and the set of question and answer pairs as inputs, and perform adjustment training on the large language model based on Qwen2.5-72B to obtain an adjusted large language model; Step S104: Establish a text segmentation and understanding model based on the adjusted large language model; Step S105: Evaluate the accuracy of the text segmentation and understanding model in understanding ancient Chinese by using the ancient Chinese text data set, and digitize the ancient building information in the ancient Chinese text through the text segmentation and understanding model to obtain ancient building structure data; Step S106: Input the ancient building structure data into GH or Blender to construct a parametric Mesh basic model A1.

3. The method for reconstructing a 3D model of an ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, characterized in that Step S3 specifically includes the following steps: Step S301: Annotate the ancient building image set X1 according to historical style, integrate the ancient building image set X1 and the image description annotations to generate an ancient building image style dataset X2 including images and their annotations; Step S302: input the ancient building image style dataset X2 into the Flux model base for LoRA model training to generate a historical style LoRA model; Step S303: using the historical style LoRA model, inputting a preset prompt word, generating multiple ancient building images, and selecting the one with the best effect; Step S304: Select the LoRA model corresponding to the ancient building image with the best effect as the LoRA model for subsequent use.

4. The method for reconstructing a 3D model of an ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, characterized in that, In step S6, the frame rate is controlled by the following formula to control the extracted frame set X3: X3 = V S × FPS, X3 ∈ V N ; Among them, X3 is a set of frame numbers, and V S is the duration of the video generated by AI, and V N is a set of N multi-view ancient building videos.

5. The method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, wherein, The pre-processing in step S7 includes improving the clarity of each frame of image by denoising, brightness equalization and contrast enhancement. The brightness equalization and contrast enhancement are achieved by histogram equalization.

6. The method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the excavation of ancient text information according to claim 1, characterized in that In step S8, the camera pose set C1 is determined based on the processed sequence frame images through COLMAP, and the camera's external parameter matrix T is obtained and expressed as follows: T = [R|t]; Where R is the rotation matrix and t is the translation vector.

7. The method for reconstructing a 3D model of an ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, characterized in that, Step S9 specifically includes the following steps: Step S901: input the camera pose set C1 and the image features matching the camera pose into COLMAP to generate a sparse 3D point cloud; Step S902: Input the sparse 3D point cloud and camera pose set C1 into the neural sparse voxel field model, and use the sparse voxel grid to perform 3D space division on each input frame image, dividing each frame image into a uniform 3D voxel grid; Step S903: using the intersection of the camera ray and the 3D voxel grid to determine which 3D voxels are valid voxels, and eliminating invalid voxels. Valid voxels are 3D voxels that the camera ray passes through and interacts with. Step S904: assigning an initial learnable voxel feature vector to each valid voxel; Step S905: Using the neural sparse voxel field grid to learn the density, structure and texture of the image through sparse voxels and volume rendering formulas, a neural sparse voxel field 3D model of the ancient building is generated.

8. The method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the mining of ancient text information according to claim 7, characterized in that, Step S905 is specifically as follows: using the sparse convolutional network Sparse ConvNet to extract the structure and texture of each frame image from the neural sparse voxel field grid, the volume rendering module, each ray is emitted from a determined camera, interacts with the neural sparse voxel field grid along the path, calculates the color based on the volume rendering formula, and generates a neural sparse voxel field 3D model of the ancient building.

9. The method for reconstructing the 3D model of the ancient building neural sparse voxel field based on the mining of ancient text information according to claim 1, characterized in that, Step S10 specifically includes the following steps: Step S1001: training the neural sparse voxel field 3D model of the ancient building by minimizing the difference between the rendered image and the real image to reduce rendering loss; Step S1002: During the training process, the ancient building neural sparse voxel field 3D model continuously optimizes the sparsification of three-dimensional voxels and the structure of the neural sparse voxel grid, making the ancient building neural sparse voxel field 3D model closest to the sparse three-dimensional point cloud; Step S1003: Automatically annotate the material and component information of the ancient building in the ancient building neural sparse voxel field 3D model and store them in layers according to the building structure.

10. An ancient building neural sparse voxel field 3D model reconstruction device based on ancient text information mining, characterized in that, It includes a processor that can execute the method for reconstructing the ancient building neural sparse voxel field 3D model based on the mining of ancient text information according to any one of claims 1-8.

11. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is executed, it can execute the method for reconstructing the ancient building neural sparse voxel field 3D model based on the mining of ancient text information according to any one of claims 1-8.

Images