Architectural reconstruction method based on structural prior and small sample domain adaptation

Abstract

The application relates to the technical field of three-dimensional reconstruction and sparse view angle scene reconstruction, in particular to an ancient building reconstruction method based on structure prior and small sample domain self-adaption, which comprises the following steps: establishing a self-reference prior by detecting the symmetry and repeated structure of a building, performing small sample domain self-adaption fine adjustment on a general generation prior, and combining an uncertainty perception confidence fusion and a retake guide to realize a quick and reliable reconstruction Gaussian splashing method, which is suitable for the scene of quick modeling of an ancient building monomer in the digital protection of cultural heritage under the condition that the on-site collection is limited.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction and sparse view scene reconstruction technology, and in particular to a method for reconstructing ancient buildings based on structural priors and small sample domain adaptation. Background Technology

[0002] Existing Gaussian splash regularization methods for sparse viewpoints often introduce monocular depth priors or geometric smoothness constraints to alleviate overfitting problems caused by insufficient viewpoint numbers. These prior models are usually trained on general scene datasets. When dealing with scenes with unique styles and complex geometric forms, such as carved components, brackets, and painted decorations in ancient architecture, the estimation results of general priors are prone to oversmoothing or deviations from the actual component shapes, making it difficult to preserve the geometric authenticity of detailed carved components.

[0003] Existing sparse view completion methods based on diffusion generative models provide additional training supervision by generating pseudo-observed images for unobserved viewpoints. These generative models are also trained on general image datasets and lack prior knowledge of specific ancient architectural styles, such as the component proportions, painted patterns, and brick and stone masonry methods of traditional wooden buildings in specific regions. When filling in areas not covered by sparse viewpoints, they are prone to generating fabricated details that are inconsistent with the actual style of the target building, affecting the authenticity of the reconstruction results and the value of cultural relics records.

[0004] Existing generalizable feedforward Gaussian splash generation models rely on joint training with large-scale multi-scene data to obtain cross-scene generalization ability. However, ancient buildings generally have unique shapes and material characteristics. There is a significant domain difference between the distribution of training data and the actual geometric material distribution of the target ancient building. Existing feedforward methods have limited generalization effect in such cross-domain scenarios, and their performance degrades significantly when directly applied to new ancient building targets.

[0005] Chinese Patent Publication No. CN119477770A discloses a method, apparatus, and storage medium for generating orthophotos based on three-dimensional Gaussian splashing. The method includes: responding to an orthophoto generation signal for a measured building area, acquiring multi-view images of the measured building area from multiple perspectives, and determining the image acquisition parameters of the image acquisition device corresponding to the multi-view images; performing cross-image matching on feature points in each image, and determining feature point pairs in each image based on the matching results; performing three-dimensional reconstruction on the regional points in the measured building area represented by the feature point pairs based on the feature point pairs and the image acquisition parameters to obtain a three-dimensional point cloud, and performing Gaussian distribution processing on the three-dimensional point cloud to obtain multiple three-dimensional Gaussian ellipsoids; determining a two-dimensional projection plane, and determining the depth information of each three-dimensional Gaussian ellipsoid from the two-dimensional projection plane; and splashing each three-dimensional Gaussian ellipsoid onto the two-dimensional projection plane based on the depth information to obtain an orthophoto of the measured building area. However, the above scheme has the following problems: it does not utilize the structural regularity of the building as prior information, fails to use the reconstruction results of the fully observed part of the same building as the self-reference prior of the unobserved part of the building, and repeatedly uses the structural regularity of the building itself to alleviate the lack of observation from a sparse perspective. Summary of the Invention

[0006] To address this, the present invention provides a method for reconstructing ancient buildings based on structural priors and small sample domain adaptation. This method overcomes the shortcomings of existing technologies that fail to utilize the structural regularities of ancient buildings, such as left-right symmetry and repetitive arrangement of components, as prior information. It also fails to use the reconstruction results of the fully observed parts of the same building as self-reference priors for the unobserved parts of the building, thus repeatedly utilizing the structural regularities of the building itself to alleviate the problem of insufficient observation from sparse perspectives.

[0007] To achieve the above objectives, this invention provides a method for reconstructing ancient buildings based on structural priors and small-sample domain adaptation, comprising: Step S1: Acquire small sample sparse view images of the target building, complete preliminary pose recovery and sparse geometry reconstruction, generate a small sample sparse view image sequence as input, and output preliminary camera pose, sparse point cloud and initial Gaussian set. Step S2: Detect the symmetry planes and repeating component groups contained in the target building, construct a structural relationship graph for self-reference completion, input the preliminary camera pose, the sparse point cloud and the initial Gaussian set into the structural relationship graph, and output the symmetry plane parameters and the self-reference relationship graph; Step S3: Using the small sample sparse view image and its symmetric augmented version as fine-tuning data, perform lightweight domain adaptive fine-tuning to generate a prior model on the general pre-trained prior model so that the prior model can be adapted to the target building. Input the general pre-trained image generation or monocular depth prior model, the small sample sparse view image and the symmetry plane parameters into the prior model, and output the domain adaptation generation prior and adaptation parameters. Step S4: Combine the self-reference graph and the domain adaptation generation prior to provide corresponding pseudo-supervision signals for regions not covered by different types of sparse views. The inputs are the initial Gaussian set, the self-reference graph and the domain adaptation generation prior. The output is the Gaussian parameters after joint supervision optimization. Step S5: Dynamically adjust the fusion weights of the self-reference prior and the generated prior based on the uncertainty of the reconstruction results of each region, and adjust the reshooting perspective. Generate rendering residuals based on the Gaussian parameters optimized by joint supervision. Input the pseudo-supervision source labels, rendering residuals and the flags indicating whether the site has the conditions for supplementary acquisition for each region. Output the dynamically adjusted fusion weights, the angle of the reshooting perspective and the regional uncertainty labels. Step S6: After the reconstruction converges, evaluate the reconstruction results and deposit the style adaptation knowledge increment obtained from this domain adaptive fine-tuning. Input the converged reconstruction results, the adaptation parameters and the target architectural style category label, and output the reconstruction quality assessment report and the updated style adaptation parameter library.

[0008] Furthermore, the process of outputting the initial camera pose, sparse point cloud, and initial Gaussian set includes, Step S101: Perform motion reconstruction structure solving on the small sample sparse view image to obtain sparse point cloud, and use a preset matching threshold and local feature descriptor type in the feature matching stage. Step S102: Perform preliminary filtering on the sparse point cloud obtained by the solution to remove geometric outliers and generate a preliminary sparse point cloud; Step S103: Using the initial sparse point cloud combined with the pose, initialize the position, opacity and initial scale of the three-dimensional Gaussian set, and output the initial camera pose, sparse point cloud and initial Gaussian set. In particular, Gaussian sparse points are not generated in the spatial regions not covered by the sparse point cloud.

[0009] Furthermore, the process of outputting the symmetry plane parameters and the self-reference graph includes, Step S201: Perform symmetry detection on the input preliminary camera pose, sparse point cloud and initial Gaussian set. By comparing the mirror matching degree of the point cloud distribution on both sides of the candidate symmetry plane, identify several symmetry planes of the target building as a whole or in part, and output symmetry plane parameters based on the symmetry planes. Step S202: Perform similarity clustering on the local geometry of the target building, identify the repeating component groups with repeated arrangement of components, record several spatial transformation relationships within each repeating component group, and output the repeating component group; Step S203: Based on the detection results of the symmetry plane and the repeating component group, mark each spatial region in the target building to indicate whether there is a symmetric or repeating correspondence that can be used for self-reference. Based on the observation degree of the spatial region, record the correspondence of the spatial region and the corresponding spatial transformation parameters, and construct and output the self-reference relationship diagram.

[0010] Furthermore, based on the observation level of the spatial region, the correspondence of the spatial regions and the corresponding spatial transformation parameters are recorded, wherein, If the spatial region has been observed from a sparse perspective on the other side of the symmetry plane or in the remaining instances of the repeating component group to which it belongs, then record the correspondence and the corresponding spatial transformation parameters.

[0011] Furthermore, the process of generating prior and adaptation parameters for the output domain adaptation includes, Step S301: Using the symmetry plane as a reference, perform a mirror transformation on the sparse view image to generate an augmented image, and combine the sparse view image and the augmented image to form a small-scale domain dataset. Step S302: Introduce lightweight adaptation parameters into the general pre-trained prior model and perform fine-tuning training. Keep the main parameters of the general pre-trained prior model frozen and drive the adaptation parameters to converge with the small-scale domain dataset to generate the prior model. Step S303: Verify the validity of the fine-tuned adaptation parameters to output the domain adaptation prior and the adaptation parameters.

[0012] Furthermore, the process of validating the adaptation parameters after fine-tuning training includes, The validation images were used to compare the consistency between the prior model's generated results for local areas of the target building before and after fine-tuning training and the actual observation results. If the degree of consistency is greater than the preset degree threshold, then the consistency is determined to be improved and the fine-tuning is effective; If the degree of consistency is less than or equal to a preset threshold, it is determined that the fine-tuning hyperparameters should be adjusted and the fine-tuning training should be re-executed.

[0013] Furthermore, the process of outputting the Gaussian parameters optimized through joint supervision includes, Step S401: For the unobserved areas marked as having corresponding relationships in the self-reference relationship diagram, according to the corresponding spatial transformation parameters, the Gaussian parameter transformation of the corresponding area of ​​the fully observed part of the target building itself is mapped to the unobserved area, which serves as the self-reference pseudo-supervision of the area. Step S402: For regions that have no corresponding relationship and are not directly observed by sparse viewpoints, generate pseudo-observation images or depth estimates for the regions using the domain adaptation generation prior, as pseudo-supervision for the generation prior of the regions. Step S403: Construct the photometric loss of real sparse view observation, the self-reference pseudo-supervision loss and the generated prior pseudo-supervision loss according to their respective applicable regions, so as to jointly drive the joint optimization of Gaussian parameters and output the Gaussian parameters after joint supervision optimization. Among them, the photometric loss of the real sparse view observation, the self-reference pseudo-supervision loss, and the generation prior pseudo-supervision loss are all effective independently and do not overlap with each other.

[0014] Furthermore, the process of dynamically adjusting the fusion weights, the angle of the reshoot view, and the annotation of regional uncertainties includes, Step S501: For each spatial region, compare the self-reference prior with the generated prior to obtain the consistency between the pseudo-supervision results. Based on the consistency between the pseudo-supervision results, calculate the uncertainty of each region and use the residual between the rendering result of the region with high uncertainty and any available real observation as a supplementary measure of uncertainty. Step S502: For areas that are determined to be highly uncertain and have the conditions for supplementary acquisition on site, calculate the angle of the supplementary shooting perspective that can maximize the observation information gain based on the current geometric estimation of the area, and re-input the new image obtained by the actual supplementary shooting as the real observation into steps S1 and S4 for reprocessing. Step S503: For areas with high uncertainty but no on-site supplementary acquisition conditions, reduce the weight of the generated prior pseudo-supervision in the total loss, increase the uncertainty label of the reconstruction result in the area, identify the area, output the dynamically adjusted fusion weight for the next round of training, and output the angle and area uncertainty label of the supplementary shooting perspective.

[0015] Furthermore, based on the uncertainty of the degree of consistency among pseudo-supervision results in different regions, among which, If the consistency between the pseudo-supervision results is greater than or equal to the preset supervision consistency threshold, then the region is determined to be low uncertainty. If the consistency between the pseudo-supervision results is less than the preset supervision consistency threshold, then the region is determined to be of low uncertainty.

[0016] Furthermore, the process of outputting the reconstruction quality assessment report and the updated style adaptation parameter library includes, Step S601: Statistically analyze the overall reconstruction reliability distribution of the reconstruction results according to regional uncertainty labels to form a reconstruction quality assessment report for this task; Step S602: Archive the adaptation parameters obtained from this task training into the style adaptation parameter library according to the architectural style category; Step S603: When encountering a target building of the same style category in the future, the existing adaptation parameters of the corresponding style category in the style adaptation parameter library can be directly used as the starting point for fine-tuning training.

[0017] Compared with the prior art, the beneficial effect of the present invention is that, through building self-similar structure detection and self-reference relationship graph, the sparse view-uncovered area can preferentially obtain prior support from the symmetry and repetitive structural rules of the building itself. Compared with simply relying on general generation priors, the reconstruction result has a higher consistency with the actual form of the target building.

[0018] Furthermore, this invention generates prior fine-tuning through small-sample domain adaptive generation, using small-sample field data that is much smaller than the data scale required by typical feedforward generalization models. This enables the general prior to be specifically adapted to the target architectural style, alleviating the domain difference problem between the training data distribution and the actual distribution of the target ancient buildings in existing methods.

[0019] Furthermore, this invention guides the direct acquisition of real observations to eliminate uncertainty when on-site conditions permit, through confidence fusion of uncertainty perception and supplementary shooting perspective suggestions. When supplementary shooting conditions are not available, low-confidence areas are clearly marked to avoid presenting fabricated content with the same confidence level as real observations.

[0020] Furthermore, through a cross-building prior knowledge incremental accumulation mechanism, this invention enables the reuse of existing adaptive knowledge in subsequent reconstruction tasks of ancient buildings of the same style, thereby gradually reducing the amount of data and convergence time required for domain adaptation in tasks of the same style. Attached Figure Description

[0021] Figure 1 This is an overall flowchart of the ancient building reconstruction method based on structural prior and small sample domain adaptation according to an embodiment of the present invention; Figure 2 A flowchart for outputting the initial camera pose, sparse point cloud, and initial Gaussian set in an embodiment of the present invention; Figure 3 This is a flowchart for outputting symmetry plane parameters and self-reference relationship diagrams in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0023] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0024] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0025] Please see Figure 1 The diagram shown is an overall flowchart of the ancient building reconstruction method based on structural prior and small sample domain adaptation according to an embodiment of the present invention. This embodiment of the present invention provides an ancient building reconstruction method based on structural prior and small sample domain adaptation, comprising: Step S1: Acquire small sample sparse view images of the target building, complete preliminary pose recovery and sparse geometry reconstruction, generate a small sample sparse view image sequence as input, and output preliminary camera pose, sparse point cloud and initial Gaussian set. Specifically, it provides basic geometric and pose references for subsequent self-similar structure detection and prior fusion, while clearly identifying the areas not directly covered by sparse viewpoints.

[0026] Step S2: Detect the symmetry planes and repeating component groups contained in the target building, construct a structural relationship graph for self-reference completion, input the preliminary camera pose, the sparse point cloud and the initial Gaussian set into the structural relationship graph, and output the symmetry plane parameters and the self-reference relationship graph; Specifically, this provides a structured basis for using the reconstruction results of the fully observed parts of the building as a priori source for the unobserved parts. Compared with general generative priors, this type of prior is more in line with the actual form of the target building itself.

[0027] Step S3: Using the small sample sparse view image and its symmetric augmented version as fine-tuning data, perform lightweight domain adaptive fine-tuning to generate a prior model on the general pre-trained prior model so that the prior model can be adapted to the target building. Input the general pre-trained image generation or monocular depth prior model, the small sample sparse view image and the symmetry plane parameters into the prior model, and output the domain adaptation generation prior and adaptation parameters. Specifically, by using small on-site sample data, which is much smaller than the training data required by typical feedforward generalization models, a general prior is generated to specifically adapt to the target architectural style, thus alleviating the domain difference problem between the distribution of training data and the actual distribution of target ancient buildings in existing methods.

[0028] Step S4: Combine the self-reference graph and the domain adaptation generation prior to provide corresponding pseudo-supervision signals for regions not covered by different types of sparse views. The inputs are the initial Gaussian set, the self-reference graph and the domain adaptation generation prior. The output is the Gaussian parameters after joint supervision optimization. Specifically, this method enables regions not directly observed by sparse perspectives to preferentially obtain self-referenced prior support from the structural regularities of the building itself, and only degenerates into dependent domain adaptation generation priors when no self-references are available. Compared with existing methods that simply rely on generation priors, this method improves the consistency between the reconstruction results of unobserved regions and the actual form of the target building.

[0029] Step S5: Dynamically adjust the fusion weights of the self-reference prior and the generated prior based on the uncertainty of the reconstruction results of each region, and adjust the reshooting perspective. Generate rendering residuals based on the Gaussian parameters optimized by joint supervision. Input the pseudo-supervision source labels, rendering residuals and the flags indicating whether the site has the conditions for supplementary acquisition for each region. Output the dynamically adjusted fusion weights, the angle of the reshooting perspective and the regional uncertainty labels. Specifically, the credibility of the two types of priors is continuously verified and adjusted during the training process using a closed-loop feedback method, and supplementary on-site observations are guided as much as possible to directly eliminate uncertainty rather than relying solely on prior speculation.

[0030] Step S6: After the reconstruction converges, evaluate the reconstruction results and deposit the style adaptation knowledge increment obtained from this domain adaptive fine-tuning. Input the converged reconstruction results, the adaptation parameters and the target architectural style category label, and output the reconstruction quality assessment report and the updated style adaptation parameter library.

[0031] Specifically, this invention enables the ability to continuously reduce cross-domain differences between ancient buildings of the same style as tasks accumulate, forming an incremental accumulation mechanism that expands and reuses from a single task to a group of tasks of the same style.

[0032] Please see Figure 2 The diagram shows a flowchart illustrating the process of outputting the initial camera pose, sparse point cloud, and initial Gaussian set according to an embodiment of the present invention. The process of outputting the initial camera pose, sparse point cloud, and initial Gaussian set includes... Step S101: Perform motion reconstruction structure solving on the small sample sparse view image to obtain sparse point cloud, and use a preset matching threshold and local feature descriptor type in the feature matching stage. Step S102: Perform preliminary filtering on the sparse point cloud obtained by the solution to remove geometric outliers and generate a preliminary sparse point cloud; Step S103: Using the initial sparse point cloud combined with the pose, initialize the position, opacity and initial scale of the three-dimensional Gaussian set, and output the initial camera pose, sparse point cloud and initial Gaussian set. In particular, Gaussian sparse points are not generated in the spatial regions not covered by the sparse point cloud.

[0033] Please see Figure 3 The diagram shown is a flowchart illustrating the output of symmetry plane parameters and a self-reference graph according to an embodiment of the present invention. The process of outputting the symmetry plane parameters and the self-reference graph includes... Step S201: Perform symmetry detection on the input preliminary camera pose, sparse point cloud and initial Gaussian set. By comparing the mirror matching degree of the point cloud distribution on both sides of the candidate symmetry plane, identify several symmetry planes of the target building as a whole or in part, and output symmetry plane parameters based on the symmetry planes. Step S202: Perform similarity clustering on the local geometry of the target building, identify the repeating component groups with repeated arrangement of components, record several spatial transformation relationships within each repeating component group, and output the repeating component group; Step S203: Based on the detection results of the symmetry plane and the repeating component group, mark each spatial region in the target building to indicate whether there is a symmetric or repeating correspondence that can be used for self-reference. Based on the observation degree of the spatial region, record the correspondence of the spatial region and the corresponding spatial transformation parameters, and construct and output the self-reference relationship diagram.

[0034] Specifically, based on the observation level of the spatial region, the correspondence between the spatial regions and the corresponding spatial transformation parameters are recorded, wherein... If the spatial region has been observed from a sparse perspective on the other side of the symmetry plane or in the remaining instances of the repeating component group to which it belongs, then record the correspondence and the corresponding spatial transformation parameters.

[0035] Specifically, the process of generating prior and adaptation parameters for output domain adaptation includes, Step S301: Using the symmetry plane as a reference, perform a mirror transformation on the sparse view image to generate an augmented image, and combine the sparse view image and the augmented image to form a small-scale domain dataset. Step S302: Introduce lightweight adaptation parameters into the general pre-trained prior model and perform fine-tuning training. Keep the main parameters of the general pre-trained prior model frozen and drive the adaptation parameters to converge with the small-scale domain dataset to generate the prior model. Step S303: Verify the validity of the fine-tuned adaptation parameters to output the domain adaptation prior and the adaptation parameters.

[0036] Specifically, the process of validating the adaptation parameters after fine-tuning includes, The validation images were used to compare the consistency between the prior model's generated results for local areas of the target building before and after fine-tuning training and the actual observation results. If the degree of consistency is greater than the preset degree threshold, then the consistency is determined to be improved and the fine-tuning is effective; If the degree of consistency is less than or equal to a preset threshold, it is determined that the fine-tuning hyperparameters should be adjusted and the fine-tuning training should be re-executed.

[0037] Specifically, the peak signal-to-noise ratio (PSNR) of the results generated by the prior model before and after fine-tuning training on the validation image is used as a measure of consistency with the actual observation.

[0038] Specifically, a preset degree threshold is set to compare the consistency between the prior model's generated results for the local area of ​​the target building and the actual observation results before and after fine-tuning training. The preset degree threshold is the product of a preset degree threshold reference value and a degree deviation factor. The preset degree threshold reference value is the average of the preset degree thresholds for the same target building volume in historical data. The degree deviation factor can be set by those skilled in the art based on the accuracy requirements of the consistency judgment of the prior model before and after fine-tuning training. The higher the accuracy requirement, the larger the value should be. The value range can be [1.0, 1.1], preferably 1.05.

[0039] Specifically, the process of outputting the Gaussian parameters optimized by joint supervision includes: Step S401: For the unobserved areas marked as having corresponding relationships in the self-reference relationship diagram, according to the corresponding spatial transformation parameters, the Gaussian parameter transformation of the corresponding area of ​​the fully observed part of the target building itself is mapped to the unobserved area, which serves as the self-reference pseudo-supervision of the area. Step S402: For regions that have no corresponding relationship and are not directly observed by sparse viewpoints, generate pseudo-observation images or depth estimates for the regions using the domain adaptation generation prior, as pseudo-supervision for the generation prior of the regions. Step S403: Construct the photometric loss of real sparse view observation, the self-reference pseudo-supervision loss and the generated prior pseudo-supervision loss according to their respective applicable regions, so as to jointly drive the joint optimization of Gaussian parameters and output the Gaussian parameters after joint supervision optimization. Among them, the photometric loss of the real sparse view observation, the self-reference pseudo-supervision loss, and the generation prior pseudo-supervision loss are all effective independently and do not overlap with each other.

[0040] Specifically, the process of dynamically adjusting the fusion weights, the angle of the reshoot view, and the annotation of regional uncertainties includes, Step S501: For each spatial region, compare the self-reference prior with the generated prior to obtain the consistency between the pseudo-supervision results. Based on the consistency between the pseudo-supervision results, calculate the uncertainty of each region and use the residual between the rendering result of the region with high uncertainty and any available real observation as a supplementary measure of uncertainty. Step S502: For areas that are determined to be highly uncertain and have the conditions for supplementary acquisition on site, calculate the angle of the supplementary shooting perspective that can maximize the observation information gain based on the current geometric estimation of the area, and re-input the new image obtained by the actual supplementary shooting as the real observation into steps S1 and S4 for reprocessing. Step S503: For areas with high uncertainty but no on-site supplementary acquisition conditions, reduce the weight of the generated prior pseudo-supervision in the total loss, increase the uncertainty label of the reconstruction result in the area, identify the area, output the dynamically adjusted fusion weight for the next round of training, and output the angle and area uncertainty label of the supplementary shooting perspective.

[0041] Specifically, the uncertainty in the degree of consistency between pseudo-supervision results across different regions, where... If the consistency between the pseudo-supervision results is greater than or equal to the preset supervision consistency threshold, then the region is determined to be low uncertainty. If the consistency between the pseudo-supervision results is less than the preset supervision consistency threshold, then the region is determined to be highly uncertain.

[0042] Specifically, a preset supervision consistency threshold is set to compare the consistency between the pseudo-supervision results given by the self-reference prior and the generated prior for each spatial region. The preset supervision consistency threshold is the product of the preset supervision consistency threshold reference value and the supervision bias factor. The preset supervision consistency threshold reference value is the average value of the preset supervision consistency thresholds for the same spatial region volume in historical data. The supervision bias factor can be set by those skilled in the art based on the accuracy requirements of the consistency determination between the pseudo-supervision results given by the self-reference prior and the generated prior. The higher the accuracy requirement, the larger the value should be. The value range can be [1.0, 1.1], and preferably, it can be 1.05.

[0043] Specifically, the process of outputting the reconstruction quality assessment report and the updated style adaptation parameter library includes, Step S601: Statistically analyze the overall reconstruction reliability distribution of the reconstruction results according to regional uncertainty labels to form a reconstruction quality assessment report for this task; Step S602: Archive the adaptation parameters obtained from this task training into the style adaptation parameter library according to the architectural style category; Step S603: When encountering a target building of the same style category in the future, the existing adaptation parameters of the corresponding style category in the style adaptation parameter library can be directly used as the starting point for fine-tuning training.

[0044] Specifically, this invention uses building self-similar structure detection and self-reference relationship graphs to enable areas not covered by sparse perspectives to preferentially obtain prior support from the symmetry and repetitive structural rules of the building itself. Compared with simply relying on general generation priors, the reconstruction results have higher consistency with the actual form of the target building.

[0045] Specifically, this invention generates prior fine-tuning through small-sample domain adaptive generation. With a small sample size of field data that is much smaller than that required by typical feedforward generalization models, the general prior is generated to be specifically adapted to the target architectural style, thus alleviating the domain difference problem between the distribution of training data and the actual distribution of target ancient buildings in existing methods.

[0046] Specifically, this invention guides the direct acquisition of real observations to eliminate uncertainty when on-site conditions permit, through confidence fusion of uncertainty perception and supplementary shooting perspective suggestions. When supplementary shooting conditions are not available, low-confidence areas are clearly marked to avoid presenting fabricated content with the same confidence level as real observations.

[0047] Specifically, this invention utilizes a cross-building prior knowledge incremental accumulation mechanism to enable the reuse of existing adaptive knowledge in subsequent reconstruction tasks of ancient buildings of the same style, thereby gradually reducing the amount of data and convergence time required for domain adaptation in tasks of the same style.

[0048] In a specific embodiment, let the building's symmetry plane be Π. If an unobserved region x in the self-reference graph has a corresponding observed region x′ on the other side of Π, then the two satisfy the spatial transformation relationship x equals Π(x′). The corresponding Gaussian parameters (except for attributes such as normal and color that need to be transformed according to the mirror relationship) can be directly obtained from the optimized Gaussian parameters at x′, denoted as the self-reference pseudo-supervision G_self(x). For regions without self-reference correspondence, the domain adaptation generation prior provides pseudo-supervision G_gen(x). The adaptation parameter θ_adapt of this prior model is obtained by minimizing the reconstruction loss of the generated result and the domain dataset with a small sample domain dataset {I_1,…,I_k} (including the original and symmetric augmented images) based on the frozen pre-trained main parameters θ_base. That is, θ_adapt is equal to the result of minimizing the training loss with respect to the adaptation parameters. θ_base remains unchanged during training. The total loss of joint supervision can be expressed as a piecewise loss summed over the entire region: the actual observation region uses standard photometric loss, the self-reference available region uses a combination of Gaussian parameter transformation consistency loss and photometric loss, and the generated prior available region uses generation prior pseudo-supervision loss reduced by uncertainty weight κ(x), where κ(x) is jointly determined by the consistency between the self-reference and the generated prior and the availability of actual field observations. The calculation of the supplementary shooting viewpoint suggestion can be expressed as follows: for high uncertainty regions, search in the candidate shooting pose space for the pose that maximizes the expected observation information gain for that region. The information gain is approximately measured by the degree to which the candidate pose's view frustum covers the region and the angle between the line of sight and the currently known normal is moderate (avoiding grazing angle observations). The pseudocode is described as follows: Perform motion restoration structure calculation on sparse viewpoint images and initialize Gaussians; detect building symmetry planes and repeating component groups, and construct a self-reference relationship graph; perform lightweight adaptation parameter fine-tuning on the general generated prior using symmetry augmented images; construct corresponding losses for each region based on whether there are real observations, self-reference correspondences, or only generated priors are available, and jointly optimize Gaussian parameters; continuously evaluate the uncertainty of each region, output reshoot suggestions when on-site conditions permit, and re-execute the aforementioned process after obtaining new observations; when reshoot conditions are not available, dynamically adjust the generated prior weights according to uncertainty and label low-confidence regions; after convergence, archive the adaptation parameters of this time to the style parameter library for reuse in subsequent similar style tasks.

[0049] The confidence fusion closed loop set up in this invention runs through steps S4 and S5: the consistency between the rendering residual generated by joint supervised optimization and the two types of priors drives the uncertainty assessment. The assessment result, in turn, adjusts the prior fusion weights of each region in the next round of training, and triggers reshoots when possible to directly obtain real observations to replace prior speculations. This cycle continues until the uncertainty converges or the on-site acquisition conditions are exhausted. This closed loop, together with the knowledge accumulation mechanism in step S6, constitutes the complete technical solution of this invention at two levels: dynamic optimization within a single task and knowledge reuse across tasks.

[0050] In one specific embodiment, this embodiment takes the on-site emergency data collection task of a single building, the main hall of a traditional wooden courtyard house, as the object. Due to the limitations of the on-site conditions, only fourteen sparse perspective images were obtained, and the complete coverage of the building's rear facade could not be achieved. This illustrates the specific implementation process of the method of the present invention.

[0051] Step S1: Perform motion reconstruction on the fourteen images. Use a relaxed matching threshold to improve the matching success rate under limited viewpoints. Finally, obtain thirteen usable camera poses (one failed to be solved due to severe occlusion) and about 86,000 sparse point clouds. About 40% of the space of the rear facade of the main hall was not directly observed from any viewpoint.

[0052] Step S2 involves performing a symmetry test on the initially reconstructed geometry. This identifies the overall layout of the main hall with the central axis of the main bay as the plane of symmetry, achieving a symmetry detection confidence level of 0.91. Simultaneously, it identifies the eaves brackets as a repeating component group, with eight out of twelve bracket sets already covered by sparse perspective observations. Based on the detection results, a self-reference map is constructed. Approximately 60% of the unobserved area on the rear facade can find its self-reference source in the corresponding area on the front facade through the plane of symmetry. The remaining approximately 40% (mainly parts of the gable wall) lacks a self-reference correspondence.

[0053] Step S3 involves constructing a domain fine-tuning dataset using fourteen original images and twenty-six mirror-enhanced images generated based on detected symmetry planes. This dataset is then used to fine-tune the lightweight adaptation parameters introduced into the general pre-trained generation prior model for three hundred rounds of training. Two reserved validation images are used to compare the generated results before and after fine-tuning. The structural similarity index between the generated results after fine-tuning and the actual observed structures is improved, validating the effectiveness of the fine-tuning.

[0054] Step S4: For the self-referenceable region of the rear facade, the optimized Gaussian parameter mirror mapping of the front facade is used as a self-reference pseudo-supervision based on the symmetry plane transformation parameters; for the region of the gable wall without a self-reference source, the pseudo-observation image generated by domain adaptation is used as pseudo-supervision for that region. The real observation photometric loss, self-reference consistency loss, and generated prior pseudo-supervision loss participate in joint optimization in their respective applicable regions.

[0055] In step S5, during the training process, it was detected that the prior pseudo-supervision results generated in a local area of ​​the gable wall fluctuated greatly between different training rounds, and it was determined to be a high uncertainty area. Since the on-site workers were still present on the day and the conditions for reshooting were available, the system calculated and output two suggested reshooting angles for this area. After on-site reshooting, two new images were obtained, and after being reintroduced into the training, the uncertainty of this area was significantly reduced. In the repeating group of brackets, the four unobserved dougong were not reshootable because the scaffolding had been removed from the site. The system reduced the prior weight of this area to 40% of the normal weight and clearly marked the details of these four dougong in the results annotation as prior speculation results with low confidence.

[0056] Step S6: After training convergence, a statistical reconstruction quality assessment report is generated. The overall confidence level of the real observation and self-reference support areas is high, while areas marked as low confidence account for approximately 6% of the total building surface area. The adaptation parameters obtained in this fine-tuning are archived into the style adaptation parameter library according to the traditional wooden architectural style of the region. This style category has previously accumulated adaptation parameters for two similar tasks, and the results of this iteration are merged and updated with the existing parameters. When conducting similar small-sample reconstruction tasks on another ancient building of the same style, the corresponding entries in the updated parameter library can be used directly as the starting point for fine-tuning. The specific thresholds, weights, and training rounds involved in the embodiment are preferred reference ranges set in conjunction with internal test data. In actual implementation, they can be adjusted within the range given in this specification according to the on-site collection conditions and architectural style.

[0057] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for reconstructing ancient buildings based on structural priors and small-sample domain adaptation, characterized in that, include: Step S1: Acquire small sample sparse view images of the target building, complete preliminary pose recovery and sparse geometry reconstruction, generate a small sample sparse view image sequence as input, and output preliminary camera pose, sparse point cloud and initial Gaussian set. Step S2: Detect the symmetry planes and repeating component groups contained in the target building, construct a structural relationship graph for self-reference completion, input the preliminary camera pose, the sparse point cloud and the initial Gaussian set into the structural relationship graph, and output the symmetry plane parameters and the self-reference relationship graph; Step S3: Using the small sample sparse view image and its symmetric augmented version as fine-tuning data, perform lightweight domain adaptive fine-tuning to generate a prior model on the general pre-trained prior model so that the prior model can be adapted to the target building. Input the general pre-trained image generation or monocular depth prior model, the small sample sparse view image and the symmetry plane parameters into the prior model, and output the domain adaptation generation prior and adaptation parameters. Step S4: Combine the self-reference graph and the domain adaptation generation prior to provide corresponding pseudo-supervision signals for regions not covered by different types of sparse views. The inputs are the initial Gaussian set, the self-reference graph and the domain adaptation generation prior. The output is the Gaussian parameters after joint supervision optimization. Step S5: Dynamically adjust the fusion weights of the self-reference prior and the generated prior based on the uncertainty of the reconstruction results of each region, and adjust the reshooting perspective. Generate rendering residuals based on the Gaussian parameters optimized by joint supervision. Input the pseudo-supervision source labels, rendering residuals and the flags indicating whether the site has the conditions for supplementary acquisition for each region. Output the dynamically adjusted fusion weights, the angle of the reshooting perspective and the regional uncertainty labels. Step S6: After the reconstruction converges, evaluate the reconstruction results and deposit the style adaptation knowledge increment obtained from this domain adaptive fine-tuning. Input the converged reconstruction results, the adaptation parameters and the target architectural style category label, and output the reconstruction quality assessment report and the updated style adaptation parameter library.

2. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 1, characterized in that, The process of outputting the initial camera pose, sparse point cloud, and initial Gaussian set includes: Step S101: Perform motion reconstruction structure solving on the small sample sparse view image to obtain sparse point cloud, and use a preset matching threshold and local feature descriptor type in the feature matching stage. Step S102: Perform preliminary filtering on the sparse point cloud obtained by the solution to remove geometric outliers and generate a preliminary sparse point cloud; Step S103: Using the initial sparse point cloud combined with the pose, initialize the position, opacity and initial scale of the three-dimensional Gaussian set, and output the initial camera pose, sparse point cloud and initial Gaussian set. In particular, Gaussian sparse points are not generated in the spatial regions not covered by the sparse point cloud.

3. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 2, characterized in that, The process of outputting the symmetry plane parameters and the self-reference relation diagram includes, Step S201: Perform symmetry detection on the input preliminary camera pose, sparse point cloud and initial Gaussian set. By comparing the mirror matching degree of the point cloud distribution on both sides of the candidate symmetry plane, identify several symmetry planes of the target building as a whole or in part, and output symmetry plane parameters based on the symmetry planes. Step S202: Perform similarity clustering on the local geometry of the target building, identify the repeating component groups with repeated arrangement of components, record several spatial transformation relationships within each repeating component group, and output the repeating component group; Step S203: Based on the detection results of the symmetry plane and the repeating component group, mark each spatial region in the target building to indicate whether there is a symmetric or repeating correspondence that can be used for self-reference. Based on the observation degree of the spatial region, record the correspondence of the spatial region and the corresponding spatial transformation parameters, and construct and output the self-reference relationship diagram.

4. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 3, characterized in that, Based on the observation level of the spatial region, the correspondence between the spatial regions and the corresponding spatial transformation parameters are recorded, wherein... If the spatial region has been observed from a sparse perspective on the other side of the symmetry plane or in the remaining instances of the repeating component group to which it belongs, then record the correspondence and the corresponding spatial transformation parameters.

5. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 4, characterized in that, The process of generating prior and adaptation parameters for output domain adaptation includes: Step S301: Using the symmetry plane as a reference, perform a mirror transformation on the sparse view image to generate an augmented image, and combine the sparse view image and the augmented image to form a small-scale domain dataset. Step S302: Introduce lightweight adaptation parameters into the general pre-trained prior model and perform fine-tuning training. Keep the main parameters of the general pre-trained prior model frozen and drive the adaptation parameters to converge with the small-scale domain dataset to generate the prior model. Step S303: Verify the validity of the fine-tuned adaptation parameters to output the domain adaptation prior and the adaptation parameters.

6. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 5, characterized in that, The process of validating the adaptation parameters after fine-tuning training includes: The validation images were used to compare the consistency between the prior model's generated results for local areas of the target building before and after fine-tuning training and the actual observation results. If the degree of consistency is greater than the preset degree threshold, then the consistency is determined to be improved and the fine-tuning is effective; If the degree of consistency is less than or equal to a preset threshold, it is determined that the fine-tuning hyperparameters should be adjusted and the fine-tuning training should be re-executed.

7. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 6, characterized in that, The process of outputting the Gaussian parameters optimized by joint supervision includes: Step S401: For the unobserved areas marked as having corresponding relationships in the self-reference relationship diagram, according to the corresponding spatial transformation parameters, the Gaussian parameter transformation of the corresponding area of ​​the fully observed part of the target building itself is mapped to the unobserved area, which serves as the self-reference pseudo-supervision of the area. Step S402: For regions that have no corresponding relationship and are not directly observed by sparse viewpoints, generate pseudo-observation images or depth estimates for the regions using the domain adaptation generation prior, as pseudo-supervision for the generation prior of the regions. Step S403: Construct the photometric loss of real sparse view observation, the self-reference pseudo-supervision loss and the generated prior pseudo-supervision loss according to their respective applicable regions, so as to jointly drive the joint optimization of Gaussian parameters and output the Gaussian parameters after joint supervision optimization. Among them, the photometric loss of the real sparse view observation, the self-reference pseudo-supervision loss, and the generation prior pseudo-supervision loss are all effective independently and do not overlap with each other.

8. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 7, characterized in that, The process of dynamically adjusting the fusion weights, the angle of the reshoot view, and the annotation of regional uncertainties in the output includes... Step S501: For each spatial region, compare the self-reference prior with the generated prior to obtain the consistency between the pseudo-supervision results. Based on the consistency between the pseudo-supervision results, calculate the uncertainty of each region and use the residual between the rendering result of the region with high uncertainty and any available real observation as a supplementary measure of uncertainty. Step S502: For areas that are determined to be highly uncertain and have the conditions for supplementary acquisition on site, calculate the angle of the supplementary shooting perspective that can maximize the observation information gain based on the current geometric estimation of the area, and re-input the new image obtained by the actual supplementary shooting as the real observation into steps S1 and S4 for reprocessing. Step S503: For areas with high uncertainty but no on-site supplementary acquisition conditions, reduce the weight of the generated prior pseudo-supervision in the total loss, increase the uncertainty label of the reconstruction result in the area, identify the area, output the dynamically adjusted fusion weight for the next round of training, and output the angle and area uncertainty label of the supplementary shooting perspective.

9. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 8, characterized in that, The uncertainty in the degree of consistency among pseudo-supervision results across different regions, among which... If the consistency between the pseudo-supervision results is greater than or equal to the preset supervision consistency threshold, then the region is determined to be low uncertainty. If the consistency between the pseudo-supervision results is less than the preset supervision consistency threshold, then the region is determined to be of low uncertainty.

10. The ancient building reconstruction method based on structural prior and small sample domain adaptation according to claim 9, characterized in that, The process of outputting the reconstruction quality assessment report and the updated style adaptation parameter library includes: Step S601: Statistically analyze the overall reconstruction reliability distribution of the reconstruction results according to regional uncertainty labels to form a reconstruction quality assessment report for this task; Step S602: Archive the adaptation parameters obtained from this task training into the style adaptation parameter library according to the architectural style category; Step S603: When encountering a target building of the same style category in the future, the existing adaptation parameters of the corresponding style category in the style adaptation parameter library can be directly used as the starting point for fine-tuning training.

Images