A method for automatic identification of cultural heritage landscape genes and dynamic pedigree construction

Abstract

This application discloses an automated method for identifying cultural heritage landscape genes and constructing dynamic lineages, relating to the field of cultural heritage digitization technology. This method maps multimodal data of cultural heritage landscapes to a unified semantic space by constructing a multimodal feature encoder; collects and spatiotemporally registers target multimodal data, filters it through multi-dimensional evaluation, extracts features using a multi-scale feature pyramid network, and identifies cultural heritage landscape genes and their connections through a hierarchical classification architecture; constructs gene codes, associates and stores them, and establishes a gene knowledge base, thereby constructing a temporal knowledge graph to complete automated identification and dynamic lineage construction. This application solves the problems of low efficiency, inconsistent results, and susceptibility to misidentification in traditional methods, achieving efficient and accurate identification and dynamic lineage construction of cultural heritage landscape genes, providing strong support for the protection, research, and revitalization of cultural heritage.

Description

Technical Field

[0001] This application relates to the field of cultural heritage digitization technology, specifically to a method for automated identification and dynamic lineage construction of cultural heritage landscape genes. Background Technology

[0002] Cultural landscape heritage refers to rare and irreplaceable cultural landscapes recognized by UNESCO and the World Heritage Committee. These are universally acknowledged works of nature and humanity, possessing outstanding significance and universal value. Identifying the genes of existing cultural heritage landscapes primarily relies on professionals using tools such as total stations, measuring tapes, and plumb lines to conduct on-site manual measurements, recording the dimensions, orientation, and component relationships of buildings. Simultaneously, non-geometric information such as decorative patterns and colors is recorded through hand-drawing and photography. Alternatively, researchers consult local chronicles, historical archives, family genealogies, and conduct oral history interviews with local artisans and residents to obtain intangible information such as the construction methods, cultural taboos, and historical evolution of the cultural heritage. Or, experts in the field (such as architectural historians and archaeologists) conduct manual analysis and summarization based on measurement data and oral records, leveraging their own knowledge. They summarized the typical characteristics of this cultural heritage, such as: hip roof, raised beam frame, six-rafter eaves with two columns, and painted decorations, and attempted to establish a classification and coding system (such as using a linear classification method to divide the roof into primary categories, and the hip roof, gable roof, etc., into secondary categories).

[0003] Therefore, it can be determined that existing technologies mainly rely on human labor, with technical tools serving only as auxiliary means of measurement and recording. This is a highly subjective and non-standardized process, which leads to problems such as low efficiency, inconsistent identification results, and / or easy misidentification in the process of identifying the genes of cultural heritage landscapes using existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide an automated method for identifying cultural heritage landscape genes and constructing dynamic lineages, which solves the problems of low efficiency, inconsistent identification results and / or easy misidentification in existing technologies.

[0005] This application is achieved through the following technical solution:

[0006] A method for automated identification and dynamic lineage construction of cultural heritage landscape genes, comprising:

[0007] A multimodal feature encoder is constructed, and the existing multimodal data in the training set corresponding to the cultural heritage landscape is mapped to a unified semantic space using the multimodal feature encoder.

[0008] Collect target multimodal data corresponding to the cultural heritage landscape to be classified, and perform spatiotemporal registration on the target multimodal data to obtain spatiotemporally registered target multimodal data;

[0009] The processed target multimodal data is evaluated from multiple dimensions using semantic features in the semantic space and a multimodal feature encoder to obtain multidimensional evaluation results. The processed target multimodal data is then filtered based on the multidimensional evaluation results to obtain filtered target multimodal data.

[0010] A multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to the multimodal features are extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connection relationships between cultural heritage landscape genes.

[0011] Gene codes are constructed, and the gene codes, cultural heritage landscape genes, and the connections between cultural heritage landscape genes are associated and stored. A gene knowledge base is established, and a temporal knowledge graph is constructed based on the gene knowledge base to complete the automated identification and dynamic lineage construction of cultural heritage landscape genes.

[0012] In one possible implementation, after constructing the multimodal feature encoder, the following is also included:

[0013] A dual-modal coding query architecture is constructed using a multimodal feature encoder;

[0014] A dual-modal coding query architecture is used to match the user-input query modality data with the target modality data in the existing multimodal data, and the retrieved target modality data is returned to the user. The query modality refers to the modality data input by the user as a retrieval condition, and the target modality data refers to the modality data required by the user.

[0015] In one possible implementation, a multimodal feature encoder is constructed, and the multimodal feature encoding is used to map existing multimodal data in the training set corresponding to cultural heritage landscapes to a unified semantic space, including:

[0016] A visual encoder for encoding visual modal data is constructed using Vision Transformer; a point cloud encoder for encoding point cloud modal data is constructed using Point Transformer; and a text encoder for encoding text modal data is constructed using BERT.

[0017] The visual encoder, point cloud encoder, and text encoder are used together as a multimodal feature encoder, and the semantic features encoded by the visual encoder, point cloud encoder, and text encoder have the same dimension.

[0018] A visual encoder is used to encode and map existing visual modal data, a point cloud encoder is used to encode and map existing point cloud modal data, and a text encoder is used to encode and map existing text modal data, forming a unified semantic space composed of multimodal semantic features. The multimodal feature encoding includes existing multimodal data in the training set corresponding to the cultural heritage landscape, including existing visual modal data, existing point cloud modal data, and existing multimodal data corresponding to the same cultural heritage landscape gene.

[0019] In one possible implementation, target multimodal data corresponding to the cultural heritage landscape to be classified is collected, and the target multimodal data is spatiotemporally registered to obtain spatiotemporally registered target multimodal data, including:

[0020] Collect target point cloud data, target image data, and historical documents corresponding to the cultural heritage landscape to be classified to obtain target multimodal data;

[0021] Spatial registration is performed on the target point cloud data and the target image data, and a dynamic time warping algorithm is used to align the data at different acquisition time points to obtain the spatiotemporally registered target point cloud data and the target image data.

[0022] The historical documents are preprocessed and subjected to text recognition processing to obtain digitized historical documents.

[0023] The digitized historical documents, the spatiotemporally registered target point cloud data, and the target image data are collectively used as the spatiotemporally registered target multimodal data.

[0024] In one possible implementation, after spatially registering the target point cloud data and the target image data, the method further includes:

[0025] Anomaly detection is performed on the spatially registered target point cloud data and target image data to identify abnormal data regions, and anomaly masks are generated in the abnormal data regions to provide anomaly alerts.

[0026] In one possible implementation, semantic features in the semantic space and a multimodal feature encoder are used to perform multidimensional evaluation on the processed target multimodal data to obtain multidimensional evaluation results. The processed target multimodal data is then filtered based on these multidimensional evaluation results to obtain filtered target multimodal data, including:

[0027] A multimodal feature encoder is used to extract multi-target semantic features in a unified semantic space from the processed target multimodal data, and the pairwise similarity between the multi-target semantic features is calculated. If the pairwise similarity of the processed target multimodal data is greater than a preset threshold, the data quality is determined to be qualified; otherwise, the data quality is determined to be unqualified.

[0028] The cross-modal contrast loss corresponding to the multi-target semantic features is obtained using semantic features in the semantic space. When the cross-modal contrast loss meets the loss constraint condition, the data quality is determined to be qualified; otherwise, the data quality is determined to be unqualified.

[0029] The target multimodal data that does not meet the data quality standards is filtered out to obtain the filtered target multimodal data.

[0030] In one possible implementation, a multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to these multimodal features are then extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connections between these genes, including:

[0031] Based on the filtered target multimodal data, a multi-scale feature pyramid network is used to extract image features from the spatiotemporally registered target image data. At the same time, text features from the digitized historical documents are extracted. A hierarchical classification architecture is used to perform gene classification and recognition on the image features and text features to obtain hierarchical cultural heritage landscape genes.

[0032] After spatiotemporal registration, the spatial relationships between cultural heritage landscape genes are extracted from the target point cloud data. The cultural heritage landscape genes are used as graph nodes, and the spatial relationships are used as edges. Neighborhood information is aggregated through a message passing mechanism to learn the dependencies between genes and output the cultural heritage landscape genes and the connection relationships between them.

[0033] In one possible implementation, a gene code is constructed, and the gene code, cultural heritage landscape genes, and the connections between cultural heritage landscape genes are stored together to establish a gene knowledge base. A temporal knowledge graph is then constructed based on the gene knowledge base, including:

[0034] Construct gene codes, associate and store the gene codes, cultural heritage landscape genes, and the connection relationships between cultural heritage landscape genes to obtain cultural heritage landscape gene information, and use the cultural heritage landscape gene information to establish a gene knowledge base;

[0035] Based on the spatiotemporally registered target point cloud data, the spatiotemporally registered target image data, and the cultural heritage landscape gene information in the gene knowledge base, a temporal knowledge graph is constructed using a temporal knowledge graph.

[0036] In one possible implementation, the method further includes:

[0037] Based on the cultural gene sequence composed of all cultural heritage landscape gene information corresponding to the same cultural heritage landscape in historical data, the dynamic time warping algorithm is used to calculate the similarity of different cultural gene sequences and identify similar evolutionary patterns.

[0038] Based on this, a family tree of cultural genes is constructed by grouping cultural gene sequences with a common ancestor or similar developmental paths into the same lineage through a genealogical clustering algorithm.

[0039] In one possible implementation, the method further includes:

[0040] Based on the temporal knowledge graph, multiple candidate design schemes are generated;

[0041] The NSGA-II algorithm is used to iteratively update multiple candidate design schemes, solve for the Pareto optimal solution set, output the optimal candidate design scheme as the recommended scheme, and push the recommended scheme to the user.

[0042] Compared with the prior art, this application has the following advantages and beneficial effects:

[0043] This application provides a method for automated identification and dynamic lineage construction of cultural heritage landscape genes. This method constructs a multimodal feature encoder to map multimodal data of cultural heritage landscapes to a unified semantic space; collects and spatiotemporally registers target multimodal data, filters it through multi-dimensional evaluation, extracts features using a multi-scale feature pyramid network, and identifies cultural heritage landscape genes and their connections through a hierarchical classification architecture; constructs gene codes, associates and stores them, and establishes a gene knowledge base, thereby constructing a temporal knowledge graph to complete automated identification and dynamic lineage construction. This application solves the problems of low efficiency, inconsistent results, and susceptibility to misidentification in traditional methods, achieving efficient and accurate identification and dynamic lineage construction of cultural heritage landscape genes, providing strong support for the protection, research, and revitalization of cultural heritage. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0045] Figure 1 A flowchart illustrating an automated identification and dynamic lineage construction method for cultural heritage landscape genes, provided as an embodiment of this application;

[0046] Figure 2 A schematic diagram illustrating cross-modal semantic alignment and feature fusion provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of source heterogeneous data acquisition and intelligent preprocessing provided in an embodiment of this application;

[0048] Figure 4 A schematic diagram illustrating the automated identification of cultural heritage landscape genes provided in this application embodiment;

[0049] Figure 5 A schematic diagram illustrating dynamic lineage construction and evolution analysis provided in an embodiment of this application;

[0050] Figure 6 This is a schematic diagram illustrating parametric design and semantic evaluation provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0052] like Figure 1 As shown in the embodiments of this application, a method for automated identification and dynamic lineage construction of cultural heritage landscape genes is provided, including:

[0053] S101. Construct a multimodal feature encoder and use the multimodal feature encoder to map the existing multimodal data in the training set corresponding to the cultural heritage landscape to a unified semantic space;

[0054] S102. Collect target multimodal data corresponding to the cultural heritage landscape to be classified, and perform spatiotemporal registration on the target multimodal data to obtain spatiotemporally registered target multimodal data;

[0055] S103. The processed target multimodal data is evaluated from multiple dimensions using semantic features in the semantic space and a multimodal feature encoder to obtain multidimensional evaluation results. The processed target multimodal data is then filtered based on the multidimensional evaluation results to obtain filtered target multimodal data.

[0056] S104. A multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to the multimodal features are extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connection relationships between cultural heritage landscape genes.

[0057] S105. Construct gene codes, associate and store the gene codes, cultural heritage landscape genes, and the connection relationships between cultural heritage landscape genes, establish a gene knowledge base, and construct a temporal knowledge graph based on the gene knowledge base to complete the automated identification and dynamic lineage construction of cultural heritage landscape genes.

[0058] In one possible implementation, after constructing the multimodal feature encoder, the following is also included:

[0059] A dual-modal coding query architecture is constructed using a multimodal feature encoder;

[0060] A dual-modal coding query architecture is used to match the user-input query modality data with the target modality data in the existing multimodal data, and the retrieved target modality data is returned to the user. The query modality refers to the modality data input by the user as a retrieval condition, and the target modality data refers to the modality data required by the user.

[0061] In one possible implementation, a multimodal feature encoder is constructed, and the multimodal feature encoding is used to map existing multimodal data in the training set corresponding to cultural heritage landscapes to a unified semantic space, including:

[0062] S101.1 A visual encoder for encoding visual modal data is constructed using Vision Transformer; a point cloud encoder for encoding point cloud modal data is constructed using Point Transformer; and a text encoder for encoding text modal data is constructed using BERT.

[0063] S101.2. The visual encoder, point cloud encoder and text encoder are used together as a multimodal feature encoder, and the semantic features encoded by the visual encoder, point cloud encoder and text encoder have the same dimension.

[0064] S101.3. A visual encoder is used to encode and map the existing visual modal data, a point cloud encoder is used to encode and map the existing point cloud modal data, and a text encoder is used to encode and map the existing text modal data to form a unified semantic space composed of multimodal semantic features; the multimodal feature encoding includes the existing multimodal data in the training set corresponding to the cultural heritage landscape, including the existing visual modal data, existing point cloud modal data, and existing multimodal data corresponding to the same cultural heritage landscape gene.

[0065] like Figure 2 As illustrated, the first step is multimodal feature extraction, with dedicated feature extractors designed for the characteristics of different modalities. For visual modalities, a Vision Transformer-based encoder is used to segment the image into a 16×16 tile sequence, extracting global and local features through a multi-layer self-attention mechanism. Point cloud data is processed using a Point Transformer to preserve the spatial geometric relationships of the point cloud while learning feature representations of local neighborhoods. For textual data, a BERT model pre-trained on texts from the cultural heritage domain is used to better understand specialized terminology and cultural concepts.

[0066] A multilayer perceptron is used to extract features from visual modalities, point cloud data, and text data through visual mapping, point cloud mapping, and text mapping, respectively, and project them into the same vector space. After obtaining vectors in the common space, an optimizable and comparable matrix is ​​defined as a similarity metric, ensuring that the similarity between "image-correct text" positive examples is significantly higher than the similarity between "image-incorrect text" negative examples. In cross-modal semantic alignment, the core role of the similarity matrix is ​​to compare the degree of similarity between feature vectors of different modalities in a unified semantic space. Specifically, suppose we have a batch containing N image samples and N corresponding text samples (or other modalities). After mapping to the unified semantic space, each image obtains a D-dimensional feature vector. Each text yields a D-dimensional feature vector. Therefore, the similarity matrix S between images and text is an N×N matrix, where elements... This represents the semantic similarity between the i-th image and the j-th text. Semantic similarity can be obtained using cosine similarity, scaled dot product similarity, or negative Euclidean distance.

[0067] In contrastive learning training, the similarity matrix is ​​used to construct positive and negative sample pairs: diagonal elements This represents paired (image, text) samples – positive samples with the desired high similarity. Off-diagonal elements. This represents unpaired samples—negative sample pairs—with the desired low similarity. The model is then driven to learn using a symmetric contrastive loss function (such as InfoNCE). ;in, It is the temperature coefficient, which controls the smoothness of the distribution.

[0068] A dual-encoder architecture is designed to handle the query modality and the target modality separately, performing similarity calculations within a unified semantic space. The query and target modalities are dynamically specified based on the retrieval task; the query modality is the user-inputted modality used as a retrieval condition, and the target modality is the modality the system needs to return. The dual-encoder architecture equips the query and target modalities with independent feature extraction networks, but ultimately maps them to the same semantic space for comparison. A hard example mining strategy is employed to enhance the model's ability to distinguish fine-grained differences, ensuring that similar cultural genes cluster in the semantic space. Retrieval performance on the test set reaches mAP@50=87.6%, significantly outperforming traditional methods. The dual-encoder architecture (also known as a two-tower architecture) designs two independent encoders for two different modalities, mapping their respective inputs to a unified semantic feature space, and then measuring their semantic relevance through similarity calculations (such as cosine similarity or dot product). Two encoders can be selected from three types to form the dual-encoder architecture.

[0069] In terms of technical implementation, a multi-task contrastive learning framework is adopted to optimize the semantic space construction. The main loss functions include: cross-modal contrastive loss: bringing positive sample pairs closer together and pushing negative sample pairs further apart; intramodal consistency loss: preserving the semantic structure of samples within the same modality; and semantic reconstruction loss: ensuring that the semantic representation contains sufficient information.

[0070] For example, the total loss function can be set as:

[0071] ;

[0072] in, Represents the total loss function. Indicates cross-modal contrast loss. This represents the intramodal consistency loss. Represents the semantic reconstruction loss. , as well as These represent the weights of the three types of losses, respectively.

[0073] The overall loss function is applied to a Cross-Modal Semantic Alignment Network, which consists of three sub-modules: a modality-specific encoder (Vision Transformer, Point Transformer, Domain-Adaptive BERT), a cross-modal attention fusion module, and a unified semantic space projection layer. The training objective of this network is to map data from different modalities to a unified semantic space by minimizing a multi-objective joint loss function, ensuring that semantically similar cross-modal samples are close together, while semantically dissimilar samples are far apart.

[0074] Cross-modal contrastive loss is based on the InfoNCE (Noise Contrastive Estimation) framework, used to bring positive sample pairs closer together and push negative sample pairs further apart. For a batch containing N image-text-point cloud triplet samples, the image-to-text contrastive loss is defined as follows (similar for other modal pairs):

[0075] ;

[0076] Where: sim(a,b)=a⋅b (dot product, since the features have been normalized); τ is the temperature coefficient, and in this application, τ=0.1. This represents the image features obtained by the image encoder. This represents the text features output by the text encoder.

[0077] Symmetrically, calculate the text-to-image loss. And similar terms such as image-to-point-cloud, point-cloud-to-image, text-to-point-cloud, and point-cloud-to-text. The total cross-modal contrast loss is the average of the losses for all directions:

[0078] ;

[0079] in, Indicates cross-modal contrast loss. This represents the loss in image-to-text conversion. Represents the loss from text to image. This represents the loss from the image to the point cloud. This represents the loss from point cloud to image. This represents the loss from text to point cloud. This represents the loss from point cloud to text.

[0080] Intramodal consistency loss To preserve the original semantic structure among samples within the same modality and prevent semantic space distortion, a variant of the center loss is employed, calculated separately for each modality:

[0081] ;

[0082] in: This represents the central loss corresponding to a single mode. It is the feature vector of the i-th sample under the modality mod (mod∈{img(image),txt(text),pc(point cloud)}); It is the semantic category label of the sample (such as "dougong", "xieshantai", etc.); It is the first modal mod. The learnable center vector of the class is updated during training.

[0083] final: ;

[0084] in, This represents the intramodal consistency loss. This represents the center loss corresponding to the image. This represents the central loss corresponding to the text. This represents the center loss corresponding to the point cloud. Ensure that the feature distribution of the same category is more compact across different modalities.

[0085] To prevent the loss of detailed information about the original modality in the unified semantic space, a semantic reconstruction loss is introduced. A modality-specific decoder is appended after the unified semantic features to attempt to reconstruct the original input from the unified features (e.g., a Transformer-based decoder can be used for image reconstruction). The reconstruction loss is defined as:

[0086] ;

[0087] in: Indicates the reconstruction loss. It is the mean squared error loss, used for image and text reconstruction; It is the chamfer distance, used for point cloud reconstruction:

[0088] ;

[0089] The supporting role of the above loss function is as follows: It is the core component, responsible for cross-modal semantic alignment, which directly corresponds to the innovative point of "cross-modal semantic alignment" in this application. It enhances intra-class compactness and avoids semantic drift between semantically identical samples of different modalities. This demonstrates that semantic representation does not lose modality-specific information, thus improving the robustness of the representation. This application has been validated on multiple publicly available cultural heritage datasets (such as the Palace Museum Ancient Architecture Dataset and the Dunhuang Mural Multimodal Dataset). The model trained using the aforementioned loss function performs cross-modal retrieval. It reached 87.6%, which is significantly better than using only contrast loss (82.1%) or only using contrast loss + center loss (85.3%).

[0090] In one possible implementation, target multimodal data corresponding to the cultural heritage landscape to be classified is collected, and the target multimodal data is spatiotemporally registered to obtain spatiotemporally registered target multimodal data, including:

[0091] S102.1 Collect target point cloud data, target image data, and historical documents corresponding to the cultural heritage landscape to be classified to obtain target multimodal data;

[0092] S102.2 Spatial registration is performed on the target point cloud data and the target image data, and a dynamic time warping algorithm is used to align the data at different acquisition time points to obtain the spatiotemporally registered target point cloud data and the target image data.

[0093] S102.3. Preprocess the historical documents and perform text recognition processing to obtain digitized historical documents;

[0094] S102.4. The digitized historical documents, the spatiotemporally registered target point cloud data, and the target image data are used together as the spatiotemporally registered target multimodal data.

[0095] In one possible implementation, after spatially registering the target point cloud data and the target image data, the method further includes:

[0096] Anomaly detection is performed on the spatially registered target point cloud data and target image data to identify abnormal data regions, and anomaly masks are generated in the abnormal data regions to provide anomaly alerts.

[0097] In one possible implementation, semantic features in the semantic space and a multimodal feature encoder are used to perform multidimensional evaluation on the processed target multimodal data to obtain multidimensional evaluation results. The processed target multimodal data is then filtered based on these multidimensional evaluation results to obtain filtered target multimodal data, including:

[0098] S103.1. A multimodal feature encoder is used to extract multi-target semantic features in a unified semantic space from the processed target multimodal data, and the pairwise similarity between the multi-target semantic features is calculated. If the pairwise similarity of the processed target multimodal data is greater than a preset threshold, the data quality is determined to be qualified; otherwise, the data quality is determined to be unqualified.

[0099] S103.2. Obtain the cross-modal contrast loss corresponding to the multi-target semantic features using semantic features in the semantic space, and determine that the data quality is qualified when the cross-modal contrast loss meets the loss limitation condition; otherwise, determine that the data quality is unqualified.

[0100] S103.3 Filter out the target multimodal data that does not meet the data quality standards to obtain the filtered target multimodal data.

[0101] like Figure 3 As illustrated, in the digitization of cultural heritage, the standardization and intelligentization of data collection are crucial to ensuring the accuracy of subsequent analysis. This system designs differentiated data collection schemes tailored to the characteristics of different types of cultural heritage.

[0102] For large-scale architectural heritage sites, a multi-platform collaborative data collection strategy is adopted.

[0103] A ground-based 3D laser scanning system acquires detailed point cloud data of the building facade and interior space, with the point spacing controlled within 2mm to ensure the geometric accuracy of key components. A drone-based oblique photography system captures the building's top and overall shape from the air, simultaneously acquiring multi-angle images using a five-lens camera to generate a realistic 3D model with lifelike textures. The two datasets are precisely registered using an automatic feature point matching algorithm, with a registration error controlled within 5mm. By periodically collecting multispectral data at the same location, changes in color, pigment peeling rates, and other factors can be quantitatively monitored to assess the building's health status.

[0104] Regarding data quality control, a multi-dimensional evaluation system was established by combining the cross-modal contrastive learning architecture and semantic benchmarks from step S101. Geometric quality evaluation includes indicators such as point cloud density, flatness, and integrity; texture quality evaluation includes parameters such as resolution, color consistency, and illumination uniformity. For example, semantic consistency evaluation, cross-modal contrastive loss monitoring, and / or semantic benchmark detection in anomaly detection can be performed.

[0105] The principle of semantic consistency assessment is as follows: For each acquired multimodal data unit (e.g., point cloud of a building component + corresponding photograph + corresponding literature description), the cross-modal encoder in step S101 extracts the vectors of each unit in a unified semantic space, and calculates the pairwise similarity between the three. Judgment criteria: If the cosine similarity between the three is greater than a preset threshold (e.g., 0.85), the data set is semantically consistent and of acceptable quality. If the similarity between two modalities is lower than the threshold (e.g., the similarity between the point cloud and the text is only 0.3), it indicates that the data set has a modal alignment error (e.g., the point cloud and the text description are not the same component) or a data acquisition defect (e.g., incomplete point cloud leading to loss of geometric semantics). The system automatically marks data units with low similarity and generates an anomaly report, prompting for re-acquisition or correction.

[0106] The principle of cross-modal contrastive loss monitoring is as follows: After training in step S101, the parameters of the cross-modal encoder are fixed. For the newly acquired data in step S102, the cross-modal contrastive loss (InfoNCE loss) is calculated. The judgment criteria are: if the distance between positive sample pairs in the semantic space of the new data is consistent with the distribution of the training set (i.e., the loss value is within ±1σ of the mean loss of the training set), then the data quality is good. If the loss value is significantly high, it indicates that there is modal distribution drift in the new data (e.g., excessive point cloud noise, abnormal image lighting, or text description deviating from standard terminology), requiring preprocessing or re-acquisition.

[0107] The principle of semantic benchmark detection in anomaly detection is as follows: In anomaly detection algorithms (point cloud holes, texture distortion, color distortion), semantic features are introduced as an additional criterion in addition to geometric / texture features. For example, for an image, its semantic vector is first extracted by an image encoder, and then compared with the semantic vectors of historical high-quality images in the same scene. If the difference between the two exceeds a threshold, an anomaly flag is triggered. This method can detect semantic-level defects that are difficult to find using traditional geometric / texture methods (such as errors in literature descriptions or loss of cultural information due to shooting angle).

[0108] Only data that passes the evaluation system (semantic consistency ≥ threshold, contrast loss normal) enters the gene recognition module. Unqualified data is directly marked and temporarily suspended to prevent noisy data from contaminating the recognition model.

[0109] A deep learning-based anomaly detection algorithm was developed to automatically identify common problems in data acquisition, such as missing point clouds, texture distortion, and color distortion, and to provide repair suggestions. The steps of the anomaly detection algorithm are as follows:

[0110] 1. Data Preprocessing: Transforming raw data into a format easily understood by the model. Point Cloud: Voxelization (converting to a regular 3D mesh) is performed, or local neighborhood features of each point are extracted using models such as PointNet++. Images: Size standardization and color space conversion are required.

[0111] 2. Feature Extraction and Analysis. Point Cloud Hole Detection: The model determines anomalies by comparing the differences between the geometry surrounding a point and the overall surface morphology. Graph Asynchronous Transfer Model (GAT) analyzes the relationships between points to identify holes that shouldn't exist. Texture Distortion Detection: The model learns the statistical patterns of normal textures. When distortion or stretching occurs, its statistical features deviate from the "normal value," thus being flagged. Color Distortion Detection: The model automatically estimates the color of the light source during capture. If the estimated light deviates too far from white, it indicates a color cast in the image.

[0112] 3. Generate anomaly mask. The algorithm segments the regions that the model determines to be abnormal (missing, distorted, or discolored) at the pixel or point level, generating a highly visual anomaly mask (a black and white marked image) that clearly indicates where the problem occurs.

[0113] A dedicated digitization pipeline was designed for historical documents: The first step is preliminary preparation and document preprocessing, prioritizing historical documents and inspecting each page before scanning to smooth out creases, remove dust and stains, and repair damage. The second step involves digitizing the documents using high-precision scanning equipment, setting scanning parameters, recording scanning time, equipment parameters, document number, and operator information. The third step is processing and optimizing the original images, using an improved document image binarization algorithm to process faded or damaged documents. The fourth step is OCR text recognition, which adds a specialized module for recognizing ancient Chinese characters and variant characters in addition to general text recognition. For handwritten documents, a temporal convolutional neural network is used for continuous handwritten text recognition, achieving an accuracy rate of over 85%. The final step is data storage.

[0114] The binarization algorithm transforms a grayscale / color image into a binary image with only black text and a white background. The CS-SWF algorithm is employed, using a filter that preserves edges and texture to extract stroke features. Different regions are then clustered and segmented to ultimately separate the text from the background. First, the image is preprocessed by taking the maximum values ​​of the RGB channels to convert the color image to grayscale. Next, a side-window filter (SWF) is used to calculate the filtering result, prioritizing the result closest to the original pixels, smoothing the strokes while preserving texture continuity. Then, the overall image structure is preserved, local outliers are corrected, and pixels with similar brightness values ​​are grouped together, classifying pixels into text and background categories. Finally, isolated noise points or broken strokes are processed to obtain a complete and clean binary image.

[0115] Regarding the unification of spatiotemporal benchmarks, an optimized algorithm for spatiotemporal registration of multi-source data is proposed. First, feature extraction is performed using algorithms such as SIFT and ORB to detect extreme points and key points in the image. Since there are many incorrect matches in the initial matching, the RANSAC algorithm is then used to eliminate these mismatches. This involves randomly selecting a small number of point pairs to calculate the transformation model, validating the model with the remaining point pairs, and counting how many point pairs conform to the model. This process is repeated multiple times, retaining the model with the most correct matches. Finally, high-precision registration of data from different sources is achieved. For time-series data, a dynamic time warping algorithm is used to align data from different acquisition time points, establishing a unified time axis.

[0116] 1) For example, spatial registration (geometric alignment);

[0117] Input: Source data A (such as ground laser scanning point cloud), target data B (such as dense point cloud or image generated by UAV oblique photography).

[0118] step:

[0119] (1) Feature point extraction (performed separately for the two types of data);

[0120] For point clouds: Calculate a fast point feature histogram for each point to describe the local geometry, generating a 33-dimensional feature vector for each point.

[0121] For the image: extract SIFT keypoints, and generate a 128-dimensional descriptor for each keypoint.

[0122] Quantity: Extract at least 500 feature points per frame of data.

[0123] (2) Feature matching (finding corresponding point pairs);

[0124] We employ FLANN fast nearest neighbor search to find the two feature points in the target data that are closest to each feature point in the source data using Euclidean distance.

[0125] Lowe's ratio test: If the ratio of the nearest distance to the second nearest distance is < 0.7, the pair is retained as a candidate match.

[0126] Initial number of matches: approximately 200-300 pairs.

[0127] (3) RANSAC (Random Match Elimination);

[0128] Calculate the initial transformation matrix (rotation R + translation T) by randomly selecting 3 pairs of matching points.

[0129] Apply the transformation to all matching points and count the number of interior points (matches with an error of < 5mm).

[0130] Repeat the iteration 500 times to find the transformation matrix that retains the most interior points.

[0131] Final valid matching pairs: ≥100 pairs.

[0132] (4) Refined registration (ICP iteration nearest point);

[0133] Use the transformation matrix obtained from RANSAC as the initial value.

[0134] (5) Iterative execution:

[0135] Find the nearest point (Euclidean distance) in the target point cloud for each point in the source point cloud.

[0136] Calculate the optimal transformation that minimizes the root mean square error.

[0137] Update the transformation matrix.

[0138] (6) Termination condition: The root mean square error change is less than 1e-6 or the iteration reaches 50 times.

[0139] Final registration accuracy: ≤ 3mm.

[0140] 2) For example, time registration (time axis alignment).

[0141] Scenario: Data collected in the same region at different times (e.g., 2010, 2020, 2024).

[0142] step:

[0143] (1) Timestamp standardization: The collection time of each batch of data is uniformly converted into a Unix timestamp (second precision).

[0144] (2) Dynamic Time Warping (DTW): Used to align sequences of changes caused by irregular acquisition time intervals (such as annual deformation of buildings).

[0145] Let sequence P (reference time point) and Q (time point to be aligned) have lengths m and n, respectively.

[0146] Construct an m×n distance matrix, with element d(i,j) = |P(i) - Q(j)|.

[0147] Find the path with the minimum cumulative distance from (1,1) to (m,n), where the slope can be 1, 2, or 1 / 2.

[0148] Output the aligned time index map.

[0149] (3) Linear interpolation (filling in missing time points): For time points where no data was collected, linear interpolation is performed using the data from the two consecutive data points according to the time ratio:

[0150] ;

[0151] Time alignment accuracy: up to the monthly level (for annual monitoring data) or the millisecond level (for dynamic scan sequences).

[0152] The registered data is stored as a unified four-dimensional tuple: (x,y,z,t,modality,quality_score); x,y,z: spatial coordinates (millimeters); t: standardized timestamp (seconds); modality: modality identifier (0=point cloud, 1=image, 2=text, etc.); quality_score: the overall quality score (0~1) of the data point, given by the quality assessment in step S102.

[0153] In one possible implementation, a multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to these multimodal features are then extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connections between these genes, including:

[0154] S104.1 Based on the filtered target multimodal data, a multi-scale feature pyramid network is used to extract the image features of the spatiotemporally registered target image data, and at the same time, the text features of the digitized historical documents are extracted. A hierarchical classification architecture is used to perform gene classification and recognition on the image features and text features to obtain hierarchical cultural heritage landscape genes.

[0155] It is worth noting that existing technologies can also be used to extract image and text features, and then a hierarchical classification architecture can be used to perform gene classification and recognition on the image and text features.

[0156] S104.2. After spatiotemporal registration, the spatial relationships between cultural heritage landscape genes are extracted from the target point cloud data. The cultural heritage landscape genes are used as graph nodes, and the spatial relationships are used as edges. Neighborhood information is aggregated through a message passing mechanism to learn the dependencies between genes and output the cultural heritage landscape genes and the connection relationships between them.

[0157] like Figure 4 As shown, for example, cultural heritage landscape gene identification is the core function of the system. It uses high-quality, uniformly formatted data provided in step S02 and adopts a progressive identification strategy from local to global and from morphology to semantics.

[0158] In the low-level feature extraction stage, due to the large differences in target scale, and the fixed receptive field of traditional convolutional networks, for example, if the network is too shallow, it can see small patterns but cannot see the overall structure of a large roof; if the network is too deep, it can understand large scenes, but the details of small patterns will be lost in the downsampling of layers. A multi-scale feature pyramid network is designed, which does not rely on features of a single level, but instead constructs a multi-level feature system "from fine to coarse". Features are extracted at different levels of the network to address the size differences of architectural components, ensuring that both large components such as beams, columns, and roofs can be identified, as well as small decorations such as carved patterns. Since the shape of traditional convolutional kernels is a fixed regular grid, it cannot handle the irregular, non-rigid, and diverse geometric forms in cultural heritage well. Deformable convolution is introduced, where the sampling points of the convolutional kernel are no longer limited to fixed grid positions, but can learn an offset based on the image content, deforming the sampling points to the actual shape of the target, enhancing the modeling ability of irregular shapes and adapting to the complex geometric forms commonly found in cultural heritage.

[0159] The gene classification and identification employs a hierarchical classification architecture. First, coarse-grained classification is performed to identify broad categories of building components (such as roofs, walls, foundations, etc.). Then, fine-grained classification is performed under each broad category to identify specific types (such as hip roofs, gable roofs, and hard gable roofs). This hierarchical design reduces the difficulty of classification while maintaining the systematic nature of the classification system. For example, for identified cultural heritage landscape genes, the classification output includes the following information: First-level category (broad category): such as "building components," "decorative patterns," "structural system." Second-level category (function / location): such as "roof components," "beam structure components," "column grid components." Third-level category (specific type): such as "gable roof," "hard gable roof," "hip roof." Attribute tags: such as material (wood / stone / brick), era (Tang / Song / Ming / Qing), region (North / Jiangnan / Southwest).

[0160] Spatial relational reasoning is key to understanding the structural logic of cultural heritage. A relational reasoning model based on graph neural networks is constructed, using identified cultural genes as graph nodes and spatial relationships and structural connections between genes as edges. Neighborhood information is aggregated through a message-passing mechanism to learn the dependencies between genes. For example, identifying the structural relationships between brackets and column heads, and beams, helps understand the timber frame system of traditional Chinese architecture. Each identified cultural gene is treated as a graph node, with spatial adjacencies or structural connections between genes as edges (e.g., the connection between brackets and column heads). Neighborhood aggregation: Each node aggregates the feature vectors of its neighbors into itself through weighted summation or attention mechanisms. Commonly used formulas:

[0161] ;

[0162] in, Edge weights (can be learned). It is the set of neighboring nodes.

[0163] Update node representation: The aggregated neighborhood information is concatenated with its own features, and a new node embedding is obtained through a fully connected layer.

[0164] Multi-level stacking: Repeat steps 2-3 times (usually K=3) to enable nodes to capture dependencies within a range of K hops.

[0165] Its role in gene classification:

[0166] Through message passing, a gene node can perceive the type and attributes of its associated genes (such as a dougong knowing the type of beam it supports), thereby correcting or enhancing its own classification.

[0167] For example, isolated identification may misclassify "column head", but the presence of neighboring nodes "dougong" and "liangfang" will guide the model to correctly classify it as "column head".

[0168] Finally, the classification probability of each gene node after fusing neighborhood information is output, which significantly improves the recognition accuracy of genes related to spatial structure.

[0169] Based on the cultural landscape gene theory, a gene knowledge base is established by linking landscape gene information through gene coding under two major categories: material genes and non-material genes. The coding system includes six dimensions: regional code, gene category (material category M, non-material category I), secondary classification (construction type code), tertiary classification (specific type code), attribute code (material age, etc.), and instance number. For example, the coding example for the wooden beam component of a building in Datian, Fujian is Min-DT-M-02-001-M-00001.

[0170] In terms of verifying the recognition results, a multi-layered verification mechanism is established. First, unreliable results are filtered out using predicted confidence levels. Then, spatial consistency checks are used to eliminate obvious errors (such as suspended components or combinations that do not conform to structural logic). Finally, cross-validation with historical documents ensures the reliability of the recognition results. For example, the credibility quantification of recognition results can be achieved by using the consistency score of its corresponding multimodal data in the semantic space as part of the confidence level for each cultural gene identified in the first step. For instance, the credibility of a dougong (bracket set) recognition result is higher if its point cloud, image, and text are highly semantically consistent. Low-consistency data output by the evaluation system (especially samples with moderate semantic similarity that may be misjudged by humans) can be recommended to experts for annotation, thereby effectively improving the robustness of the recognition model.

[0171] In one possible implementation, a gene code is constructed, and the gene code, cultural heritage landscape genes, and the connections between cultural heritage landscape genes are stored together to establish a gene knowledge base. A temporal knowledge graph is then constructed based on the gene knowledge base, including:

[0172] S105.1 Construct gene coding, associate and store the gene coding, cultural heritage landscape genes and the connection relationship between cultural heritage landscape genes to obtain cultural heritage landscape gene information, and use the cultural heritage landscape gene information to establish a gene knowledge base;

[0173] S105.2 Based on the spatiotemporally registered target point cloud data, the spatiotemporally registered target image data, and the cultural heritage landscape gene information in the gene knowledge base, a temporal knowledge graph is constructed using a temporal knowledge graph.

[0174] In one possible implementation, the method further includes:

[0175] Based on the cultural gene sequence composed of all cultural heritage landscape gene information corresponding to the same cultural heritage landscape in historical data, the dynamic time warping algorithm is used to calculate the similarity of different cultural gene sequences and identify similar evolutionary patterns.

[0176] Based on this, a family tree of cultural genes is constructed by grouping cultural gene sequences with a common ancestor or similar developmental paths into the same lineage through a genealogical clustering algorithm.

[0177] like Figure 5 As illustrated, dynamic genealogical analysis aims to reveal the evolutionary patterns of cultural heritage over time, providing a scientific basis for its protection, restoration, and cultural dissemination. Temporal knowledge graphs, building upon traditional knowledge graphs, add a temporal dimension, allowing each fact to be labeled with "when it was established." The construction employs a four-tuple representation (subject-relationship-object-time), using spatiotemporal registration and gene identification of multi-source data to find the same gene in data collected from different periods, accurately recording the state changes of cultural genes in different historical periods. The time attribute supports three representation methods: point time, time interval, and fuzzy time, adapting to the temporal uncertainty in historical research. A temporal reasoning mechanism is established to automatically infer missing time information and complete the historical development context.

[0178] Evolutionary pattern mining aims to discover the evolutionary patterns of cultural genes over time from historical data. A dynamic time warping algorithm is designed to calculate the similarity of different cultural gene sequences and identify similar evolutionary patterns. A phylogenetic clustering algorithm is used to group cultural genes with a common ancestor or similar developmental paths into the same phylogenetic family, constructing a family tree of cultural genes. Evolutionary path analysis tracks the entire process of a specific gene from its origin, development, spread, and mutation, revealing the patterns of cultural transmission. Data collected from different periods is evaluated to verify semantic consistency, ensuring that the objects compared across time are the same cultural genes. For example, the dotted clouds of dougong (bracket sets) from the Tang and Song dynasties must first confirm that their corresponding textual descriptions are aligned in the semantic space before evolutionary analysis can be performed. Each evolutionary link in the dynamic phylogenetic family can be traced back to the quality score of the original data used in the evaluation. If the data quality on which an evolutionary path depends is low, the system will issue a warning, indicating that the reliability of the evolutionary conclusion is insufficient.

[0179] This modeling of cultural transmission dynamics draws on research methods from computational social science, using mathematical equations to describe how cultural characteristics diffuse in space and time. A cultural transmission model based on reaction-diffusion equations is established, treating the spread of cultural genes as a diffusion process in space and time. Temporal knowledge graphs provide the data foundation, evolutionary pattern mining reveals patterns, and the cultural transmission dynamics model explains these patterns. This model is not a general reaction-diffusion equation, but rather a set of coupled partial differential equations designed specifically for the characteristics of cultural heritage transmission, as detailed below:

[0180] ;

[0181] in: The intensity of cultural gene propagation at spatial location x and time t (normalized value from 0 to 1); Let be the diffusion coefficient, and ; Therefore, the basic diffusion rate is (dimensionless, taken as 0.1-0.5). This represents the geographical distance (km) from the origin of the culture. Geographical attenuation factor (range 0.01-0.05); This represents the activity level of an era (e.g., 1.5 for the prosperous period of the Silk Road, and 0.3 for the period of war). It is a local reaction term (logistic growth), and , The rate of cultural innovation / mutation (dimensionless, ranging from 0.05 to 0.2); It is an external influence item, and , To influence the intensity (0~0.3), Let be the weight of the k-th cultural event (e.g., immigration, war, trade). It is a spatiotemporal impulse function, representing the occurrence of time at a specific location and time. β, ρ, and γ are optimized through inversion using gene spatiotemporal distribution data (least square fitting of historical propagation paths). σ(t): Quantifies the activity level of an era based on historical documents. I(x,t) is extracted from event identification.

[0182] Numerical solution: The finite difference method is used, with a spatial step of 10 km, a time step of 1 year, and a boundary condition of no flux (∂G / ∂n=0).

[0183] The model considers multiple influencing factors:

[0184] Geographical distance: affects the ease or difficulty of cultural transmission.

[0185] Cultural similarity: People with similar cultural backgrounds are more likely to accept foreign elements.

[0186] Social networks: spread through trade, immigration, and other means.

[0187] Political boundaries: may hinder or facilitate cultural exchange.

[0188] In one possible implementation, the method further includes:

[0189] Based on the temporal knowledge graph, multiple candidate design schemes are generated;

[0190] The NSGA-II algorithm is used to iteratively update multiple candidate design schemes, solve for the Pareto optimal solution set, output the optimal candidate design scheme as the recommended scheme, and push the recommended scheme to the user.

[0191] like Figure 6 As illustrated, the gene element library forms the basis of the design system, using the evolutionary pattern of step S04 as a foundation. Each gene element contains multi-level information: Geometric parameters: adjustable parameters controlling morphological characteristics; Construction logic: combination rules and structural relationships between components; Cultural semantics: symbolic meaning, usage context, and other cultural connotations; Materials and processes: digital descriptions of traditional materials and processing techniques.

[0192] The parametric generation engine employs a constraint-satisfying approach. After designers set constraints such as functional requirements, spatial conditions, and cultural styles, the system automatically searches for design solutions that satisfy all constraints. Through parametric modeling technology, continuous variations in design solutions are achieved, facilitating the exploration of the design solution space. Generative adversarial networks are introduced to learn the distribution characteristics of traditional styles, generating new designs that conform to traditional aesthetics.

[0193] For example, the temporal knowledge graph and evolutionary pedigree contain for each cultural gene: geometric parameter ranges (such as the overhang length of bracket sets, the upturned angle of eaves); structural constraint rules (such as "bracket sets support beams and lintels", "the ratio of column diameter to column height"); cultural semantic labels (such as "the Western Sichuan residential style must include white walls and black tiles").

[0194] Genes related to the target design scheme are screened out from the knowledge graph (for example, when designing a Western Sichuan residential building, genes such as "wooden column-and-tie construction", "overhanging gable roof", and "white wall" are extracted). The variable parameters of each gene (such as column spacing and roof slope) are taken as decision variables, and their reasonable value ranges are recorded. Within the value range of each decision variable, Latin hypercube sampling is used to generate an initial population (for example, with a population size of 100). Each individual represents a complete set of parameter combinations, that is, an initial design scheme. The structural rules extracted from the knowledge graph (such as "eave overhang length ≥ 0.8m") are taken as hard constraints. Individuals that violate the constraints are directly eliminated during initialization, and only design schemes that meet the cultural rules are retained as the initial solution set of NSGA-II, so as to obtain candidate design schemes.

[0195] The multi-objective optimization algorithm balances multiple conflicting objectives in the design. A multi-objective optimization model including cultural inheritance, functional rationality, technical feasibility, economy, etc. is established. The improved NSGA-II algorithm is used to solve the Pareto optimal solution set, providing designers with multiple excellent schemes for selection.

[0196] NSGA-II algorithm steps:

[0197] 1. Fast non-dominated sorting. Individuals in the population are assigned to different Pareto front layers. For example, in the judgment of the Pareto dominance relationship, the condition for individual a to dominate individual b is modified as: for all k, Fk(a)≤Fk(b), and there exists at least one k such that Fk(a)<Fk(b). That is, the corrected Fk is directly used to replace the original fk for comparison. In this way, even if the original objectives of two individuals are the same, the individual with higher cultural similarity will dominate the individual with lower similarity.

[0198] 2. Crowding distance calculation. For individuals in the same front layer, diversity is maintained through the crowding distance. A larger crowding distance indicates that the individual is more isolated in the objective space and should be preferentially retained.

[0199] 3. Selection, crossover and mutation.

[0200] 4. Elite retention strategy. Individuals close to the Pareto front are preferentially retained.

[0201] Optionally, the fitness of the NSGA-II algorithm can be designed as:

[0202] ;

[0203] in, For a design scheme (a vector of decision variables, such as column spacing, roof slope, component dimensions, etc.); This represents the kth original objective function value (normalized to the [0,1] interval); λ is the penalty coefficient, controlling the intensity of the penalty for semantic non-compliance. In this application, λ = 10 (an empirical value that significantly worsens the fitness of individuals violating semantics). The penalty term is generated only when S < 1, and the penalty term increases linearly as the similarity decreases. The semantic similarity between design scheme x and target cultural gene cluster C is calculated using a unified semantic space (value range [0,1], where 1 indicates complete similarity).

[0204] Optionally, embodiments of this application also include a semantic evaluation system to quantify design value from multiple dimensions:

[0205] Cultural gene conformity: Calculate the semantic similarity between the design scheme and traditional genes.

[0206] Functional performance evaluation: The practicality is evaluated through techniques such as spatial analysis and streamline simulation.

[0207] User experience quality: Combining objective measurement methods such as eye tracking and physiological signals.

[0208] Technical and economic indicators: material usage, cost estimation, construction difficulty, etc.

[0209] Source data screening for the design gene bank: Only high-quality cultural genes that pass the evaluation system can enter the parametric component library, ensuring the cultural accuracy of the design basis.

[0210] Semantic rationality verification of the design scheme: For a generated design scheme, the quality scores of the original data corresponding to each gene component can be used as a component of the design reliability. If a gene element originates from low-quality data, the system will prompt the designer to conduct a manual review.

[0211] Based on the above technical solution, the beneficial effects of the embodiments of this application are as follows:

[0212] (1) Traditional cultural heritage digitization technologies exhibit a significant "data silo" phenomenon when processing multi-source heterogeneous data. Specifically, this manifests as follows: 3D laser scanning data and historical documents are stored separately, lacking effective semantic association; different modal data such as images, text, and point clouds are processed using independent workflows, making cross-modal retrieval impossible; and inconsistent data quality standards make it difficult to guarantee the reliability of subsequent analysis results.

[0213] This application establishes a unified multimodal data representation system. A Transformer-based cross-modal encoder maps heterogeneous data such as images, text, and point clouds to a unified 512-dimensional semantic space. In actual testing, the system successfully integrated 2.5TB of multi-source data, including high-precision point clouds, historical maps, and archival documents, in the Shu King's Palace digitization project, achieving a cultural gene identification accuracy of 95.2%, a 41.5% improvement over traditional methods. An intelligent data quality control system was developed. Through real-time quality monitoring algorithms, key indicators such as point cloud density, image resolution, and coverage integrity are automatically evaluated. In the Dunhuang Grottoes digitization project, this system increased the data acquisition qualification rate from 67% using traditional methods to 92%, significantly reducing rework costs. Cross-modal semantic-level retrieval was achieved. Users can directly retrieve relevant 3D model components through text descriptions or find similar historical cases by inputting sketches. Test data shows that the mAP@50 of cross-modal retrieval reached 87.6%, a 52.3% improvement over traditional keyword-based retrieval methods.

[0214] (2) Most existing cultural heritage identification systems remain at the level of surface feature recognition: traditional image processing methods can only identify obvious visual features and cannot understand cultural connotations; machine learning models lack domain knowledge guidance and are not sensitive enough to fine-grained cultural differences; the identification results lack semantic interpretation and are difficult to support in-depth academic research.

[0215] This application achieves a qualitative leap in semantic understanding through a knowledge-enhanced cognitive reasoning mechanism. First, it constructs a cultural heritage knowledge graph containing over 3,500 core concepts. This graph clearly defines the hierarchical structure and semantic relationships of concepts such as architectural components, decorative patterns, and craft techniques. In the Jiangnan water town preservation project, the system successfully identified 15 regionally distinctive architectural genes and accurately labeled their cultural connotations and technical characteristics. Second, it develops a multi-layered feature understanding architecture. The system can not only identify "what" but also understand "why" and "how they are related." For example, when analyzing dougong (bracket set) components, the system can not only identify their type but also infer their structural function, period characteristics, and regional style. This deep understanding capability allows the system to maintain a high recognition accuracy of 92.3% even in complex scenarios. Finally, it implements relational reasoning based on graph neural networks. By analyzing the spatial relationships and semantic associations between cultural genes, the system can reconstruct the structural logic of traditional architecture. In the study of Fujian Tulou (earthen buildings), the system successfully restored the organic combination of the unique defensive system and living space, providing a new technical means for interpreting the value of cultural heritage.

[0216] (3) Traditional research methods have obvious shortcomings in the analysis of cultural evolution: they rely on manual sorting of literature, which is inefficient and highly subjective; it is difficult to quantify the transmission path and evolution law of cultural genes; and they lack the ability to model the dynamics of cultural evolution.

[0217] This application achieves quantitative analysis of cultural evolution through temporal mapping and dynamic genealogy techniques. Technically, it innovatively proposes a four-tuple temporal knowledge representation method. This method accurately records the state changes of cultural genes in different historical periods, supporting three representation modes: point time, time interval, and fuzzy time. In the study of the Grand Canal Cultural Belt, the system successfully constructed a cultural gene evolution map spanning 2000 years, clearly demonstrating the coordinated development of shipping technology, water conservancy projects, and towns along the canal. In terms of analytical methods, a cultural transmission dynamics model is introduced. Based on the reaction-diffusion equation, the system can simulate the transmission process of cultural genes in space and time, quantitatively analyzing the influence of factors such as geographical distance, cultural similarity, and social networks. The simulation results show an 89.2% agreement with historical records, providing a scientific basis for the formulation of cultural heritage protection plans. Regarding visualization, a multi-granularity spatiotemporal evolution display system has been developed. Users can observe the diachronic changes of cultural landscapes by navigating the timeline or analyze the inheritance relationships of specific genes through genealogy trees. This intuitive display method greatly improves the efficiency of communication of research results.

[0218] (4) There is a significant gap in the application of existing technologies for the revitalization of cultural heritage. Digital achievements are disconnected from design practices, making it difficult to directly support innovative designs. There is a lack of quantitative evaluation systems, making it difficult to accurately measure the cultural value of design schemes. The integration of traditional crafts with modern technologies is insufficient, resulting in low efficiency in innovation transformation.

[0219] This application establishes a complete chain from preservation to revitalization through a parametric design and semantic evaluation system. First, a parametric component library of cultural genes was established. Each gene component contains multi-dimensional information such as geometric morphology, structural logic, and cultural semantics. Designers can generate design schemes that meet modern needs by adjusting parameter values. In the Chengdu Kuanzhai Alley renovation project, the parametric design scheme generated by the system preserved the traditional genes of western Sichuan folk houses while meeting the needs of modern commercial functions, resulting in a 35% increase in visitor satisfaction after the project's implementation. Second, a multi-objective optimization design framework was constructed. The system simultaneously considers multiple objectives such as cultural inheritance, functional rationality, technical feasibility, and economic efficiency, automatically generating a Pareto optimal solution set through an improved NSGA-II algorithm. Test data shows that the system improves the efficiency of design scheme generation by 4.8 times and increases the cultural conformity score to 87.6 points. Finally, an objective evaluation method based on physiological signals was developed. Through technologies such as eye tracking and heart rate variability analysis, the system can quantitatively evaluate the cultural communication effect of the design scheme. In the design of a museum exhibition, optimization based on the evaluation results increased the average viewing time of the exhibits by 42% and improved the completeness of information reception by 28%.

[0220] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for automated identification and dynamic lineage construction of cultural heritage landscape genes, characterized in that, include: A multimodal feature encoder is constructed, and the existing multimodal data in the training set corresponding to the cultural heritage landscape is mapped to a unified semantic space using the multimodal feature encoder. Collect target multimodal data corresponding to the cultural heritage landscape to be classified, and perform spatiotemporal registration on the target multimodal data to obtain spatiotemporally registered target multimodal data; The processed target multimodal data is evaluated from multiple dimensions using semantic features in the semantic space and a multimodal feature encoder to obtain multidimensional evaluation results. The processed target multimodal data is then filtered based on the multidimensional evaluation results to obtain filtered target multimodal data. A multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to the multimodal features are extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connection relationships between cultural heritage landscape genes. Gene codes are constructed, and the gene codes, cultural heritage landscape genes, and the connections between cultural heritage landscape genes are associated and stored. A gene knowledge base is established, and a temporal knowledge graph is constructed based on the gene knowledge base to complete the automated identification and dynamic lineage construction of cultural heritage landscape genes.

2. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 1, characterized in that, After constructing the multimodal feature encoder, the following is also included: A dual-modal coding query architecture is constructed using a multimodal feature encoder; A dual-modal coding query architecture is used to match the user-input query modality data with the target modality data in the existing multimodal data, and the retrieved target modality data is returned to the user. The query modality refers to the modality data input by the user and used as the retrieval condition, and the target modality data refers to the modality data required by the user.

3. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 1, characterized in that, Constructing a multimodal feature encoder, and using the multimodal feature encoding to map existing multimodal data in the training set corresponding to cultural heritage landscapes to a unified semantic space, including: A visual encoder for encoding visual modal data is constructed using Vision Transformer; a point cloud encoder for encoding point cloud modal data is constructed using Point Transformer; and a text encoder for encoding text modal data is constructed using BERT. The visual encoder, point cloud encoder, and text encoder are used together as a multimodal feature encoder, and the semantic features encoded by the visual encoder, point cloud encoder, and text encoder have the same dimension. A visual encoder is used to encode and map existing visual modal data, a point cloud encoder is used to encode and map existing point cloud modal data, and a text encoder is used to encode and map existing text modal data, forming a unified semantic space composed of multimodal semantic features. The multimodal feature encoding includes existing multimodal data in the training set corresponding to the cultural heritage landscape, including existing visual modal data, existing point cloud modal data, and existing multimodal data corresponding to the same cultural heritage landscape gene.

4. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 3, characterized in that, Collect target multimodal data corresponding to the cultural heritage landscape to be classified, and perform spatiotemporal registration on the target multimodal data to obtain spatiotemporally registered target multimodal data, including: Collect target point cloud data, target image data, and historical documents corresponding to the cultural heritage landscape to be classified to obtain target multimodal data; Spatial registration is performed on the target point cloud data and the target image data, and a dynamic time warping algorithm is used to align the data at different acquisition time points to obtain the spatiotemporally registered target point cloud data and the target image data. The historical documents are preprocessed and subjected to text recognition processing to obtain digitized historical documents. The digitized historical documents, the spatiotemporally registered target point cloud data, and the target image data are collectively used as the spatiotemporally registered target multimodal data.

5. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 4, characterized in that, After spatial registration of the target point cloud data and the target image data, the method further includes: Anomaly detection is performed on the spatially registered target point cloud data and target image data to identify abnormal data regions, and anomaly masks are generated in the abnormal data regions to provide anomaly alerts.

6. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 4, characterized in that, The processed target multimodal data is evaluated from multiple dimensions using semantic features in the semantic space and a multimodal feature encoder to obtain multidimensional evaluation results. Based on these multidimensional evaluation results, the processed target multimodal data is then filtered to obtain filtered target multimodal data, including: A multimodal feature encoder is used to extract multi-target semantic features in a unified semantic space from the processed target multimodal data, and the pairwise similarity between the multi-target semantic features is calculated. If the pairwise similarity of the processed target multimodal data is greater than a preset threshold, the data quality is determined to be qualified; otherwise, the data quality is determined to be unqualified. The cross-modal contrast loss corresponding to the multi-target semantic features is obtained using semantic features in the semantic space. When the cross-modal contrast loss meets the loss constraint condition, the data quality is determined to be qualified; otherwise, the data quality is determined to be unqualified. The target multimodal data that does not meet the data quality standards is filtered out to obtain the filtered target multimodal data.

7. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 6, characterized in that, A multi-scale feature pyramid network is used to extract features from the filtered target multimodal data to obtain multimodal features. The spatial relationships corresponding to these multimodal features are then extracted, and a hierarchical classification architecture is used to perform gene classification and identification on the multimodal features to obtain cultural heritage landscape genes and the connections between these genes, including: Based on the filtered target multimodal data, a multi-scale feature pyramid network is used to extract image features from the spatiotemporally registered target image data. At the same time, text features from the digitized historical documents are extracted. A hierarchical classification architecture is used to perform gene classification and recognition on the image features and text features to obtain hierarchical cultural heritage landscape genes. After spatiotemporal registration, the spatial relationships between cultural heritage landscape genes are extracted from the target point cloud data. The cultural heritage landscape genes are used as graph nodes, and the spatial relationships are used as edges. Neighborhood information is aggregated through a message passing mechanism to learn the dependencies between genes and output the cultural heritage landscape genes and the connection relationships between them.

8. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 7, characterized in that, Construct gene codes, associate and store the gene codes, cultural heritage landscape genes, and the connections between cultural heritage landscape genes, establish a gene knowledge base, and construct a temporal knowledge graph based on the gene knowledge base, including: Construct gene codes, associate and store the gene codes, cultural heritage landscape genes, and the connection relationships between cultural heritage landscape genes to obtain cultural heritage landscape gene information, and use the cultural heritage landscape gene information to establish a gene knowledge base; Based on the spatiotemporally registered target point cloud data, the spatiotemporally registered target image data, and the cultural heritage landscape gene information in the gene knowledge base, a temporal knowledge graph is constructed using a temporal knowledge graph.

9. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 8, characterized in that, Also includes: Based on the cultural gene sequence composed of all cultural heritage landscape gene information corresponding to the same cultural heritage landscape in historical data, the dynamic time warping algorithm is used to calculate the similarity of different cultural gene sequences and identify similar evolutionary patterns. Based on this, a family tree of cultural genes is constructed by grouping cultural gene sequences with a common ancestor or similar developmental paths into the same lineage through a genealogical clustering algorithm.

10. The method for automated identification and dynamic phylogenetic construction of cultural heritage landscape genes according to claim 1, characterized in that, Also includes: Based on the temporal knowledge graph, multiple candidate design schemes are generated; The NSGA-II algorithm is used to iteratively update multiple candidate design schemes, solve for the Pareto optimal solution set, output the optimal candidate design scheme as the recommended scheme, and push the recommended scheme to the user.

Images