Label-based multi-modal cultural relic knowledge graph construction method, system and equipment

Abstract

The invention provides a multi-modal cultural relic knowledge graph construction method, system and device based on labels, and relates to the technical field of cultural relic knowledge graph construction. The method comprises the steps of obtaining multi-spectral image and texture image multi-modal data of a target rammed earth cultural relic, analyzing the multi-spectral image based on a pre-constructed rammed earth material standard reflection spectrum database, and obtaining a multi-modal historical relic knowledge graph; obtaining material component features of each space region, and correlating corresponding texture images to generate material and morphology feature vectors; then, the vectors are mapped into graph network node initial features, updated aggregation features are obtained through iterative propagation aggregation, composite labels containing material attributes, disease types and spatial positions are generated accordingly, finally, a target rammed earth cultural relic knowledge graph is constructed based on all the composite labels, and rammed earth cultural relic multi-modal data fusion can be achieved. And a composite label is accurately generated and a knowledge graph is constructed, so that systematic management and application of cultural relic information are facilitated.

Description

Technical Field

[0001] This application relates to the field of cultural relic knowledge graph construction technology, and in particular to a label-based multimodal cultural relic knowledge graph construction method, system and device. Background Technology

[0002] Cultural relic knowledge graphs are important tools for systematically integrating multi-dimensional information about cultural relics. They provide data support for the protection, research, and display of cultural relics and have broad application prospects in the field of cultural heritage transmission. By linking information such as the attributes, condition, and environment of cultural relics, they help relevant personnel quickly grasp the core information of cultural relics, improving work efficiency and scientific rigor.

[0003] Currently, existing tag-based multimodal cultural relic knowledge graph construction methods mostly involve acquiring various types of data such as images and texts of cultural relics, generating tags based on simple data matching or basic feature extraction methods, and then constructing a knowledge graph based on tag associations. These methods have been applied in some cultural relic information integration scenarios, but they do not fully consider the deep integration of key features such as the material and appearance of cultural relics.

[0004] However, current construction methods have significant shortcomings. They often fail to accurately capture the specific attributes and state relationships of different spatial regions on the surface of cultural relics, resulting in incomplete and inaccurate generated label information, which in turn affects the practicality of the knowledge graph. Furthermore, the analysis depth of multimodal data from cultural relics is insufficient, making it impossible to effectively uncover the potential relationships behind the data. Summary of the Invention

[0005] The purpose of this application is to provide a label-based multimodal cultural relic knowledge graph construction method, system, and device to solve the problems of insufficient accuracy and information completeness in the construction of cultural relic knowledge graphs in the prior art.

[0006] To address the aforementioned technical problems, firstly, this application provides a tag-based multimodal cultural relic knowledge graph construction method, comprising:

[0007] Acquire multimodal data of the target rammed earth cultural relic, the multimodal data including multispectral image data covering the surface of the cultural relic, and texture image data corresponding to the surface texture of the cultural relic;

[0008] Based on a pre-constructed standard reflectance spectrum database of rammed earth materials, spectral analysis is performed on the multispectral image data to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic.

[0009] The material composition features corresponding to each spatial region are associated with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region.

[0010] The material feature vector and shape feature vector corresponding to each spatial region are mapped to the initial features of the corresponding nodes in a pre-constructed graph network. Based on the initial features of the nodes and the connection relationship of the edges, the node features are iteratively propagated and aggregated on the graph network to obtain the updated aggregated features of each node.

[0011] Based on the updated aggregation features of each node, a composite label is generated for the spatial region corresponding to the node. The composite label contains the material properties, disease type, and spatial location information of the spatial region.

[0012] Based on all generated composite tags, construct a knowledge graph of the target rammed earth artifact.

[0013] Optionally, the step of generating composite labels for the spatial region corresponding to each node based on the updated aggregation features of each node includes:

[0014] For each node in the graph network, the updated aggregated features of the node are input into a preset classification model to parse and obtain the material property category and disease type category corresponding to the node;

[0015] Obtain the spatial location information represented by the node;

[0016] The material property category, the disease type category obtained from the parsing, and the extracted spatial location information are combined to generate the composite label corresponding to the node.

[0017] Optionally, the step of constructing a knowledge graph of the target rammed earth artifact based on all generated composite tags includes:

[0018] For the material properties, disease types, and spatial location information parsed from the composite tags, corresponding entities are created or matched in a preset knowledge graph framework;

[0019] Establish an initial attribute association between the entity representing the spatial location information and the entity representing the material properties and disease types;

[0020] All created or matched entities and their initial attribute associations are integrated to complete the construction of the knowledge graph of the target rammed earth cultural relic.

[0021] Optionally, after integrating all created or matched entities and established initial attribute associations to complete the construction of the knowledge graph of the target rammed earth cultural relic, the method further includes:

[0022] In the knowledge graph, a group of spatial location entities is identified, wherein all spatial location entities in the group are adjacent to each other in the graph network and point to the same material property entity or the same disease type entity through the initial attribute association.

[0023] For the spatial location entities within the group, create a new aggregate entity and integrate the aggregate entity into the knowledge graph of the target rammed earth artifact. The aggregate entity is used to represent a continuous physical region composed of the spatial location entities within the group.

[0024] Optionally, before mapping the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in the pre-constructed graph network, the method further includes:

[0025] A graph network is constructed, comprising multiple nodes and edges connecting the nodes, wherein each node corresponds to a spatial region on the surface of the artifact, and the edges are constructed based on the adjacency relationships and feature similarities between the spatial regions.

[0026] Optionally, the step of performing spectral analysis on the multispectral image data based on a pre-constructed standard reflectance spectral database of rammed earth materials to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic includes:

[0027] For any spatial region in the multispectral image data, extract the spectral response sequence of the spatial region;

[0028] In a pre-constructed database of standard reflectance spectra for rammed earth materials, standard reflectance spectra with the highest similarity to the spectral response sequences of this spatial region are retrieved and identified.

[0029] If the highest similarity is higher than a preset similarity threshold, the rammed earth material corresponding to the standard reflectance spectrum is determined as the material composition characteristic of the spatial region.

[0030] Optionally, the step of associating the material composition features corresponding to each spatial region with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region includes:

[0031] For each spatial region in the multispectral image data, the material composition features corresponding to the spatial region are encoded to generate a material feature vector for the spatial region;

[0032] In the texture image data, a neighborhood centered on the spatial region is determined;

[0033] The distribution of texture values ​​within the neighborhood is statistically analyzed to generate the shape feature vector of the spatial region.

[0034] Optionally, the step of mapping the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in a pre-constructed graph network, and performing iterative propagation and aggregation of node features on the graph network based on the initial features of the nodes and the connection relationships of the edges to obtain the updated aggregated features of each node, includes:

[0035] For each node in the graph network, the material feature vector and shape feature vector of its corresponding spatial region are aggregated to generate the initial features of the node;

[0036] Based on the initial characteristics of each node, the following propagation and aggregation operations are repeated until a preset iteration termination condition is met:

[0037] For any target node in the graph network, obtain the current feature vectors of all neighboring nodes of the target node;

[0038] Aggregate the current feature vectors of all adjacent nodes to generate neighborhood aggregated features;

[0039] The current feature vector of the target node is combined with the neighborhood aggregated features to update the feature vector of the target node;

[0040] The node feature vector that is finally updated when the iteration termination condition is met will be used as the aggregated feature of the updated node.

[0041] Secondly, this application provides a tag-based multimodal cultural relic knowledge graph construction system, including:

[0042] The acquisition module is used to acquire multimodal data of the target rammed earth cultural relic. The multimodal data includes multispectral image data covering the surface of the cultural relic and texture image data corresponding to the surface texture of the cultural relic.

[0043] The determination module is used to perform spectral analysis on the multispectral image data based on a pre-constructed standard reflectance spectrum database of rammed earth materials, so as to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic.

[0044] The generation module is used to associate the material composition features corresponding to each spatial region with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region.

[0045] The processing module is used to map the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in a pre-constructed graph network, and to perform iterative propagation and aggregation of node features on the graph network based on the initial features of the nodes and the connection relationship of the edges to obtain the updated aggregated features of each node.

[0046] The generation module is also used to generate a composite label for the spatial region corresponding to the node based on the updated aggregation features of each node. The composite label contains the material properties, disease type and spatial location information of the spatial region.

[0047] A construction module is used to build a knowledge graph of the target rammed earth artifact based on all generated composite tags.

[0048] Thirdly, this application provides an electronic device, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor, used to execute the computer program to implement the steps of the tag-based multimodal cultural relic knowledge graph construction method as described in the first aspect above.

[0051] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the tag-based multimodal cultural relic knowledge graph construction method described in the first aspect above.

[0052] The tag-based multimodal cultural relic knowledge graph construction method provided in this application can comprehensively collect relevant information about the surface of cultural relics by acquiring multimodal data of multispectral and texture images of target rammed earth cultural relics, providing data support for subsequent analysis; by analyzing multispectral image data based on a pre-constructed standard reflectance spectrum database of rammed earth materials, it can accurately obtain the material composition characteristics of each spatial region of the cultural relic; by generating material and morphological feature vectors by associating material composition features with texture image data, it can integrate key feature information of cultural relics, facilitating subsequent processing; by mapping feature vectors to initial features of graph network nodes and performing iterative propagation and aggregation, it can mine the correlation between features and optimize feature expression; by generating composite tags containing material properties, disease types, and spatial locations based on updated aggregated features, it can accurately label key information of each region of the cultural relic; and by constructing a knowledge graph based on all composite tags, it can systematically integrate cultural relic information and achieve orderly information management.

[0053] Furthermore, for each node in the graph network, the updated aggregated features are input into a pre-defined classification model to parse out the material attribute category and the disease type category. Then, the spatial location information represented by the node is obtained, and finally, the three types of information are combined to generate a composite label. This step can further improve the accuracy and standardization of composite label generation, ensuring that the label information is complete and consistent with the actual situation of cultural relics, and providing more reliable label support for the high-quality construction of knowledge graphs. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a tag-based multimodal cultural relic knowledge graph construction method provided in this application embodiment;

[0056] Figure 2 A schematic diagram of feature mapping for a tag-based multimodal cultural relic knowledge graph construction method provided in this application embodiment;

[0057] Figure 3 A schematic diagram of graph network iterative propagation and aggregation for a tag-based multimodal cultural relic knowledge graph construction method provided in this application embodiment;

[0058] Figure 4 This is a schematic diagram illustrating the knowledge graph construction method for a tag-based multimodal cultural relic knowledge graph construction method provided in this application embodiment;

[0059] Figure 5 This is a schematic diagram of the structure of a tag-based multimodal cultural relic knowledge graph construction system provided in an embodiment of this application. Detailed Implementation

[0060] In the field of cultural relic knowledge graph construction, existing label-based multimodal construction methods have obvious limitations: they generate labels by simply matching data or extracting basic features, failing to deeply integrate key information such as the material and morphology of cultural relics, making it difficult to accurately capture the attribute and state relationships of different spatial areas on the surface of cultural relics. This results in one-sided and inaccurate label information, and the inability to explore the potential connections behind multimodal data. Ultimately, this leads to insufficient accuracy and information completeness in knowledge graph construction, making it difficult to meet the actual needs of cultural relic protection and research.

[0061] To address this issue, this application proposes a label-based multimodal cultural relic knowledge graph construction method. The core of this method is to generate accurate labels through multi-dimensional data processing and deep fusion. Specifically, multispectral and texture image data of the cultural relics are first acquired. Material composition characteristics are then analyzed using a standard reflectance spectral database. Next, texture data is correlated to generate material and morphological feature vectors. These vectors are then iteratively aggregated and optimized using a graph network to ultimately generate composite labels containing material properties, damage types, and spatial locations. Based on these composite labels, a knowledge graph is constructed. This method achieves deep fusion of key features of cultural relics and the mining of potential data associations. It accurately compensates for the shortcomings of existing methods, such as incomplete labeling and insufficient information integration, effectively improving the quality of cultural relic knowledge graph construction and providing more reliable information support for cultural relic-related work.

[0062] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] The core of this application is to provide a tag-based multimodal cultural relic knowledge graph construction method, and a flowchart of one specific implementation method is shown below. Figure 1 As shown, the method includes:

[0064] S101. Obtain multimodal data of the target rammed earth cultural relic.

[0065] The target rammed earth artifacts primarily refer to immovable rammed earth remains, such as the Western Xia imperial tombs. Multimodal data includes multispectral imagery covering the artifact's surface, as well as texture imagery corresponding to the artifact's surface texture. Multispectral imagery includes images reflecting the reflectivity of different wavelengths on the artifact's surface, showcasing the characteristics of different materials. Texture imagery is used to clearly present details such as cracks, flaking marks, and weathering on the artifact's surface.

[0066] In one specific implementation, a drone equipped with a multispectral camera is used to take all-around aerial photos of the rammed earth cultural relics, capturing reflection information of different wavelengths on the surface of the relics to form multispectral image data; at the same time, a drone equipped with a high-definition camera is used to take close-up photogrammetry data, with the flight route planned in advance, and the appropriate flight altitude and shooting angle controlled, to take dense photos of the surface of the cultural relics at a high resolution of centimeters to millimeters, and obtain texture image data that can accurately reproduce the surface texture.

[0067] S102. Based on a pre-constructed standard reflectance spectrum database of rammed earth materials, perform spectral analysis on multispectral image data to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic.

[0068] The rammed earth material standard reflectance spectrum database is a collection storing reflectance characteristic data of different types of rammed earth materials in various multispectral bands. It includes standard reflectance data for various common rammed earth materials such as healthy dry rammed earth, salt-enriched rammed earth, high-moisture rammed earth, and biofilm-covered rammed earth. This data can serve as a reference for material composition identification. Material composition characteristics are used to clarify the material properties of different spatial areas on the surface of cultural relics, such as salt enrichment status, moisture content distribution, and weathering degree.

[0069] Optionally, step S102 may specifically include the following steps:

[0070] S1021. For any spatial region in the multispectral image data, extract the spectral response sequence of the spatial region.

[0071] The spectral response sequence refers to an ordered data combination of reflectance values ​​of a specific spatial region in different bands of a multispectral image, reflecting the reflectance characteristics of the material in that region to light of different bands. The spatial region can be a single pixel in the multispectral image data or a set of pixels, depending on the requirements.

[0072] In one specific implementation, the multispectral image is first divided into several independent spatial regions according to a preset spatial resolution, with each spatial region corresponding to a pixel or pixel block in the image. Subsequently, image processing techniques are used to extract the reflection intensity values ​​of each spatial region in each band of the multispectral image, such as the visible light band and the near-infrared band. These values ​​are then arranged in band order to obtain the spectral response sequence of the spatial region.

[0073] For example, continuing the above example, the multispectral image of a rammed earth cultural relic is divided into 10,000 spatial regions. One region located in the center of the relic is selected, and its reflectance values ​​in four bands are extracted as 0.2, 0.4, 0.6, and 0.8. Arranged in band order, the spectral response sequence of this region is [0.2, 0.4, 0.6, 0.8]. The above example is merely one illustration of this application. In practical applications, the method of dividing the spatial regions can be adjusted according to image resolution and analysis requirements; this application does not limit this.

[0074] S1022. In the pre-constructed standard reflectance spectrum database of rammed earth materials, retrieve and determine the standard reflectance spectrum that has the highest similarity to the spectral response sequence of the spatial region.

[0075] Among them, the standard reflectance spectrum refers to the typical spectral response sequences of various known rammed earth materials stored in the standard reflectance spectrum database of rammed earth materials, which serves as a reference benchmark for determining the material composition of unknown spatial regions.

[0076] In one specific implementation, the cosine similarity algorithm is first used to calculate the similarity between the spectral response sequence of the spatial region to be identified and the standard reflectance spectra in the database. The core of the cosine similarity algorithm is to measure the degree of similarity between two vectors by calculating the cosine of the angle between them, as shown in formula (1) below:

[0077] (1)

[0078] In equation (1), The cosine similarity value ranges from 1 to 10. Between these values, the closer the value is to 1, the higher the similarity. This represents the number of bands in the spectrum. The first spectral response sequence of the spatial region to be identified Reflectance values ​​for each band; For a certain standard reflectance spectrum in the database The reflection values ​​of each band.

[0079] After the calculation is completed, all similarity values ​​are compared, and the standard reflectance spectrum corresponding to the highest similarity value is selected. This spectrum is the standard reflectance spectrum with the highest similarity to the spectral response sequence of the spatial region to be identified.

[0080] In another specific implementation, the Euclidean distance algorithm can also be used to calculate the similarity. The smaller the Euclidean distance, the higher the similarity. The formula (2) is as follows:

[0081] (2)

[0082] In equation (2), The distance is Euclidean. This represents the number of bands in the spectrum. The first spectral response sequence of the spatial region to be identified Reflectance values ​​for each band; For a certain standard reflectance spectrum in the database The reflection values ​​of each band.

[0083] In practical applications, continuing the example from the above steps, the spectral response sequence of this spatial region is [0.2, 0.4, 0.6, 0.8]. The database contains the standard reflectance spectra of three materials: healthy dry rammed earth, salt-enriched rammed earth, and high-moisture-content rammed earth, which are [0.22, 0.42, 0.62, 0.82], [0.5, 0.7, 0.9, 0.95], and [0.7, 0.5, 0.3, 0.1], respectively.

[0084] Similarity is calculated using equation (1):

[0085] Similarity between the spectral response sequence of this spatial region and the standard spectrum of healthy, dry rammed earth:

[0086] ;

[0087] Similarity between the spectral response sequence of this spatial region and the standard spectrum of salt-enriched rammed soil:

[0088] ;

[0089] Similarity between the spectral response sequence of this spatial region and the standard spectrum of rammed soil with high water content:

[0090] .

[0091] Comparing the three similarity values ​​of 0.982, 1.008, and 0.597, the highest similarity is 1.008. Values ​​greater than 1 are assigned a value of 1, corresponding to the standard reflectance spectrum of salt-enriched rammed soil. The above example is merely one illustration of this application. In practical applications, other similarity calculation algorithms can be selected based on data characteristics, and this application does not limit such selection.

[0092] S1023. If the highest similarity is higher than the preset similarity threshold, the rammed earth material corresponding to the standard reflectance spectrum is determined as the material composition characteristic of the spatial region.

[0093] The preset similarity threshold is the critical value for determining whether the spectrum of the spatial region to be identified matches the standard reflectance spectrum, and it is set according to the actual recognition accuracy requirements.

[0094] In this embodiment, firstly, based on the results of numerous spectral matching experiments on rammed earth materials and considering the requirements for material identification accuracy in the field of cultural relic protection, a preset similarity threshold of 0.95 is set. Then, the highest similarity obtained in S1022 is compared with this threshold. If the highest similarity is greater than 0.95, the material in the spatial region is determined to be consistent with the rammed earth material corresponding to the standard reflectance spectrum, and the rammed earth material corresponding to the standard reflectance spectrum is identified as the material composition characteristic of the spatial region. If the highest similarity is less than or equal to 0.95, the matching is deemed to have failed, and the spectral data needs to be re-examined or the standard reflectance spectrum in the database needs to be supplemented.

[0095] This application achieves rapid and accurate identification of the material composition on the surface of rammed earth cultural relics through a standardized spectral database and a precise similarity calculation algorithm, avoiding the subjectivity and limitations of traditional manual testing.

[0096] S103. Associate the material composition features corresponding to each spatial region with the corresponding texture image data to generate the material feature vector and morphology feature vector corresponding to each spatial region.

[0097] In this step, the material feature vector is an ordered combination of data formed by digitally encoding the material composition features, used to quantitatively characterize the material properties of a spatial region; the morphology feature vector is a data vector generated based on the texture distribution information of the spatial region and its surroundings in the texture image, used to reflect the surface structure state of the spatial region; the two together constitute a comprehensive feature description of the spatial region, realizing the correlation and integration of material properties and surface morphology information.

[0098] Optionally, step S103 may specifically include the following steps:

[0099] S1031. For each spatial region in the multispectral image data, the material composition features corresponding to the spatial region are encoded to generate a material feature vector of the spatial region.

[0100] Encoding is the process of converting non-digital material composition characteristics into digital forms that can be recognized and processed by computers. It establishes a correspondence between material composition characteristics and digital vectors through preset encoding rules.

[0101] In one specific implementation, all possible material composition feature types are first identified, such as healthy dry rammed earth, salt-enriched rammed earth, high-moisture-content rammed earth, and biofilm-covered rammed earth, and a unique numerical identifier is assigned to each feature type. Then, a one-hot encoding method is used to convert the material composition features of each spatial region into a corresponding binary vector. The vector length is equal to the total number of material composition feature types, with only the position corresponding to the feature type set to 1, and the remaining positions set to 0, thus generating the material feature vector for that spatial region.

[0102] In another specific implementation, a numerical encoding method can also be used, assigning different values ​​based on the relevant properties of the material composition, such as moisture content and salt content, and arranging these values ​​in a preset order to form a material feature vector.

[0103] For example, suppose there are four types of material composition characteristics: healthy dry rammed soil (number 1), salt-enriched rammed soil (number 2), high-moisture-content rammed soil (number 3), and biofilm-covered rammed soil (number 4). Using a unique thermal coding rule, if the material composition characteristic of a certain spatial region is salt-enriched rammed soil, the corresponding material feature vector is [0,1,0,0]; if the material composition characteristic is healthy dry rammed soil, the material feature vector is [1,0,0,0]. The above example is only one example of this application. In practical applications, the coding method can be adjusted according to the number and type of material composition characteristics, and this application does not limit this.

[0104] S1032. In the texture image data, determine a neighborhood centered on the spatial region.

[0105] The neighborhood refers to a certain range of surrounding areas defined in the texture image with the target spatial region as the center. It is used to comprehensively consider the texture information of the target region and its surroundings, and to more comprehensively reflect the surface morphology features of the target region.

[0106] In one specific implementation, the size parameter of the neighborhood is set according to the resolution of the texture image and the size of the spatial region. If the spatial region corresponds to a pixel in the texture image, the neighborhood can be set as a 3×3 or 5×5 pixel matrix, that is, with the target pixel as the center, it includes the pixels adjacent to it in the top, bottom, left, right and diagonal directions. The position of the target spatial region in the texture image is determined by coordinate positioning, and then the corresponding range centered on that position is selected according to the set neighborhood size, which is the neighborhood of the target spatial region.

[0107] For example, continuing the previous example, assuming a spatial region is represented by pixels, let's take the pixel coordinates of a certain spatial region in the texture image as (50, 50), and set a neighborhood matrix of size 3×3. Centered on (50, 50), select the nine pixels corresponding to coordinates (49, 49), (49, 50), (49, 51), (50, 49), (50, 50), (50, 51), (51, 49), (51, 50), and (51, 51) to form the neighborhood of this spatial region. If the spatial region is a region composed of multiple pixels, the neighborhood of that spatial region can also be determined in the same way.

[0108] S1033. Statistically analyze the distribution of texture values ​​in the neighborhood to generate a shape feature vector of the spatial region.

[0109] Among them, texture value distribution refers to the value and distribution pattern of texture parameters corresponding to each pixel in the neighborhood, such as gray value, contrast, roughness, etc., which is the core basis for generating shape feature vector.

[0110] In one specific implementation, firstly, the grayscale value of each pixel within the neighborhood is extracted as a texture value. Then, statistical characteristics of the grayscale values ​​within the neighborhood are calculated, including the mean, variance, maximum value, minimum value, and median. The mean value reflects the overall brightness of the texture within the neighborhood and is obtained by summing the grayscale values ​​of all pixels and dividing by the total number of pixels. The variance reflects the dispersion of the grayscale values ​​within the neighborhood, i.e., the roughness of the texture, and is obtained by averaging the squared differences between each pixel's grayscale value and the mean value. These statistical characteristics are arranged in a preset order to form a morphological feature vector for the spatial region, achieving a quantitative description of the surface morphology of the spatial region.

[0111] For example, continuing the example from the steps above, suppose the gray values ​​of 9 pixels in a 3×3 neighborhood of a certain spatial region are [120, 125, 122, 123, 121, 124, 126, 123, 122]. Next, calculate the average: add the 9 gray values ​​together to get a total of 1096, then divide by the total number of pixels, 9, that is, 1096 ÷ 9 ≈ 121.78.

[0112] Then calculate the variance: First, calculate the squared difference between each gray value and the average value of 121.78, which are respectively... , , , , , , , , Sum the squares of these differences to get a total of 39.968, then divide by the total number of pixels, 9, which gives... .

[0113] Simultaneously, the maximum grayscale value within this neighborhood is determined to be 126, the minimum to be 120, and the median to be 123. Arranging these values ​​in the order of "mean, variance, maximum, minimum, median," the morphological feature vector for this spatial region is obtained as [121.78, 4.44, 126, 120, 123]. The above example is merely one illustration of this application; in practical applications, other texture parameters or statistical features can be selected to generate the morphological feature vector, and this application does not impose any limitations on this.

[0114] This application achieves an effective correlation between material composition features and texture information through coding technology and neighborhood statistical analysis. The generated feature vector can comprehensively and quantitatively describe the material and morphological attributes of a spatial region, overcoming the limitations of traditional single feature description and improving the completeness and usability of feature information.

[0115] S104. Map the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in the pre-constructed graph network. Based on the initial features of the nodes and the connection relationship of the edges, iteratively propagate and aggregate the node features on the graph network to obtain the updated aggregated features of each node.

[0116] In this step, the graph network is a structured network model composed of nodes and edges. Nodes correspond to spatial regions on the surface of the artifact, and edges represent the relationships between nodes, such as geographical proximity or similar features. Initial features are the basic feature data of the nodes, generated by aggregating the material feature vectors and morphological feature vectors of the corresponding spatial regions. Iterative propagation and aggregation refer to the process in the graph network where nodes continuously receive feature information from neighboring nodes through edge connections and fuse and update it, gradually improving their own features. Aggregated features are the feature vectors finally formed after multiple iterations of propagation and aggregation, which can comprehensively reflect the feature information of the node itself and its surrounding related nodes. Specifically, such as... Figure 2 As shown, the surface of the artifact is pre-divided into spatial regions, resulting in spatial region 1, spatial region 2, spatial region 3, and spatial region 4. It should be noted that the above four spatial region division results are only examples, and in reality, it can be divided into N spatial regions. Through the calculation process of steps 101 to 103 above, the material feature vector and morphological feature vector corresponding to each spatial region can be determined. For example, spatial region 1 corresponds to material feature vector A1 and morphological feature vector B1.

[0117] By mapping spatial regions to nodes in a graph network, multiple vectors corresponding to the spatial regions are mapped to the initial features of the nodes. For example... Figure 2 As shown, spatial region 1 is mapped to node 1 in the graph network, and the material feature vector A1 and shape feature vector B1 corresponding to spatial region 1 are mapped to the initial feature C1 of node 1.

[0118] Furthermore, based on the initial features of the nodes and the connectivity of the edges, iterative propagation and aggregation of node features are performed on the graph network to obtain the updated aggregated features of each node. For example... Figure 3 As shown, after iterative propagation and aggregation of the initial feature C1 of node 1, an aggregated feature is obtained, which is the aggregation result of the initial feature C1 and the current feature vectors of all neighboring nodes. The following is the specific implementation process of step S104:

[0119] Optionally, step S104 may specifically include the following steps:

[0120] S1041. For each node in the graph network, aggregate the material feature vector and shape feature vector of its corresponding spatial region to generate the initial features of the node.

[0121] Aggregation is the process of combining material feature vectors and shape feature vectors, two types of feature data with different dimensions, to form comprehensive feature data with a unified dimension, used to fully characterize the spatial region attributes corresponding to a node. In one specific implementation, aggregation is performed using vector concatenation, where the material feature vector and shape feature vector corresponding to each node are directly concatenated in sequence to form a longer vector as the initial feature of that node. Vector concatenation does not change the numerical values ​​of the original feature data; feature fusion is achieved only through dimensional expansion. This method is simple to operate and completely preserves the original information of both types of features.

[0122] In another specific implementation, a weighted summation method can also be used for aggregation. Weights are assigned according to the importance of material features and shape features, and the values ​​of the corresponding dimensions of the two types of vectors are weighted and summed to obtain the initial feature vector.

[0123] For example, continuing the previous example, the material feature vector corresponding to a certain node is [0,1,0,0], and the shape feature vector is [121.78,4.44,126,120,123]. Using vector concatenation, the two vectors are joined in the order of "material feature vector first, shape feature vector second," resulting in the initial features of the node being [0,1,0,0,121.78,4.44,126,120,123].

[0124] S1042. Based on the initial characteristics of each node, repeat the following propagation and aggregation operations until the preset iteration termination condition is met.

[0125] Among them, the iteration termination condition is the basis for determining whether the iteration propagation and aggregation operation should stop. It is used to control the number of iterations or ensure that the feature update reaches a stable state. Common conditions include setting a fixed number of iterations or the change in node features between two adjacent iterations being less than a set threshold.

[0126] Specifically, a fixed number of iterations is preset as the termination condition. Based on the number and complexity of the spatial regions on the surface of the cultural relic, the number of iterations is set to 3-5. When the preset number of iterations is reached, the propagation and aggregation operations stop, and the feature vector of the node at this point is the updated aggregated feature.

[0127] In another specific implementation, the termination condition is that the change in node features is less than a threshold. The average difference between the feature vectors of all nodes after each iteration and the feature vectors of the previous iteration is calculated. When the average difference is less than a preset threshold, it indicates that the feature update has stabilized, and the iteration operation is stopped.

[0128] For example, for a graph network with 1000 nodes, considering the relatively simple node relationships, the preset iteration termination condition is a fixed 3 iterations. After completing 3 rounds of propagation and aggregation operations, the iteration stops and the feature vectors of each node are retained. The above example is only one example of this application. In practical applications, the iteration termination condition can be adjusted according to the actual scenario, and this application does not limit it in this regard.

[0129] S1043. For any target node in the graph network, obtain the current feature vectors of all neighboring nodes of the target node.

[0130] Here, the target node refers to the node currently undergoing feature updates; the adjacent node refers to the node directly connected to the target node through an edge, and its current feature vector refers to the feature vector of the adjacent node that has not yet been updated in this iteration, or the initial feature vector in the first iteration.

[0131] Specifically, an adjacency matrix for the graph network is pre-constructed. The values ​​of the elements in the adjacency matrix are used to identify whether there is an adjacency relationship between nodes. A value of 1 indicates that the two corresponding nodes are adjacent nodes, while a value of 0 indicates that they are not adjacent nodes. For any target node, by querying the elements of the corresponding row or column in the adjacency matrix, the nodes corresponding to the positions with values ​​of 1 are selected, which are all the adjacent nodes of the target node. Then, the current feature vectors of these adjacent nodes are extracted from the feature storage container.

[0132] For example, continuing the previous example, let the target node be node 2, and its adjacent nodes in the graph network be nodes 1, 3, and 4. After confirming this adjacency relationship by querying the adjacency matrix, the current feature vectors of nodes 1, 3, and 4 in this iteration are extracted from the feature storage container, which are [1,0,0,0,120.50,3.20,125,118,122], [0,0,1,0,122.30,5.10,127,121,124], and [0,0,0,1,121.10,4.00,124,119,121], respectively. The above example is only one example of this application. In practical applications, adjacent nodes can be queried and feature vectors can be obtained in other ways, and this application does not limit this.

[0133] S1044. Aggregate the current feature vectors of all the obtained neighboring nodes to generate neighborhood aggregation features.

[0134] Among them, the neighborhood aggregation feature is a feature vector obtained by comprehensively processing the current feature vectors of all adjacent nodes, which is used to centrally reflect the overall feature state of the neighborhood surrounding the target node.

[0135] In one specific implementation, mean aggregation is used to generate neighborhood aggregation features. The values ​​of the corresponding dimensions of the current feature vectors of all neighboring nodes are summed, and then divided by the number of neighboring nodes to obtain the average value for each dimension. These average values, arranged in order, form the neighborhood aggregation features. Mean aggregation can evenly reflect the feature influence of all neighboring nodes, is computationally simple, and has strong stability.

[0136] For example, continuing the example in the above steps, the current feature vectors of the three neighboring nodes of the target node 2 are as follows: Node 1: [1,0,0,0,120.50,3.20,125,118,122], Node 3: [0,0,1,0,122.30,5.10,127,121,124], Node 4: [0,0,0,1,121.10,4.00,124,119,121].

[0137] Calculate the average values ​​for each dimension: Dimension 1: (1+0+0)÷3≈0.33; Dimension 2: (0+0+0)÷3=0; Dimension 3: (0+1+0)÷3≈0.33; Dimension 4: (0+0+1)÷3≈0.33; Dimension 5: (120.50+122.30+121.10)÷3=363.90÷3=121.30; Dimension 6: Dimension 1: (3.20+5.10+4.00)÷3=12.30÷3=4.10; Dimension 2: (125+127+124)÷3=376÷3≈125.33; Dimension 3: (118+121+119)÷3=358÷3≈119.33; Dimension 4: (122+124+121)÷3=367÷3≈122.33.

[0138] Then, the average values ​​of each dimension are arranged in order to obtain the neighborhood aggregation feature [0.33,0,0.33,0.33,121.30,4.10,125.33,119.33,122.33]. The above example is only one example of this application. In practical applications, other methods such as summation aggregation and maximum value aggregation can also be used to generate neighborhood aggregation features, and this application does not limit this.

[0139] S1045. Combine the current feature vector of the target node with the neighborhood aggregation features to update the feature vector of the target node.

[0140] The process of combining the target node’s current feature vector with the aggregated features of its neighborhood to generate a new feature vector that is more comprehensive and better reflects the actual state of the node is the core step in updating node features.

[0141] In one specific implementation, a weighted fusion method is used to combine the features. Weights are assigned to the current feature vector of the target node and the aggregated features of its neighborhood, with the sum of these weights being 1. The values ​​of the corresponding dimensions of the two types of feature vectors are multiplied by their respective weights and then summed to obtain the updated feature vector of the target node. By adjusting the weights, the influence of the node's own features and neighborhood features on the update result can be controlled. Typically, the weight of the target node's own features is set higher than the weight of the aggregated features of its neighborhood to ensure the dominant position of the node's own attributes.

[0142] The fusion formula (3) is as follows:

[0143] (3)

[0144] In equation (3), The updated feature vector for the target node; The weight of the current feature vector of the target node, with a value between 0 and 1; This is the current feature vector of the target node; The weights of the neighborhood aggregation features; This is a neighborhood aggregation feature.

[0145] For example, continuing the previous example, the current feature vector of target node 2 for Neighborhood aggregation features for Set weights ,but .

[0146] Calculate the updated values ​​for each dimension according to formula (3): Dimension 1: 0×0.7+0.33×0.3≈0+0.10=0.10; Dimension 2: 1×0.7+0×0.3=0.7+0=0.7; Dimension 3: 0×0.7+0.33×0.3≈0+0.10=0.10; Dimension 4: 0×0.7+0.33×0.3≈0+0.10=0.10; Dimension 5: 121.78×0.7+121.30×0.3≈85.25+3 6.39 = 121.64; 6th dimension: 4.44 × 0.7 + 4.10 × 0.3 ≈ 3.11 + 1.23 = 4.34; 7th dimension: 126 × 0.7 + 125.33 × 0.3 ≈ 88.2 + 37.60 = 125.80; 8th dimension: 120 × 0.7 + 119.33 × 0.3 ≈ 84 + 35.80 = 119.80; 9th dimension: 123 × 0.7 + 122.33 × 0.3 ≈ 86.1 + 36.70 = 122.80.

[0147] The updated feature vector of target node 2 is obtained as [0.10, 0.7, 0.10, 0.10, 121.64, 4.34, 125.80, 119.80, 122.80]. The above example is only one example of this application. In practical applications, other combination methods such as direct concatenation and element-wise addition can also be used, and this application does not limit them.

[0148] S1046, and when the iteration termination condition is met, the finally updated node feature vector is used as the aggregated feature after the node update.

[0149] Among them, the aggregated feature is the final feature vector formed after a node has gone through a complete iterative propagation and aggregation process. It integrates the node's own initial features and the feature information transmitted by neighboring nodes during the iteration process, and can more comprehensively and accurately characterize the spatial region state and surrounding influences of the node.

[0150] In this embodiment, operations S1043 to S1045 are continuously executed according to a preset iteration termination condition. If the termination condition is a fixed number of iterations, after completing the preset number of iterations, the feature vector of each node after the last update is recorded as the aggregated feature of that node; if the termination condition is that the feature change is less than a threshold, the update change of the feature vectors of all nodes is calculated after each iteration. When the change meets the threshold requirement, the iteration stops and the feature vectors of each node at this time are saved as the aggregated feature.

[0151] For example, continuing the previous example, the preset iteration termination condition is a fixed 3 iterations. After the first iteration, the feature vector of target node 2 is updated to [0.10,0.7,0.10,0.10,121.64,4.34,125.80,119.80,122.80]; in the second iteration, the neighborhood aggregation feature is recalculated and fused based on the updated neighbor node features, resulting in the updated feature vector of [0.15,0.5,0.20,0.15,121.52,4.28,125.50,119.60,122.60]; after the third iteration, the final updated feature vector is [0.18,0.4,0.22,0.20,121.45,4.25,125.30,119.50,122.50]. This vector is the aggregated feature of target node 2 after its update, which integrates the feature information of itself and its neighboring nodes in the three iterations.

[0152] This application achieves deep integration of node characteristics and neighborhood association characteristics through graph network modeling and iterative propagation aggregation technology, overcoming the limitations of traditional single node feature analysis. It can more comprehensively capture the correlation influence of the surface spatial area of ​​cultural relics and improve the accuracy and completeness of feature representation.

[0153] Optionally, before step S104, which involves "mapping the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in the pre-constructed graph network", the method further includes:

[0154] A graph network is constructed, consisting of multiple nodes and edges connecting the nodes. Each node corresponds to a spatial region on the surface of the artifact, and the edges are constructed based on the adjacency relationship and feature similarity between the spatial regions.

[0155] Among them, adjacency relationship refers to the geographical correlation of different spatial regions on the surface of cultural relics, and adjacent spatial regions have an adjacency relationship; feature similarity refers to the degree of similarity between the material feature vector and the morphological feature vector of different spatial regions, and the higher the degree of similarity, the stronger the feature similarity; edge is the link connecting nodes in the graph network, used to characterize the association between the spatial regions corresponding to the nodes.

[0156] Specifically, firstly, based on the high-resolution 3D model of the cultural relic, the surface of the relic is divided into multiple independent spatial regions at a preset resolution, with each spatial region corresponding to a node in a graph network. Then, the adjacency relationship between the spatial regions is determined. If two spatial regions are geographically adjacent, they are preliminarily determined to be connectable. Next, the similarity of the feature vectors of these two spatial regions is calculated. If the similarity is higher than a preset threshold, an edge is constructed between the corresponding nodes. If only the adjacency relationship is satisfied but the feature similarity does not meet the threshold, an edge can also be constructed as needed to fully represent the spatial relationships.

[0157] In another specific implementation, the feature similarity between all spatial regions can be calculated first, and region pairs with similarity higher than the threshold can be selected. Then, based on their adjacency relationship, edges can be constructed only for nodes corresponding to regions that are both adjacent and feature similar, highlighting the core association.

[0158] S105. Based on the updated aggregation features of each node, generate a composite label for the spatial region corresponding to the node. The composite label contains the material properties, disease type, and spatial location information of the spatial region.

[0159] Optionally, step S105 may specifically include the following steps:

[0160] Among them, the composite label is a comprehensive identifier that integrates key information from multiple dimensions of a spatial area, and can intuitively and centrally present the core state of the spatial area; the material properties are the material composition and related characteristics of the spatial area, such as the degree of salt enrichment and moisture content; the damage type is the form of damage existing in the spatial area, including categories such as sheet erosion, scouring, cracks, gullies, and biological damage; the spatial location information is the specific geographical coordinates or relative position description of the spatial area on the surface of the cultural relic.

[0161] S1051. For each node in the graph network, the updated aggregated features of the node are input into a preset classification model to parse and obtain the material property category and disease type category corresponding to the node.

[0162] The preset classification model is a pre-trained algorithm model used to identify the category information corresponding to the feature vector. It can extract key information from the aggregated features and match them to the corresponding material property category and disease type category.

[0163] In this embodiment, the classification model is constructed using the Support Vector Machine (SVM) algorithm. The model training process is as follows: First, a large number of aggregated feature samples labeled with material property categories and disease type categories are collected, and the samples are divided into training sets and test sets. Then, the SVM model is trained using the training set data. By adjusting the model parameters, the classification boundary is optimized so that the model can accurately learn the correspondence between features and categories. Finally, the model performance is verified using the test set data. When the classification accuracy reaches the preset standard, the model training is completed and the model is put into use.

[0164] The updated aggregated features are input into the classification model. The model calculates the distance between the feature vector and the classification boundary of each category to determine the material property category and disease type category that best matches the feature vector, thus completing the analysis.

[0165] In another specific implementation, the random forest algorithm can be used to construct the classification model, and the reliability of the classification results can be improved by ensemble decision-making of multiple decision trees.

[0166] For example, continuing the previous example, the updated aggregated features of a certain node are [0.18, 0.4, 0.22, 0.20, 121.45, 4.25, 125.30, 119.50, 122.50]. This feature vector is input into a trained support vector machine classification model. After parsing, the model outputs the material property category as moderately saline rammed soil and the disease type category as a potential high-risk area for erosion. The above example is merely one illustration of this application. In practical applications, other algorithms can be selected to construct models according to classification requirements, and this application does not limit this.

[0167] S1052. Obtain the spatial location information represented by the node.

[0168] Among them, spatial location information is the specific location data of the spatial region corresponding to the node in the three-dimensional model of the cultural relic, which can clearly define the location and coordinate range of the region on the surface of the cultural relic.

[0169] In one specific implementation, relying on the high-precision 3D model of the cultural relic constructed in the early stage, each node in the model corresponds to a unique 3D coordinate information (X, Y, Z). By querying the mapping relationship table between the node and the 3D model coordinates, the 3D coordinates corresponding to the node are directly extracted as spatial location information; at the same time, combined with the overall structural division of the cultural relic, the 3D coordinates are converted into a more easily understood relative position description, such as the middle of the northeast side of the tower body, the west side of the bottom, etc.

[0170] For example, continuing the previous example, by querying the mapping table corresponding to the target node, its three-dimensional coordinates are extracted as (35.2, 42.6, 12.8). Combining this with the overall structure of the cultural relic, these coordinates are converted into a relative position describing the center of the northeast side of the tower. Finally, the spatial location information of this node is determined as three-dimensional coordinates (35.2, 42.6, 12.8) and a relative position of the center of the northeast side of the tower. The above example is merely one illustration of this application. In practical applications, different methods of representing location information can be selected according to the positioning accuracy requirements, and this application does not limit this.

[0171] S1053. Combine the parsed material attribute categories, disease type categories, and extracted spatial location information to generate composite labels corresponding to the nodes.

[0172] The combination process involves integrating three different dimensions of information—material attribute category, disease type category, and spatial location information—into a pre-defined format to form a composite label with a unified structure and complete information.

[0173] In one specific implementation, the preset composite label combination format is "relative location - material attribute category - disease type category". Following this format, the extracted relative location information, the parsed material attribute category, and the disease type category are sequentially concatenated to generate the composite label. This format is concise and clear, quickly presenting the core information of a spatial area, facilitating review and use in subsequent conservation work.

[0174] This application generates composite tags by combining classification model analysis with multi-dimensional information. These tags can comprehensively and accurately integrate key information of a spatial region, overcoming the problems of single and fragmented information in traditional tags. They achieve centralized and intuitive presentation of information, improving the convenience and effectiveness of information use.

[0175] S106. Based on all generated composite tags, construct a knowledge graph of the target rammed earth cultural relic.

[0176] Among them, a knowledge graph is a structured information display model that uses entities as the core and relationships as the link to intuitively present the logical connections between different information. The preset knowledge graph framework is a pre-built empty structure model that includes basic settings such as entity category definition and relationship type classification. Entities are the basic units that carry information in the knowledge graph, corresponding to specific content such as material properties, disease types, and spatial location information. Initial attribute relationships are the basic relationships established between entities based on composite tag logic, used to clarify the correspondence between different types of entities. Optionally, step S106 may specifically include the following steps:

[0177] S1061. Create or match corresponding entities in the preset knowledge graph framework for the material properties, disease types and spatial location information parsed from the composite tags.

[0178] Among them, creating an entity means adding a brand new entity unit when the parsed information has no corresponding record in the knowledge graph framework; matching an entity means directly associating the existing entity with the parsed information that already has a corresponding record in the knowledge graph framework, thus avoiding duplicate creation.

[0179] In one specific implementation, all composite tags are first parsed in batches to extract material properties, disease type, and spatial location information from each tag. Then, for each type of information, it is compared with existing entities under the corresponding category in a preset knowledge graph framework. The comparison uses a string similarity matching algorithm to calculate the similarity between the parsed information and the name of an existing entity. When the similarity is higher than a preset threshold, it is determined to be a successful match, and the existing entity is directly associated; when the similarity is lower than the preset threshold, it is determined that there is no corresponding entity, a new entity is created under the corresponding category, and the basic description information of the entity is improved.

[0180] For example, continuing the previous example, the information parsed from the composite tag includes the material property of moderately saline rammed earth, the disease type of potential high-risk erosion development zone, and the spatial location of the northeast center of the tower. The material property of moderately saline rammed earth is compared with existing entities under the material property category in the framework. If no entity with a similarity higher than 0.9 is found, a new entity, moderately saline rammed earth, is created. If the disease type of potential high-risk erosion development zone already exists under the disease type category, the existing entity is directly matched. Since there is no corresponding entity for the spatial location of the northeast center of the tower, a new entity, the northeast center of the tower, is created. The above example is only one example of this application. In practical applications, the comparison algorithm and threshold can be adjusted according to the information type, and this application does not limit this.

[0181] S1062. Establish initial attribute associations between entities representing spatial location information and entities representing material properties and disease types.

[0182] Among them, the initial attribute association is based on the inherent logic of composite tags, which clarifies the attribution relationship between spatial location entities and corresponding material attribute entities and disease type entities, and is the basic form of entity association in knowledge graphs.

[0183] In this embodiment, association rules are established for each spatial location entity based on the correspondence of composite tags. If a composite tag contains spatial location entity A, material attribute entity B, and disease type entity C, then an initial attribute association is established between the three: "Spatial location A has material attribute B" and "Spatial location A has disease type C". After the association is established, it is presented in the knowledge graph as directed edges, with the edge labels indicating the association type, such as "has" or "exists", making the association relationship clear and identifiable.

[0184] For example, continuing the example from the above steps, the spatial location entity is the central part of the northeast side of the tower, the corresponding material attribute entity is moderately saline rammed earth, and the disease type entity is a potential high-risk development zone for erosion. Based on the logic of composite tags, an initial attribute association is established between "the central part of the northeast side of the tower has moderately saline rammed earth" and "the central part of the northeast side of the tower has a potential high-risk development zone for erosion." In the knowledge graph, the corresponding entities are connected by directed edges labeled "has" and "exists." The above example is only one example of this application. In practical applications, more association types can be defined according to the complexity of information association, and this application does not limit this.

[0185] S1063. Integrate all created or matched entities and their initial attribute associations to complete the construction of the knowledge graph of the target rammed earth cultural relic.

[0186] Integration is the process of organizing and sorting out scattered entities and initial attributes according to a preset knowledge graph framework to form a logically coherent and structurally complete knowledge graph. This includes operations such as entity classification, association verification, and removal of redundant information.

[0187] In one specific implementation, all created or matched entities are first categorized into three types: material properties, disease type, and spatial location, and then assigned to the corresponding entity sets within the knowledge graph framework. Next, all initial attribute associations are validated to check their rationality and completeness, eliminating logically contradictory or duplicate relationships. Finally, entities and their relationships are presented graphically using visualization technology, with entities represented by nodes and relationships by directed edges, forming a complete knowledge graph that allows users to browse, query, and edit the graph.

[0188] For example, continuing the previous example, after the previous steps, we obtained multiple entities, including material properties such as moderately saline rammed soil and healthy dry rammed soil; disease types such as potential high-risk erosion development zones and sheet erosion; and spatial locations such as the northeast center of the tower and the northwest bottom of the tower, as well as their corresponding initial attribute associations. These entities are categorized into their corresponding sets, and after verifying that the associations are consistent, they are presented as a knowledge graph using visualization tools. Nodes are labeled with entity names, and edges are labeled with association types such as "has" or "exists." Users can intuitively see the association between the northeast center of the tower and the moderately saline rammed soil and the potential high-risk erosion development zone, as well as the associations between other spatial locations and their corresponding material properties and disease types.

[0189] For example, such as Figure 4 As shown, assuming that composite label 1 and composite label 2 are generated through the above steps, composite label 1 contains the material attribute "rammed earth A", the disease type "alkali efflorescence", and the spatial location "northeast upper-middle part", while composite label 2 contains the material attribute "rammed earth A", the disease type "alkali efflorescence", and the spatial location "northeast lower part". The entity material attributes "rammed earth A", disease type "alkali efflorescence", spatial location 1 (northeast upper-middle part), and spatial location 2 (northeast lower part) are parsed to obtain the entity material attributes "rammed earth A", "disease type "alkali efflorescence", and spatial location 1 (northeast upper-middle part) and 2 (northeast lower part) respectively. The relationships between the entities are marked according to the association between them: "has" and "exists".

[0190] The knowledge graph constructed in this application not only intuitively presents the core state of each area of ​​the cultural relic, but also clearly sorts out the logical connections between different areas. Compared with traditional manual recording and single-dimensional data statistics, it can more comprehensively restore the distribution pattern of cultural relic diseases and material properties, and provide more systematic and reliable decision support for risk prediction and restoration plan formulation in the preventive protection of cultural relics.

[0191] Optionally, after integrating all created or matched entities and established initial attribute associations to complete the construction of the knowledge graph of the target rammed earth artifact, the following steps are also included:

[0192] In the knowledge graph, a group of spatial entities is identified, where all spatial entities within this group are adjacent to each other in the graph network and point to the same material attribute entity or the same disease type entity through initial attribute association. A new aggregate entity is created for this group of spatial entities, and the aggregate entity is integrated into the knowledge graph of the target rammed earth artifact. The aggregate entity represents a continuous physical region constituted by this group of spatial entities.

[0193] In the above steps, the aggregated entity is a new entity created based on the common characteristics and adjacency relationships of a group of spatial entities. It is used to characterize the continuous physical area jointly formed by these spatial entities and can intuitively reflect the concentrated distribution of similar attributes or similar defects on the surface of cultural relics.

[0194] In one specific implementation, all spatial entities in the knowledge graph are first traversed. Based on the adjacency relationships in the graph network, groups of mutually adjacent spatial entities are selected. Next, the initial attribute associations of all entities within this group are checked to determine if they all point to the same material attribute entity or the same disease type entity. If both conditions are met, an aggregate entity is created for this group of spatial entities. The name must reflect the attribute characteristics of the continuous region, such as... Figure 4 As shown, the disease type of composite label 1 and composite label 2 is efflorescence. A new aggregate entity is generated, which is used to reflect the existence of continuous efflorescence regions. Finally, the aggregate entity is added to the knowledge graph, and the association relationship between the aggregate entity and the corresponding spatial location entity, material attribute entity or disease type entity is established to complete the integration.

[0195] By generating aggregated entities and supplementing them to the knowledge graph, it is beneficial to more comprehensively restore the distribution patterns and material properties of cultural relics' diseases, and provide more systematic and reliable decision support for risk prediction and restoration plan formulation in the preventive protection of cultural relics.

[0196] This application presents a label-based multimodal cultural relic knowledge graph construction method. Through a full-process technical approach encompassing multimodal data acquisition, feature fusion, graph network modeling, and knowledge graph construction, it achieves precise correlation and structured presentation of material properties, disease types, and spatial location information of rammed earth cultural relics. It overcomes the limitations of traditional manual surveys, addressing the issues of insufficient single data dimensions and fragmented information. Furthermore, through intelligent algorithms and network models, it uncovers the correlations and influences of surface areas on cultural relics, making disease identification and risk assessment more comprehensive and objective. Simultaneously, the structured knowledge graph makes the status information of cultural relics intuitive and easy to understand, providing clear decision-making basis for cultural relic protection work and promoting the transformation of preventive protection of ancient sites from experience-driven to data-driven approaches.

[0197] Figure 5 This is a schematic diagram illustrating a specific implementation of a tag-based multimodal cultural relic knowledge graph construction system provided in this application. (Refer to...) Figure 5 The system may include:

[0198] The acquisition module 51 is used to acquire multimodal data of the target rammed earth cultural relic. The multimodal data includes multispectral image data covering the surface of the cultural relic and texture image data corresponding to the surface texture of the cultural relic.

[0199] The determination module 52 is used to perform spectral analysis on multispectral image data based on a pre-built standard reflectance spectrum database of rammed earth materials, so as to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic.

[0200] The generation module 53 is used to associate the material composition features corresponding to each spatial region with the corresponding texture image data to generate the material feature vector and morphology feature vector corresponding to each spatial region.

[0201] The processing module 54 is used to map the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in the pre-constructed graph network, and to perform iterative propagation and aggregation of node features on the graph network based on the initial features of the nodes and the connection relationship of the edges to obtain the updated aggregated features of each node.

[0202] The generation module is also used to generate composite labels for the spatial regions corresponding to each node based on the updated aggregation features of each node. The composite labels contain the material properties, disease types and spatial location information of the spatial regions.

[0203] Module 55 is used to construct a knowledge graph of the target rammed earth cultural relic based on all generated composite tags.

[0204] This application provides a tag-based multimodal cultural relic knowledge graph construction system to implement the aforementioned tag-based multimodal cultural relic knowledge graph construction method. Therefore, the specific implementation of the tag-based multimodal cultural relic knowledge graph construction system can be found in the embodiment section of the tag-based multimodal cultural relic knowledge graph construction method above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0205] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of a tag-based multimodal cultural relic knowledge graph construction method as described above.

[0206] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described tag-based multimodal cultural relic knowledge graph construction methods.

[0207] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0208] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the tag-based multimodal cultural relic knowledge graph construction method.

[0209] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0210] The foregoing has provided a detailed description of the tag-based multimodal cultural relic knowledge graph construction method, system, and device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for constructing a tag-based multimodal cultural relic knowledge graph, characterized in that, include: Acquire multimodal data of the target rammed earth cultural relic, the multimodal data including multispectral image data covering the surface of the cultural relic, and texture image data corresponding to the surface texture of the cultural relic; Based on a pre-constructed standard reflectance spectrum database of rammed earth materials, spectral analysis is performed on the multispectral image data to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic. The material composition features corresponding to each spatial region are associated with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region. The material feature vector and shape feature vector corresponding to each spatial region are mapped to the initial features of the corresponding nodes in a pre-constructed graph network. Based on the initial features of the nodes and the connection relationship of the edges, the node features are iteratively propagated and aggregated on the graph network to obtain the updated aggregated features of each node. Based on the updated aggregation features of each node, a composite label is generated for the spatial region corresponding to the node. The composite label contains the material properties, disease type, and spatial location information of the spatial region. Based on all generated composite tags, construct a knowledge graph of the target rammed earth artifact.

2. The method according to claim 1, characterized in that, The step of generating composite labels for the spatial region corresponding to each node based on the updated aggregation features of each node includes: For each node in the graph network, the updated aggregated features of the node are input into a preset classification model to parse and obtain the material property category and disease type category corresponding to the node; Obtain the spatial location information represented by the node; The material property category, the disease type category obtained from the parsing, and the extracted spatial location information are combined to generate the composite label corresponding to the node.

3. The method according to claim 1, characterized in that, The step of constructing a knowledge graph of the target rammed earth cultural relic based on all generated composite tags includes: For the material properties, disease types, and spatial location information parsed from the composite tags, corresponding entities are created or matched in a preset knowledge graph framework; Establish an initial attribute association between the entity representing the spatial location information and the entity representing the material properties and disease types; All created or matched entities and their initial attribute associations are integrated to complete the construction of the knowledge graph of the target rammed earth cultural relic.

4. The method according to claim 3, characterized in that, After integrating all created or matched entities and their initial attributes to complete the construction of the knowledge graph for the target rammed earth artifact, the process further includes: In the knowledge graph, a group of spatial location entities is identified, wherein all spatial location entities in the group are adjacent to each other in the graph network and point to the same material property entity or the same disease type entity through the initial attribute association. For the spatial location entities within the group, create a new aggregate entity and integrate the aggregate entity into the knowledge graph of the target rammed earth artifact. The aggregate entity is used to represent a continuous physical region composed of the spatial location entities within the group.

5. The method according to claim 1, further comprising, before mapping the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in the pre-constructed graph network: A graph network is constructed, comprising multiple nodes and edges connecting the nodes, wherein each node corresponds to a spatial region on the surface of the artifact, and the edges are constructed based on the adjacency relationships and feature similarities between the spatial regions.

6. The method according to claim 1, characterized in that, The step of performing spectral analysis on the multispectral image data based on the pre-constructed standard reflectance spectral database of rammed earth materials to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic includes: For any spatial region in the multispectral image data, extract the spectral response sequence of the spatial region; In a pre-constructed database of standard reflectance spectra for rammed earth materials, standard reflectance spectra with the highest similarity to the spectral response sequences of this spatial region are retrieved and identified. If the highest similarity is higher than a preset similarity threshold, the rammed earth material corresponding to the standard reflectance spectrum is determined as the material composition characteristic of the spatial region.

7. The method according to claim 1, characterized in that, The step of associating the material composition features corresponding to each spatial region with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region includes: For each spatial region in the multispectral image data, the material composition features corresponding to the spatial region are encoded to generate a material feature vector for the spatial region; In the texture image data, a neighborhood centered on the spatial region is determined; The distribution of texture values ​​within the neighborhood is statistically analyzed to generate the shape feature vector of the spatial region.

8. The method according to claim 1, characterized in that, The step of mapping the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in a pre-constructed graph network, and iteratively propagating and aggregating the node features on the graph network based on the initial features of the nodes and the connection relationships of the edges, to obtain the updated aggregated features of each node, includes: For each node in the graph network, the material feature vector and shape feature vector of its corresponding spatial region are aggregated to generate the initial features of the node; Based on the initial characteristics of each node, the following propagation and aggregation operations are repeated until a preset iteration termination condition is met: For any target node in the graph network, obtain the current feature vectors of all neighboring nodes of the target node; Aggregate the current feature vectors of all adjacent nodes to generate neighborhood aggregated features; The current feature vector of the target node is combined with the neighborhood aggregated features to update the feature vector of the target node; The node feature vector that is finally updated when the iteration termination condition is met will be used as the aggregated feature of the updated node.

9. A tag-based multimodal cultural relic knowledge graph construction system, characterized in that, include: The acquisition module is used to acquire multimodal data of the target rammed earth cultural relic. The multimodal data includes multispectral image data covering the surface of the cultural relic and texture image data corresponding to the surface texture of the cultural relic. The determination module is used to perform spectral analysis on the multispectral image data based on a pre-constructed standard reflectance spectrum database of rammed earth materials, so as to obtain the material composition characteristics corresponding to each spatial region on the surface of the cultural relic. The generation module is used to associate the material composition features corresponding to each spatial region with the corresponding texture image data to generate a material feature vector and a shape feature vector corresponding to each spatial region. The processing module is used to map the material feature vector and shape feature vector corresponding to each spatial region to the initial features of the corresponding nodes in a pre-constructed graph network, and to perform iterative propagation and aggregation of node features on the graph network based on the initial features of the nodes and the connection relationship of the edges to obtain the updated aggregated features of each node. The generation module is also used to generate a composite label for the spatial region corresponding to the node based on the updated aggregation features of each node. The composite label contains the material properties, disease type and spatial location information of the spatial region. A construction module is used to build a knowledge graph of the target rammed earth artifact based on all generated composite tags.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a tag-based multimodal cultural relic knowledge graph construction method as described in any one of claims 1 to 8 when executing the computer program.

Images