A method and apparatus for cross-source material data semantic alignment

Abstract

The application relates to the field of material data processing and discloses a cross-source material data semantic alignment method and device, a complete technical path from multi-source material data collection, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relationship organization is constructed, the method and device can face multi-source heterogeneous material data such as literature, experiments, calculation and industrial production, fully combine semantic representation learning and intelligent analysis capability, realize unified semantic modeling and alignment at the attribute layer and the relationship layer, further eliminate the semantic gap between cross-source data, realize unified semantic modeling and standardized expression of cross-source material data, and improve material data integration, organization, retrieval and reuse capability.

Description

Technical Field

[0001] This invention relates to the field of materials data processing, and in particular to a method and apparatus for semantic alignment of cross-source materials data. Background Technology

[0002] As materials science research continues to develop towards high throughput, intelligence, and data-driven approaches, the scale of data in the materials field continues to grow, and data sources are becoming increasingly diversified.

[0003] Materials-related data typically originate from multiple sources, including publicly available literature, experimental testing, computational simulations, and industrial production. Data from different sources exhibit significant differences in content presentation, organizational structure, terminology, attribute granularity, and semantic focus.

[0004] However, due to differences in data generation mechanisms, recording habits, and application purposes, material data from different sources may exhibit inconsistencies in naming, expression, semantic granularity, and strong contextual dependencies for the same material object, experimental conditions, or performance index. This severely restricts the unified management and efficient utilization of material data. Summary of the Invention

[0005] This invention provides a method and apparatus for semantic alignment of cross-source material data, which addresses the shortcomings of related technologies in the unified management and efficient utilization of multi-source heterogeneous material data. It can effectively align the semantics of multi-source heterogeneous material data from literature, experiments, calculations, and industrial production, thereby eliminating semantic barriers between cross-source data and improving the integration, organization, retrieval, and reuse capabilities of material data.

[0006] In a first aspect, the present invention provides a method for semantic alignment of cross-source material data, comprising: Preprocessing is performed on multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items; Core structured semantic features are extracted from each of the standardized data items to obtain a unified semantic expression sequence for each standardized data item; High-dimensional semantic embedding representation is performed on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item; Clustering is performed on each of the standardized data items based on each of the high-dimensional embedding vectors to obtain multiple sets of data items; Attribute alignment and relation extraction are performed on each of the data item sets to obtain the attribute alignment data and relation set for each data item set; Each set of data items, each set of attribute-aligned data, and each set of relations are integrated to obtain a normalized data record.

[0007] Optionally, the multi-source heterogeneous material data includes data content in various formats, and each standardized data item includes multiple basic element field names and corresponding field values; The process involves preprocessing multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items, including: The data format of the multi-source heterogeneous material data is unified to obtain multiple initial standardized data items; Using the created cross-source field mapping table, the fields in each of the initial standardized data items are identified and mapped to standard fields, resulting in multiple data items after field normalization; After each field is normalized, the data item is cleaned, missing data is handled, and the expression method is standardized to obtain multiple standardized data items.

[0008] Optionally, the step of extracting core structured semantic features for each of the standardized data items to obtain a unified semantic expression sequence for each of the standardized data items includes: For any of the standardized data items, a material classification prompt template corresponding to the standardized data item is created based on the data type of the standardized data item; Combine the standardized data items and the material classification prompt template to construct a prompt input; The trained large language model is invoked to perform semantic understanding and feature extraction on the prompt input, resulting in multiple structured features corresponding to the standardized data items; wherein each structured feature includes a corresponding field name and field value; Using the created expression template, each structured feature corresponding to the standardized data item is concatenated according to the set field order to obtain a unified semantic expression sequence of the standardized data item.

[0009] Optionally, the step of performing high-dimensional semantic embedding representation on the unified semantic expression sequence of each of the standardized data items to obtain a high-dimensional embedding vector for each of the standardized data items includes: The unified semantic expression sequence of each standardized data item is input into the trained semantic embedding model for high-dimensional semantic embedding representation, and the high-dimensional embedding vector of each standardized data item is obtained from the output of the semantic embedding model. The process of clustering each standardized data item based on each high-dimensional embedding vector yields multiple sets of data items, including: For any two standardized data items, the similarity of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the corresponding similarity. If the similarity is greater than a set threshold, it is determined that there is a candidate alignment relationship between the two standardized data items, and the average value of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the average vector. Cluster each of the average vectors to obtain multiple vector sets; For any of the vector sets, the standardized data items corresponding to each average vector in the vector set are taken as a whole as a data item set.

[0010] Optionally, the step of performing attribute alignment and relation extraction on each of the data item sets to obtain attribute-aligned data and relation sets for each data item set includes: For any standardized data item in any set of data items, standard mapping is performed on each attribute name and attribute value of the standardized data item to obtain standard attribute data. Relationship extraction is performed on the standardized data item and the standard attribute data to obtain relation data. For any of the data item sets, the standard attribute data of each of the standardized data items in the data item set is taken as the attribute alignment data of the data item set, and the relation data of each of the standardized data items in the data item set is taken as the relation set of the data item set.

[0011] Optionally, the step of performing a standard mapping on each attribute name and attribute value of the standardized data item to obtain standard attribute data includes: Retrieves a collection of standard attributes that includes multiple standard attribute names; For any attribute name in the standardized data item, the similarity between the attribute name and each standard attribute name in the standard attribute set is calculated, and the standard attribute name with the highest similarity is determined as the mapping attribute name of the standardized data item. The attribute value normalization function corresponding to the mapping attribute name is obtained, and the attribute value normalization function is used to normalize the attribute value corresponding to the attribute name in the standardized data item to obtain the normalized attribute value. The mapping attribute name and the normalized attribute value are determined as the standard attribute data of the standardized data item.

[0012] Optionally, relation extraction is performed on the standardized data items and the standardized attribute data to obtain relation data, including: Based on the large language model and relational data organization mechanism, relations are extracted from the standardized data items and the standardized attribute data to obtain at least one relation triple of the standardized data items, and the whole is used as the relation data of the standardized data items; wherein, each relation triple includes the relationship between the head entity, the relation type, and the tail entity.

[0013] Optionally, after integrating each set of data items, each set of attribute-aligned data, and each set of relations to obtain normalized data records, the method further includes: The unified semantic expression sequence of each term entity in the material terminology knowledge graph is input into the semantic embedding model for high-dimensional semantic embedding representation, thereby obtaining a high-dimensional embedding vector for each term entity; For any of the standardized data items in the standardized data record, calculate the similarity between the standardized data item and the high-dimensional embedding vector of the term entity, determine the maximum similarity greater than a set threshold in each similarity, and determine the term entity corresponding to the maximum similarity as the target mapping entity of the standardized data item, and add the field name and field value corresponding to the target mapping entity to the standardized data item.

[0014] Optionally, after integrating each set of data items, each set of attribute-aligned data, and each set of relations to obtain normalized data records, the method further includes: For any set of data items, the average value of the high-dimensional embedding vector of each normalized data item in the set of data items is calculated to obtain the center representation vector of the set of data items. In response to the incremental processing instruction for the new material data, the new material data is preprocessed to obtain new standardized data items; The core structured semantic features of the new standardized data item are extracted and high-dimensional semantic embedding representation is performed to obtain the high-dimensional embedding vector of the new standardized data item; Calculate the similarity between the new standardized data item and each of the central representation vectors, and determine the target central representation vector with the highest similarity. The set of data items corresponding to the target center representation vector is determined as the optimal set of data items to which the new standardized data items belong.

[0015] In a second aspect, the present invention provides a cross-source material data semantic alignment apparatus, applied to the cross-source material data semantic alignment method described in the first aspect or any corresponding embodiment thereof, the apparatus comprising: Standardized units are used to preprocess multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items; The feature extraction unit is used to extract core structured semantic features from each of the standardized data items to obtain a unified semantic expression sequence for each of the standardized data items. The high-dimensional embedding unit performs high-dimensional semantic embedding representation on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item. A data clustering unit is used to cluster each of the standardized data items based on each of the high-dimensional embedding vectors to obtain multiple sets of data items; The data processing unit performs attribute alignment and relation extraction on each of the data item sets to obtain attribute-aligned data and relation sets for each data item set. An integration unit is used to integrate each of the data item sets, each of the attribute alignment data, and each of the relation sets to obtain a normalized data record.

[0016] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the cross-source material data semantic alignment method of the first aspect or any corresponding embodiment described above.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the cross-source material data semantic alignment method of the first aspect or any corresponding embodiment thereof.

[0018] The cross-source material data semantic alignment method and apparatus provided by this invention can preprocess multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items. Core structured semantic features are extracted from each standardized data item to obtain a unified semantic expression sequence for each standardized data item. High-dimensional semantic embedding representation is then performed on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item. Each standardized data item is clustered based on its high-dimensional embedding vector to obtain multiple data item sets. Attribute alignment and relation extraction are then performed on each data item set to obtain attribute-aligned data and relation sets for each data item set. Finally, each data item set, each attribute-aligned data set, and each relation set are integrated to obtain a standardized data record. This invention constructs a complete technical path from multi-source material data acquisition, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relation organization. It can address multi-source heterogeneous material data from literature, experiments, computation, and industrial production, fully combining semantic representation learning and intelligent parsing capabilities to achieve unified semantic modeling and alignment at the attribute and relation levels. This eliminates semantic barriers between cross-source data, enabling unified semantic modeling and standardized expression of cross-source material data, and improving the integration, organization, retrieval, and reuse capabilities of material data. Specifically, this invention has at least the following beneficial effects: 1. This invention constructs a complete technical path from multi-source material data acquisition, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relation organization, which can systematically realize unified semantic modeling and standardized expression of cross-source material data; 2. This invention achieves a unified conceptual mapping of material data from different sources, such as literature, experiments, calculations, and industrial production, through the collaborative driving of a material terminology knowledge graph and a semantic representation model. This effectively reduces the impact of terminology ambiguity, conceptual fragmentation, and entity dispersion on data integration. 3. This invention not only achieves the merging of object entities, but also further achieves the collaborative alignment of attribute and relation layers, enhancing the organization, consistency and retrieval of material data; 4. This invention has the ability to perform incremental semantic alignment of new data, which can adapt to application scenarios where material data continues to grow and evolve, and improve the system's scalability and continuous governance capabilities. 5. This invention is applicable to scenarios such as materials database construction, experimental data management, semantic retrieval, knowledge organization, and intelligent analysis, and has strong engineering applicability and promotional value. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a cross-source material data semantic alignment method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another method for semantic alignment of cross-source material data provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the structure of a cross-source material data semantic alignment device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] It should be noted that when performing intelligent processing of materials data, related technologies generally employ methods such as data cleaning, field matching, format conversion, rule mapping, terminology normalization, or database modeling to uniformly organize multi-source data. One approach focuses on converting field names, units of measurement, and data formats in heterogeneous data through predefined rules, templates, or human experience to achieve basic standardization. Another approach constructs materials databases, terminologies, or domain ontologies to standardize and constrain certain material names, attribute terms, and experimental conditions, thereby improving data consistency and searchability. These technologies have played a role in materials data organization, partial merging, and structured storage. However, these technologies still generally suffer from the following shortcomings: First, different data sources may use different names for the same material, experimental conditions, or performance indicators, which can easily lead to ambiguity in terminology, conceptual fragmentation, and semantic inconsistency. Secondly, data from different sources exhibit significant heterogeneity in field structure, attribute organization, and content expression, making it difficult to directly manage and utilize them in a unified manner. Third, it often focuses on format conversion, field mapping or local rule matching, lacks deep alignment capabilities for material domain professional semantics, and is difficult to simultaneously complete object entity recognition, attribute normalization and relationship reconstruction. Fourth, in the face of continuously increasing new experimental data, it is generally difficult to support incremental, scalable data semantic access and unified merging.

[0023] Therefore, there is an urgent need to propose a semantic alignment method for cross-source material data that can fully integrate material terminology knowledge graphs, semantic representation learning, and intelligent parsing capabilities for heterogeneous material data from multiple sources such as literature, experiments, calculations, and industrial production. This method can achieve unified semantic modeling and alignment at the object entity layer, attribute layer, and relation layer, thereby eliminating semantic barriers between cross-source data and improving the ability to integrate, organize, retrieve, and reuse material data.

[0024] The following is combined with Figures 1-2 This invention describes a cross-source material data semantic alignment method.

[0025] like Figure 1 As shown, this embodiment proposes a first method for semantic alignment of cross-source material data, which may include the following steps: S101. Preprocess the multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items.

[0026] Among them, multi-source heterogeneous material data includes one or more of the following: literature data, experimental data, computational data, and industrial production data.

[0027] Specifically, this embodiment can collect multi-source heterogeneous material data and clean, organize, and standardize the multi-source heterogeneous material data to form a standardized dataset including multiple standardized data items. This realizes the transformation of raw material data from different sources and with different formats into a standardized dataset with consistent structure, complete fields, and standardized expression, providing high-quality input for subsequent semantic parsing and alignment operations.

[0028] Optionally, the multi-source heterogeneous material data includes data content in various formats, and each standardized data item includes multiple basic element field names and corresponding field values. Step S101 includes: Data format was standardized for multi-source heterogeneous material data to obtain several initial standardized data items; Using the created cross-source field mapping table, the fields in each initial standardized data item are identified and mapped to standard fields, resulting in multiple data items after field normalization; After each field is standardized, the data items are cleaned, missing data are handled, and the representation is standardized to obtain multiple standardized data items.

[0029] Specifically, this embodiment can perform data acquisition and aggregation, collecting material-related data from multiple heterogeneous data sources such as literature databases, experimental record systems, computational simulation platforms, and industrial production information systems. This includes material composition, structural information, preparation processes, testing conditions, performance results, and source descriptions, forming a raw multi-source material data set, i.e., multi-source heterogeneous data. Taking the field of perovskite solar cells as an example, it can simultaneously collect material composition and efficiency records from literature, band structure and stability calculation results from high-throughput computing platforms, and process parameters and test results from laboratory systems, forming a raw data set covering multiple levels and granularities.

[0030] Specifically, this embodiment can unify the data format of collected multi-source heterogeneous data, converting data content from text, tables, record entries, or structured files into unified data description units. This eliminates differences in storage format, encoding method, and file structure between data from different sources, resulting in multiple initial standardized data items. Specifically, this embodiment can uniformly convert data content in different formats, such as text descriptions, HTML tables, CSV records, or JSON structured files, into standardized data description units based on data items. Each data item includes basic elements such as source identifier, material name, experimental conditions, attribute fields, and result description. The original source information is preserved during the format conversion process to support subsequent traceability verification.

[0031] Subsequently, this embodiment can perform field normalization on the initial standardized data items. It uniformly identifies and organizes the source field, material name field, experimental condition field, performance field, and result field within the initial standardized data items, establishes field correspondences, and eliminates issues such as missing fields, misaligned fields, and ambiguous fields, resulting in data items with normalized fields. Specifically, this embodiment addresses issues such as inconsistent field naming, missing fields, and misaligned fields in data from different sources by uniformly identifying and aligning the source field, material name field, experimental condition field, performance field, and result field. For example, "Band Gap (eV)" from literature sources, "Eg / eV" from calculated data, and "bandgap width (eV)" from experimental records are identified as the same type of field and uniformly mapped to the standard field "band_gap_eV," establishing a cross-source field correspondence table.

[0032] Specifically, this embodiment can also perform content cleaning and missing value processing on each field-normalized data item. It cleans noisy characters, duplicate records, format errors, and obvious outliers in multi-source heterogeneous data, and marks or performs preliminary completion based on contextual information for missing fields, resulting in completed data items. Specifically, this embodiment can clean noisy characters (such as garbled text, residual HTML tags, and extra spaces), duplicate records, format errors, and obvious outliers in the field-normalized data items. For missing fields, marking or contextual completion strategies are selected based on the degree of missingness and the importance of the field. For missing critical fields (such as material names and performance indicators), preliminary filling is performed using statistical completion based on similar records or rule inference, and the completed values ​​are distinguished from the original values ​​by marking, preserving data traceability.

[0033] Specifically, this embodiment can also standardize the expression of each completed data item, normalizing the units of measurement, numerical range, enumeration values, and symbolic expressions in the completed data items. This involves converting the units of common physical quantities such as temperature, pressure, and concentration to a standard unit system, resulting in standardized data items. Specifically, this embodiment can normalize the units of measurement, numerical range, enumeration values, and symbolic expressions in the completed data items. This includes standardizing the temperature unit to Kelvin (K) or degrees Celsius (°C); standardizing the pressure unit to Pascal (Pa); standardizing percentage values ​​such as efficiency to decimal form; and standardizing various equivalent enumeration expressions (such as "N / A", "—", "none", "null") to a null value marker.

[0034] Specifically, in this embodiment, for any original data item Standardized data items are obtained after preprocessing. The processing procedure can be expressed by the following formula:

[0035] in, This represents a standardized processing function used to perform preprocessing operations such as format conversion, field normalization, missing data repair, and expression standardization.

[0036] It should be noted that, after preprocessing, this embodiment can transform the original data from various sources into standardized data items with consistent structure and complete fields, laying the foundation for subsequent semantic parsing.

[0037] S102. Extract the core structured semantic features for each standardized data item to obtain a unified semantic expression sequence for each standardized data item.

[0038] Specifically, this embodiment can perform semantic parsing on standardized datasets based on large language models and material classification prompt templates, extract core structured semantic features that describe material objects and their contextual semantics, and transform unstructured or semi-structured content in standardized data items into a set of structured features with unified semantic orientation, providing consistent semantic representation input for subsequent embedding modeling.

[0039] Optionally, step S102 includes: For any standardized data item, create a material classification prompt template corresponding to the standardized data item based on the data type of the standardized data item; Combine standardized data items and material classification prompt templates to construct prompt input; The trained large language model is invoked to perform semantic understanding and feature extraction on the prompt input, resulting in multiple structured features corresponding to the standardized data items; each structured feature includes the corresponding field name and field value; Using the created expression template, each structured feature corresponding to the standardized data item is concatenated according to the set field order to obtain a unified semantic expression sequence of the standardized data item.

[0040] Specifically, this embodiment can construct material classification prompt templates. This embodiment can design separate material classification prompt templates for different semantic characteristics of literature data, experimental data, computational data, and industrial production data, as well as the unique properties of different metallic materials, ceramic materials, polymer materials, composite materials, semiconductor materials, etc. Each template clearly specifies the core semantic feature fields to be extracted and their expression format, including but not limited to: material name (material_name), composition information (composition), structure type (structure_type), process conditions (process_conditions), test conditions (test_conditions), performance index (property_name), result description (result_description), and source context (source_context). Taking literature data as an example, the prompt template guides the model to extract the above fields from the paper abstract or experimental description paragraph; taking computational data as an example, the prompt template guides the model to extract the structure type, computation method, and target performance index from the computation task description.

[0041] Specifically, this embodiment can perform semantic parsing of large language models. This embodiment can standardize data items. The system combines material classification prompt templates of the corresponding type to construct input prompts. It then calls a large language model (such as the GPT series, LLaMA series, Qwen series, or domain-specific material language models) for semantic understanding and feature extraction, outputting a structured feature set. Taking a literature record of a perovskite solar cell as an example, after parsing by the large language model, the following can be obtained: material name "MAPbI3", composition information "CH3NH3PbI3", structural category "perovskite structure", process conditions "spin coating method, annealing temperature 100℃", performance index "photovoltaic conversion efficiency", and result description "PCE = 18.5%". For any standardized data item... The extracted core structured semantic features are denoted as: ; in, Indicates the first A set of structured semantic features for each data item. Indicates the first Each semantic feature. In actual operation, the number of features extracted from different types of data. They may differ; the completeness of the feature set is determined by both the material classification prompt template and the parsing capability of the large language model.

[0042] Specifically, this embodiment can generate a unified semantic expression sequence. This embodiment can generate a feature set according to a preset template. The semantic features in the data are concatenated into a unified semantic expression sequence according to a fixed field order. As shown in the following formula: ; in, Represents a structured semantic organization function. A unified semantic representation sequence. The expression uses a natural language format of "field name: field value" key-value pairs, for example: "Material name: MAPbI3; Composition information: CH3NH3PbI3; Structure type: perovskite structure; Processing conditions: spin coating, annealing temperature 100℃; Performance index: photoelectric conversion efficiency; Result description: PCE=18.5%". This expression format preserves the semantic independence of each field and ensures formal consistency between data from different sources through a unified sequence structure, making it suitable for subsequent input processing of semantic embedding models.

[0043] S103. Perform high-dimensional semantic embedding representation on the unified semantic expression sequence of each standardized data item to obtain the high-dimensional embedding vector of each standardized data item.

[0044] Specifically, this embodiment can input core structured semantic features into a semantic embedding model to generate corresponding high-dimensional semantic embedding representations, thereby characterizing the conceptual features of each data item in a unified semantic space. This embodiment maps the unified semantic expression sequence of each standardized data item to a high-dimensional continuous vector space, enabling data items from different sources and with different representations to perform semantic similarity calculations within the same metric space, providing a unified vectorized foundation for subsequent entity alignment and attribute alignment.

[0045] Optionally, step S103 includes: The unified semantic representation sequence of each standardized data item is input into the trained semantic embedding model for high-dimensional semantic embedding representation, and the high-dimensional embedding vector of each standardized data item output by the semantic embedding model is obtained.

[0046] Specifically, this embodiment can perform data item semantic embedding generation. This embodiment can generate a unified semantic expression sequence. Input a semantic embedding model (such as SciBERT, MatBERT, Qwen3-Embedding, or a general text embedding model) to obtain its corresponding high-dimensional embedding vector. As shown in the following formula: ; in, This represents a semantic embedding function. Representing the embedding vector dimension, in typical applications Choose dimensions of 512, 1024, 2048, or other suitable dimensions. The semantic embedding model is preferably a model pre-trained or fine-tuned on a materials science corpus to improve the semantic expression of materials terminology, chemical symbols, and experimental descriptions. The semantic embedding model can employ a general text embedding model, a domain-adapted embedding model, a materials-specific semantic vector model, or an embedding model fine-tuned on a materials corpus. Alternatively, the hidden layer state vectors of the model can be used as the embedding.

[0047] S104. Cluster each standardized data item based on each high-dimensional embedding vector to obtain multiple data item sets.

[0048] Specifically, this embodiment can use high-dimensional semantic embedding representation and cluster analysis or similarity calculation to perform semantic alignment and normalization of multi-source heterogeneous material data at the object entity level. It can identify data records from different sources that semantically correspond to the same or similar material objects, thereby achieving semantic alignment and normalization at the object entity level. It can also identify data records from different sources that point to the same material object, similar material objects, or the same concept category, and merge them into a unified set of data items. This breaks through the limitations of simply relying on name consistency or rule matching, and effectively handles the entity dispersion problem caused by material name aliases, abbreviations, and different granular expressions.

[0049] Optionally, step S104 includes: For any two standardized data items, the similarity of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the corresponding similarity. If the similarity is greater than a set threshold, it is determined that there is a candidate alignment relationship between the two standardized data items, and the average value of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the average vector. Cluster each average vector to obtain multiple vector sets; For any set of vectors, the standardized data items corresponding to each average vector in the set of vectors are taken as a set of data items.

[0050] Specifically, this embodiment can perform candidate alignment relationship identification for object entities. For any two data items from different data sources... and Based on its high-dimensional embedding vector, the semantic proximity at the object entity level is calculated as follows: ; when Greater than the preset similarity threshold At that time, the judgment and There are candidate alignment relationships at the object entity level, namely: To illustrate with a specific example: if data items The material name is "MAPbI3", and the data item is... The material is named "methylamine lead iodide perovskite". Although the names are different, their vector similarity is similar after semantic embedding. Typically significantly greater than the threshold Therefore, it is identified as a candidate alignment pair. For example, if data items... The material name is "Q355B1", and the data item is... The material names are either "alloy" or unnamed, and their elemental compositions and proportions are similar, but their vector similarity is lower after semantic embedding. Typically significantly greater than the threshold Therefore, it is identified as a candidate alignment pair. In practical applications, the similarity threshold... It can be adjusted according to the characteristics of the dataset and the alignment accuracy requirements, with a typical value range of 0.80 to 0.92.

[0051] Specifically, this embodiment can perform semantic cluster aggregation. Based on the identification of candidate alignment relationships, it uses clustering algorithms such as hierarchical clustering, DBSCAN (HDBSCAN), or k-means to group mutually adjacent data items in the high-dimensional embedding space into the same semantic cluster, forming a data item set. As shown in the following formula: ; During the clustering process, cosine distance is used as the basis for similarity measurement. Data items within the same set in the clustering results are considered to have semantic equivalence or near-equivalence at the object entity level. Data items from different sources can coexist within the set to support subsequent cross-source attribute integration.

[0052] S105. Perform attribute alignment and relation extraction on each data item set to obtain the attribute alignment data and relation set for each data item set.

[0053] Specifically, this embodiment can perform semantic alignment and normalization at the attribute level for each data item set, uniformly mapping heterogeneous attribute names, attribute semantics and attribute value expressions from different data sources to the standard attribute system, eliminating semantic heterogeneity at the attribute level, and enhancing the comparability and reusability of material data at the attribute level.

[0054] Optionally, step S105 includes: For any standardized data item in any set of data items, perform standard mapping on each attribute name and attribute value of the standardized data item to obtain standard attribute data. Then, extract relations from the standardized data item and the standard attribute data to obtain relation data. For any set of data items, the standard attribute data of each standardized data item in the set of data items is taken as the attribute alignment data of the set of data items, and the relation data of each standardized data item in the set of data items is taken as the relation set of the set of data items.

[0055] Optionally, the above standard mapping of each attribute name and attribute value of the standardized data item yields standard attribute data, including: Retrieves a collection of standard attributes that includes multiple standard attribute names; For any attribute name in a standardized data item, calculate the similarity between the attribute name and each standard attribute name in the standard attribute set, and determine the standard attribute name with the highest similarity as the mapping attribute name of the standardized data item. Obtain the attribute value normalization function corresponding to the mapping attribute name, and use the attribute value normalization function to normalize the attribute value corresponding to the attribute name in the standardized data item to obtain the normalized attribute value. The mapping attribute name and the normalized attribute value are determined as the standard attribute data of the standardized data item.

[0056] Specifically, this embodiment can perform attribute name semantic similarity calculation. This embodiment can construct a standard attribute name set. The standard attribute names are derived from the attribute term nodes in the materials terminology knowledge graph and the materials technical terminology table. For any data item, the original attribute name... The text representation vector is obtained based on the pre-trained language model. And calculate its relationship with each standard attribute name. The overall similarity score is shown in the following formula: ; in, and These are the weighting coefficients. Default value , ; This is based on semantic similarity using embedded vectors; This is a matching correction term obtained based on the material terminology knowledge graph relationship and the professional terminology table. It is used to improve the matching score of attribute pairs with known synonyms or hierarchical relationships, thereby introducing domain prior knowledge to correct the pure semantic similarity calculation results and improve the professionalism and accuracy of attribute mapping.

[0057] Specifically, this embodiment can perform standard attribute mapping, determining the standard attribute with the highest comprehensive score as the original attribute name. The mapping result is shown in the following formula: ; For a specific example, the original attribute name "PCE(%)" from literature data, "photovoltaic conversion efficiency (%)" from experimental records, and "power conversion efficiency" from calculated data are all mapped to the standard attribute "power_conversion_efficiency" after the above calculations, achieving unified standardization of cross-source attribute names. Attributes with mapping scores below a preset threshold are marked as unrecognized attributes and are manually reviewed and supplemented or included in the attribute expansion candidate set.

[0058] Specifically, this embodiment can perform attribute value normalization, which normalizes the attribute values. According to target standard attributes The corresponding attribute value normalization function is used for processing, as shown in the following formula: ; in, Representation and standard attributes The corresponding attribute value normalization function covers the following processing operations: (1) Unit unification: convert the non-standard units attached to the attribute value into the unit system specified by the standard attribute, for example, unify "18.5%" into the value 0.185; (2) Format standardization: decompose the numerical range expression (such as "18.5±0.3") into two subfields: nominal value and uncertainty; (3) Enumeration unification: merge the enumeration values ​​with the same semantics into a unified standard item, for example, unify "spin-coating", "spin coating", and "spin coating method" into "spin_coating"; (4) Equivalent expression conversion: normalize and map equivalent expressions such as chemical formulas and abbreviations, for example, unify "MAPbI3" into "CH3NH3PbI3".

[0059] Optionally, the above process involves extracting relationships from standardized data items and standardized attribute data to obtain relational data, including: Based on the large language model and relational data organization mechanism, relations are extracted from standardized data items and standardized attribute data to obtain at least one relation triple of the standardized data item, and the whole is used as the relation data of the standardized data item; wherein, each relation triple includes the head entity, relation type and the relation between the tail entity.

[0060] Specifically, this embodiment can perform relation layer construction, attribute value completion, and unified semantic organization. Based on the completion of object entity alignment and attribute alignment, it constructs a relational pattern covering the relationship between material objects, preparation processes, experimental conditions, testing methods, and performance results, and completes and unifies the attribute values ​​to form a unified semantic organization result for cross-source material data, providing support for subsequent structured queries, semantic retrieval, and knowledge organization.

[0061] Specifically, this embodiment can perform relation triple construction. Based on the large language model and relational data organization mechanism, it extracts relations from data items after object entity alignment and attribute alignment, and constructs a set of relation triples, represented as: ,in For the head entity, For relation types, This refers to the tail entity. Relationship types include, but are not limited to: hasMaterial (the attribution relationship between a material object and an entity), hasProcess (the association between a material object and a fabrication process), hasTestCondition (the association between experiments and test conditions), hasProperty (the attribute relationship between a material object and performance indicators), hasResult (the output relationship between experiments and performance results), and sourcedFrom (the traceability relationship between data records and source information). Taking perovskite solar cell data as an example, relational triples can be constructed such as: (MAPbI3, hasProcess, spin coating method), (MAPbI3, hasProperty, photoelectric conversion efficiency), and (MAPbI3, hasResult, PCE=18.5%).

[0062] S106. Integrate each set of data items, each attribute-aligned data, and each set of relations to obtain a normalized data record.

[0063] Specifically, this embodiment can be a set of data items. Standard attribute data and relation set Integrate and form standardized data records Standardized data recording It forms the basic unit of unified semantic organization results, supporting multi-dimensional structured queries and semantic retrieval based on object entities, attribute fields, and relationship paths.

[0064] It should be noted that this embodiment can perform missing attribute value completion on normalized data records. If there are missing attribute values, they will be filled in based on context information, statistical results of similar object entities, or relational constraints, as shown in the following formula: ; in, This represents the attribute value completion function, whose input includes complete context information for the current data record. The specific completion strategy is as follows: For attribute values ​​existing in other data records within the same data item set, the mean or mode of the set is used for filling. For attribute values ​​that can be derived from relational constraints (such as optical absorption edges derived from band structures), they are filled through relational path reasoning. For critical attributes that cannot be filled in the above ways, they are marked as "to be filled," and the missing marker is retained to maintain the transparency of data quality. All completed attribute values ​​are marked to distinguish them from the original values, ensuring data traceability.

[0065] The cross-source material data semantic alignment method proposed in this embodiment can preprocess multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items. Core structured semantic features are extracted for each standardized data item to obtain a unified semantic expression sequence for each standardized data item. High-dimensional semantic embedding representation is performed on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item. Based on each high-dimensional embedding vector, each standardized data item is clustered to obtain multiple data item sets. Attribute alignment and relation extraction are performed on each data item set to obtain attribute alignment data and relation sets for each data item set. Each data item set, each attribute alignment data, and each relation set are integrated to obtain a normalized data record. This embodiment constructs a complete technical path from multi-source material data acquisition, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relation organization. It can be used for multi-source heterogeneous material data such as literature, experiments, calculations and industrial production. It fully combines semantic representation learning and intelligent parsing capabilities to achieve unified semantic modeling and alignment at the attribute layer and relation layer, thereby eliminating semantic barriers between cross-source data, realizing unified semantic modeling and standardized expression of cross-source material data, and improving the ability of material data integration, organization, retrieval and reuse.

[0066] based on Figure 1 This embodiment proposes a second method for semantic alignment of cross-source material data. After step S106, this method further includes: The unified semantic expression sequence of each term entity in the material terminology knowledge graph is input into the semantic embedding model for high-dimensional semantic embedding representation, resulting in a high-dimensional embedding vector for each term entity. For any standardized data item in the standardized data record, calculate the similarity between the standardized data item and the high-dimensional embedding vector of the term entity. In each similarity, determine the maximum similarity greater than a set threshold, and determine the term entity corresponding to the maximum similarity as the target mapping entity of the standardized data item. Add the field name and field value corresponding to the target mapping entity to the standardized data item.

[0067] Specifically, this embodiment can perform terminology entity embedding generation for materials terminology knowledge graphs. Any term entity in (e.g., "perovskite", "photoelectric conversion efficiency", "spin coating", etc.) (or other material terminology lists), using the same semantic embedding model as the data item embedding, to obtain the corresponding term entity embedding vectors. As shown in the following formula: ; in, This represents a term entity representation function, whose input is a concatenated representation of the standard name of the term entity and its knowledge graph description text. By embedding term entities and data items into the same vector space, the degree of conceptual association between them can be directly measured based on vector similarity.

[0068] Specifically, this embodiment can perform semantic similarity calculation and mapping relationship establishment for any data item embedding vector. With term entity embedding vector The semantic similarity between the two is calculated using cosine similarity, as shown in the following formula: ; when Greater than the preset similarity threshold At that time, create data items With term entities Candidate mapping relationships between them; among all candidates, the term entity corresponding to the highest similarity is determined as the target mapping entity, that is: ; Among them, the above mapping relationship By associating non-standard expressions in cross-source material data with standardized conceptual entities in domain terminology knowledge graphs, a unified knowledge reference is provided for subsequent attribute semantic mapping and concept unification.

[0069] Specifically, in application scenarios where material data is continuously accumulated, this embodiment performs an optional incremental processing procedure to perform incremental semantic alignment processing on newly added material data, thereby enabling rapid access and unified merging of new data and avoiding full reconstruction.

[0070] Optionally, after step S106, the above method further includes: For any set of data items, the average value of the high-dimensional embedding vector of each normalized data item in the set of data items is calculated to obtain the central representation vector of the set of data items. In response to the incremental processing instruction for the new material data, the new material data is preprocessed to obtain new standardized data items; Core structured semantic features are extracted and high-dimensional semantic embedding representation is performed on the new standardized data items to obtain the high-dimensional embedding vector of the new standardized data items; Calculate the similarity between the new standardized data item and each center representation vector, and determine the target center representation vector with the highest similarity. The set of data items corresponding to the target center representation vector is determined as the optimal set of data items to which the new standardized data items belong.

[0071] Specifically, for the same set of candidate object entities, i.e., the set of data items The data items in the set are represented by the mean vector of the embedding vectors of each data item as the central representation vector of the set.

[0072] Specifically, this embodiment can perform new data preprocessing and semantic embedding generation. The new data items are preprocessed, core structured semantic features are extracted, and high-dimensional semantic embedding representation is performed to obtain the high-dimensional embedding vector of the new standardized data items. The high-dimensional embedding vector of the new standardized data items is matched with the center vector set of the existing data item set to determine the maximum similarity and the optimal set of data items to which they belong.

[0073] In this embodiment, attribution determination and new entity registration can be performed. When the maximum similarity is greater than the preset attribution threshold, the new data item is included in the corresponding data item set, and the center representation vector of the set is updated. Then, the attributes and relationships of the new data item are aligned to complete its access in the unified semantic organization result. When the maximum similarity is lower than the preset attribution threshold, this embodiment can register the new data item as a new candidate object entity in the semantic organization result, initialize its corresponding candidate object entity set, expand the semantic index, and re-establish its mapping relationship with the term entities in the material terminology knowledge graph.

[0074] It should be noted that this embodiment can quickly merge new material data into the existing semantic space and knowledge organization framework without reprocessing existing data, significantly improving the system's scalability and application efficiency in scenarios of continuous data accumulation.

[0075] like Figure 2 As shown, this embodiment proposes a second method for semantic alignment of cross-source material data, which includes the following steps: S1: Collect multi-source heterogeneous material data, and clean, organize and standardize the multi-source heterogeneous material data to form a standardized dataset; multi-source heterogeneous material data includes one or more of the following: literature data, experimental data, computational data and industrial production data.

[0076] S2: Based on a large language model and material classification prompt template, semantic parsing is performed on a standardized dataset to extract core structured semantic features that describe material objects and their contextual semantics.

[0077] S3: Input the core structured semantic features into the semantic embedding model to generate the corresponding high-dimensional semantic embedding representation, so as to characterize the conceptual features of each data item in a unified semantic space.

[0078] S4: Based on high-dimensional semantic embedding representation, clustering analysis or similarity calculation is used to perform semantic alignment and normalization of multi-source heterogeneous material data at the object entity level, and to identify data records from different sources that semantically correspond to the same or similar material objects.

[0079] S5: Based on pre-trained language models, materials terminology tables, and materials terminology knowledge graphs, semantic alignment and standardization of multi-source heterogeneous material data at the attribute level are performed to achieve unified mapping of heterogeneous attribute names, attribute semantics, and attribute expression methods.

[0080] S6: Based on the large language model and relational data organization mechanism, construct the relational schema after object entity alignment and attribute alignment, and standardize, complete and unify the attribute values ​​to form a unified semantic organization result for cross-source material data.

[0081] Through the above steps, this embodiment can map material data from different sources with different structures and expressions into a unified semantic space, and establish semantic associations between data items and domain terminology entities by using a material terminology knowledge graph. This enables collaborative alignment of the object entity layer, attribute layer, and relationship layer, thereby improving the integration, organization, and reuse capabilities of material data.

[0082] This embodiment proposes a cross-source material data semantic alignment method to address the problems of significant semantic differences, inconsistent terminology, difficulty in attribute mapping, and difficulty in relationship integration in related technologies involving multi-source heterogeneous material data. This embodiment standardizes the multi-source material data, extracts structured semantic features using a large language model, generates a data semantic embedding representation, and establishes an association mapping relationship between the dataset and terminology entities based on a material terminology knowledge graph. This achieves collaborative alignment and unified expression of cross-source data at the object entity, attribute, and relationship levels, thereby supporting deep integration, semantic retrieval, and continuous incremental access of material experimental data. Compared with related technologies, it has at least the following beneficial effects: 1. This embodiment constructs a complete technical path from multi-source material data acquisition, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relation organization, which can systematically realize unified semantic modeling and standardized expression of cross-source material data; 2. This embodiment achieves a unified conceptual mapping of material data from different sources, such as literature, experiments, calculations, and industrial production, through the collaborative driving of a material terminology knowledge graph and a semantic representation model. This effectively reduces the impact of terminology ambiguity, conceptual fragmentation, and entity dispersion on data integration. 3. This embodiment not only achieves the merging of object entities, but also further achieves the collaborative alignment of attribute and relation layers, enhancing the organization, consistency and retrieval of material data; 4. This embodiment has the ability to perform incremental semantic alignment of new data, which can adapt to application scenarios where material data continues to grow and evolve, and improve the system's scalability and continuous governance capabilities. 5. This embodiment is applicable to scenarios such as materials database construction, experimental data governance, semantic retrieval, knowledge organization, and intelligent analysis, and has strong engineering applicability and promotional value.

[0083] like Figure 3 As shown, this embodiment proposes a cross-source material data semantic alignment device, which can be applied to any of the above-mentioned cross-source material data semantic alignment methods. The device includes: Standardization unit 101 is used to preprocess multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items; The feature extraction unit 102 is used to extract core structured semantic features for each standardized data item to obtain a unified semantic expression sequence for each standardized data item. The high-dimensional embedding unit 103 performs high-dimensional semantic embedding representation on the unified semantic expression sequence of each standardized data item to obtain the high-dimensional embedding vector of each standardized data item. Data clustering unit 104 is used to cluster each standardized data item based on each high-dimensional embedding vector to obtain multiple data item sets; The data processing unit 105 performs attribute alignment and relation extraction on each data item set to obtain the attribute alignment data and relation set for each data item set. Integration unit 106 is used to integrate each set of data items, each attribute-aligned data, and each set of relations to obtain normalized data records.

[0084] It should be noted that the processing procedures and beneficial effects of the standardization unit 101, feature extraction unit 102, high-dimensional embedding unit 103, data clustering unit 104, data processing unit 105, and integration unit 106 can be referred to respectively. Figure 1 Steps S101 to S106 are not described in detail here.

[0085] Optionally, the multi-source heterogeneous material data includes data content in various formats, and each standardized data item includes multiple basic element field names and corresponding field values; Standardized unit 101 is also used for: Data format was standardized for multi-source heterogeneous material data to obtain several initial standardized data items; Using the created cross-source field mapping table, the fields in each initial standardized data item are identified and mapped to standard fields, resulting in multiple data items after field normalization; After each field is standardized, the data items are cleaned, missing data are handled, and the representation is standardized to obtain multiple standardized data items.

[0086] Optionally, the feature extraction unit 102 is also used for: For any standardized data item, create a material classification prompt template corresponding to the standardized data item based on the data type of the standardized data item; Combine standardized data items and material classification prompt templates to construct prompt input; The trained large language model is invoked to perform semantic understanding and feature extraction on the prompt input, resulting in multiple structured features corresponding to the standardized data items; each structured feature includes the corresponding field name and field value; Using the created expression template, each structured feature corresponding to the standardized data item is concatenated according to the set field order to obtain a unified semantic expression sequence of the standardized data item.

[0087] Optionally, the high-dimensional embedding unit 103 is also used for: The unified semantic representation sequence of each standardized data item is input into the trained semantic embedding model for high-dimensional semantic embedding representation, and the high-dimensional embedding vector of each standardized data item output by the semantic embedding model is obtained. Data clustering unit 104 is also used for: For any two standardized data items, the similarity of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the corresponding similarity. If the similarity is greater than a set threshold, it is determined that there is a candidate alignment relationship between the two standardized data items, and the average value of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the average vector. Cluster each average vector to obtain multiple vector sets; For any set of vectors, the standardized data items corresponding to each average vector in the set of vectors are taken as a set of data items.

[0088] Optionally, the data processing unit 105 is also used for: For any standardized data item in any set of data items, perform standard mapping on each attribute name and attribute value of the standardized data item to obtain standard attribute data. Then, extract relations from the standardized data item and the standard attribute data to obtain relation data. For any set of data items, the standard attribute data of each standardized data item in the set of data items is taken as the attribute alignment data of the set of data items, and the relation data of each standardized data item in the set of data items is taken as the relation set of the set of data items.

[0089] Optionally, the data processing unit 105 is also used for: Retrieves a collection of standard attributes that includes multiple standard attribute names; For any attribute name in a standardized data item, calculate the similarity between the attribute name and each standard attribute name in the standard attribute set, and determine the standard attribute name with the highest similarity as the mapping attribute name of the standardized data item. Obtain the attribute value normalization function corresponding to the mapping attribute name, and use the attribute value normalization function to normalize the attribute value corresponding to the attribute name in the standardized data item to obtain the normalized attribute value. The mapping attribute name and the normalized attribute value are determined as the standard attribute data of the standardized data item.

[0090] Optionally, the data processing unit 105 is also used for: Based on the large language model and relational data organization mechanism, relations are extracted from standardized data items and standardized attribute data to obtain at least one relation triple of the standardized data item, and the whole is used as the relation data of the standardized data item; wherein, each relation triple includes the head entity, relation type and the relation between the tail entity.

[0091] Optionally, the above-mentioned device further includes: The mapping unit is used to integrate each set of data items, each attribute-aligned data, and each set of relations to obtain normalized data records. Then, it inputs the unified semantic expression sequence of each term entity in the material terminology knowledge graph into the semantic embedding model for high-dimensional semantic embedding representation, resulting in a high-dimensional embedding vector for each term entity. For any normalized data item in the normalized data record, it calculates the similarity between the normalized data item and the high-dimensional embedding vector of the term entity. In each similarity, it determines the maximum similarity greater than a set threshold and determines the term entity corresponding to the maximum similarity as the target mapping entity of the normalized data item. It then adds the field name and field value corresponding to the target mapping entity to the normalized data item.

[0092] Optionally, the above-mentioned device further includes an incremental unit; Incremental units are used for: After integrating each set of data items, each attribute-aligned data, and each set of relations to obtain normalized data records, for any set of data items, the average value of the high-dimensional embedding vector of each normalized data item in the set of data items is calculated to obtain the central representation vector of the set of data items. In response to the incremental processing instruction for the new material data, the new material data is preprocessed to obtain new standardized data items; Core structured semantic features are extracted and high-dimensional semantic embedding representation is performed on the new standardized data items to obtain the high-dimensional embedding vector of the new standardized data items; Calculate the similarity between the new standardized data item and each center representation vector, and determine the target center representation vector with the highest similarity. The set of data items corresponding to the target center representation vector is determined as the optimal set of data items to which the new standardized data items belong.

[0093] The cross-source material data semantic alignment device proposed in this embodiment constructs a complete technical path from multi-source material data acquisition, semantic feature extraction, semantic embedding representation, object entity alignment, attribute alignment to relation organization. It can fully combine semantic representation learning and intelligent parsing capabilities to achieve unified semantic modeling and alignment at the attribute and relation layers for multi-source heterogeneous material data such as literature, experiments, calculations and industrial production. This eliminates semantic barriers between cross-source data, realizes unified semantic modeling and standardized expression of cross-source material data, and improves the ability of material data integration, organization, retrieval and reuse.

[0094] In this embodiment, the cross-source material data semantic alignment device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0095] This invention also provides a computer device having the above-described features. Figure 3 The cross-source material data semantic alignment device shown.

[0096] Please see Figure 4 The present invention provides a schematic diagram of the structure of a computer device according to an optional embodiment. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0097] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0098] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0099] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0100] Memory 20 may include volatile memory, such as random access memory. Memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. Memory 20 may also include combinations of the above types of memory.

[0101] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0102] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for semantic alignment of cross-source material data, characterized in that, include: Preprocessing is performed on multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items; Core structured semantic features are extracted from each of the standardized data items to obtain a unified semantic expression sequence for each standardized data item; High-dimensional semantic embedding representation is performed on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item; Clustering is performed on each of the standardized data items based on each of the high-dimensional embedding vectors to obtain multiple sets of data items; Attribute alignment and relation extraction are performed on each of the data item sets to obtain the attribute alignment data and relation set for each data item set; Each set of data items, each set of attribute-aligned data, and each set of relations are integrated to obtain a normalized data record.

2. The method according to claim 1, characterized in that, The multi-source heterogeneous material data includes data content in various formats, and each standardized data item includes multiple basic element field names and corresponding field values; The process involves preprocessing multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items, including: The data format of the multi-source heterogeneous material data is unified to obtain multiple initial standardized data items; Using the created cross-source field mapping table, the fields in each of the initial standardized data items are identified and mapped to standard fields, resulting in multiple data items after field normalization; After each field is normalized, the data item is cleaned, missing data is handled, and the expression method is standardized to obtain multiple standardized data items.

3. The method according to claim 1, characterized in that, The step of extracting core structured semantic features for each of the standardized data items to obtain a unified semantic expression sequence for each standardized data item includes: For any of the standardized data items, a material classification prompt template corresponding to the standardized data item is created based on the data type of the standardized data item; Combine the standardized data items and the material classification prompt template to construct a prompt input; The trained large language model is invoked to perform semantic understanding and feature extraction on the prompt input, resulting in multiple structured features corresponding to the standardized data items; wherein each structured feature includes a corresponding field name and field value; Using the created expression template, each structured feature corresponding to the standardized data item is concatenated according to the set field order to obtain a unified semantic expression sequence of the standardized data item.

4. The method according to claim 1, characterized in that, The step of performing high-dimensional semantic embedding representation on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item includes: The unified semantic expression sequence of each standardized data item is input into the trained semantic embedding model for high-dimensional semantic embedding representation, and the high-dimensional embedding vector of each standardized data item is obtained from the output of the semantic embedding model. The process of clustering each standardized data item based on each high-dimensional embedding vector yields multiple sets of data items, including: For any two standardized data items, the similarity of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the corresponding similarity. If the similarity is greater than a set threshold, it is determined that there is a candidate alignment relationship between the two standardized data items, and the average value of the high-dimensional embedding vectors of the two standardized data items is calculated to obtain the average vector. Cluster each of the average vectors to obtain multiple vector sets; For any of the vector sets, the standardized data items corresponding to each average vector in the vector set are taken as a whole as a data item set.

5. The method according to claim 1, characterized in that, The step of performing attribute alignment and relation extraction on each of the data item sets to obtain attribute-aligned data and relation sets for each data item set includes: For any standardized data item in any set of data items, standard mapping is performed on each attribute name and attribute value of the standardized data item to obtain standard attribute data. Relationship extraction is performed on the standardized data item and the standard attribute data to obtain relation data. For any of the data item sets, the standard attribute data of each of the standardized data items in the data item set is taken as the attribute alignment data of the data item set, and the relation data of each of the standardized data items in the data item set is taken as the relation set of the data item set.

6. The method according to claim 5, characterized in that, The standard mapping of each attribute name and attribute value of the standardized data item to obtain standard attribute data includes: Retrieves a collection of standard attributes that includes multiple standard attribute names; For any attribute name in the standardized data item, the similarity between the attribute name and each standard attribute name in the standard attribute set is calculated, and the standard attribute name with the highest similarity is determined as the mapping attribute name of the standardized data item. The attribute value normalization function corresponding to the mapping attribute name is obtained, and the attribute value normalization function is used to normalize the attribute value corresponding to the attribute name in the standardized data item to obtain the normalized attribute value. The mapping attribute name and the normalized attribute value are determined as the standard attribute data of the standardized data item.

7. The method according to claim 5, characterized in that, Relationship extraction is performed on the standardized data items and the standardized attribute data to obtain relation data, including: Based on the large language model and relational data organization mechanism, relations are extracted from the standardized data items and the standardized attribute data to obtain at least one relation triple of the standardized data items, and the whole is used as the relation data of the standardized data items; wherein, each relation triple includes the relationship between the head entity, the relation type, and the tail entity.

8. The method according to claim 4, characterized in that, After integrating each set of data items, each set of attribute-aligned data, and each set of relations to obtain normalized data records, the method further includes: The unified semantic expression sequence of each term entity in the material terminology knowledge graph is input into the semantic embedding model for high-dimensional semantic embedding representation, thereby obtaining a high-dimensional embedding vector for each term entity; For any of the standardized data items in the standardized data record, calculate the similarity between the standardized data item and the high-dimensional embedding vector of the term entity, determine the maximum similarity greater than a set threshold in each similarity, and determine the term entity corresponding to the maximum similarity as the target mapping entity of the standardized data item, and add the field name and field value corresponding to the target mapping entity to the standardized data item.

9. The method according to any one of claims 1 to 8, characterized in that, After integrating each set of data items, each set of attribute-aligned data, and each set of relations to obtain normalized data records, the method further includes: For any set of data items, the average value of the high-dimensional embedding vector of each normalized data item in the set of data items is calculated to obtain the center representation vector of the set of data items. In response to the incremental processing instruction for the new material data, the new material data is preprocessed to obtain new standardized data items; The core structured semantic features of the new standardized data item are extracted and high-dimensional semantic embedding representation is performed to obtain the high-dimensional embedding vector of the new standardized data item; Calculate the similarity between the new standardized data item and each of the central representation vectors, and determine the target central representation vector with the highest similarity. The set of data items corresponding to the target center representation vector is determined as the optimal set of data items to which the new standardized data items belong.

10. A cross-source material data semantic alignment device, characterized in that, The apparatus used in the cross-source material data semantic alignment method according to any one of claims 1 to 9 includes: Standardized units are used to preprocess multi-source heterogeneous material data collected from different data sources to obtain multiple standardized data items; The feature extraction unit is used to extract core structured semantic features from each of the standardized data items to obtain a unified semantic expression sequence for each of the standardized data items. The high-dimensional embedding unit performs high-dimensional semantic embedding representation on the unified semantic expression sequence of each standardized data item to obtain a high-dimensional embedding vector for each standardized data item. A data clustering unit is used to cluster each of the standardized data items based on each of the high-dimensional embedding vectors to obtain multiple sets of data items; The data processing unit performs attribute alignment and relation extraction on each of the data item sets to obtain attribute-aligned data and relation sets for each data item set. An integration unit is used to integrate each of the data item sets, each of the attribute alignment data, and each of the relation sets to obtain a normalized data record.

Images