A method for automatically constructing and dynamically evolving domain ontology based on active metadata and semantic bridging

Abstract

This invention discloses an automatic construction and dynamic evolution method for domain ontology based on proactive metadata and semantic bridging, belonging to the interdisciplinary fields of knowledge graph construction, semantic computation, and data governance. The method first collects proactive metadata from multiple heterogeneous data sources, extracts semantic features, and vectorizes them. Then, it constructs a cross-data source semantic bridging graph. Based on the bridging graph, it automatically generates domain concept classes, attributes, and hierarchical structures, and optimizes confidence through a manual verification mechanism. This method further supports incremental updates after the addition of new data sources, realizing the dynamic evolution of the domain ontology. Compared with traditional manual construction methods, this invention reduces reliance on manual intervention, improves construction efficiency and semantic consistency, and is suitable for large-scale data governance and knowledge graph construction scenarios.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of knowledge graph construction, semantic computing, and data governance, specifically a method for the automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging. Background Technology

[0002] Domain ontology is a standardized semantic abstraction of core concepts, attributes, relationships, and constraints within a specific business domain. It serves as the core foundation for achieving cross-system data semantic unification, data interoperability, and intelligent semantic reasoning, and is widely used in core business scenarios such as knowledge graph construction, data governance, and data platform construction. As enterprise digital transformation deepens, multiple independently deployed heterogeneous business systems often exist within enterprises. These systems exhibit significant differences in field naming rules, semantic definitions, and data structures, necessitating a standardized domain ontology to achieve cross-system data semantic connectivity and provide a unified semantic foundation for upper-layer data applications.

[0003] Currently, the mainstream approach to constructing domain ontologies is still primarily a purely manual or semi-manual solution led by human experts. The core process involves domain experts with both business rules and technical knowledge systematically reviewing the structural information of all heterogeneous data sources, manually extracting core business concepts, designing the hierarchical structure of these concepts, defining their attributes and relationships, and completing the ontology construction after multiple rounds of manual verification. Subsequent iterative updates also require manual completion. This approach has unavoidable inherent drawbacks: First, it is highly dependent on human intervention, requiring a large number of senior experts with cross-domain knowledge, resulting in high labor costs and a typically long construction cycle, making it difficult to adapt to the rapid iteration needs of current business. Second, its coverage is limited; when facing large-scale heterogeneous data sources, manual review is prone to omissions, especially for legacy systems without annotations or with non-standard naming conventions, where it is difficult for humans to accurately determine the true semantic meaning of fields, leading to generally low ontology coverage. Third, semantic consistency is difficult to guarantee; different experts may have different understandings of the same business concept, easily causing semantic conflicts and resulting in a high statistical error rate when integrating data across systems.

[0004] To reduce the cost of manual construction, some rule-based semi-automatic domain ontology construction solutions have emerged in existing technologies. These solutions typically use preset naming matching rules and static metadata comparison to identify synonyms of fields, and then semi-automatically generate the domain ontology. However, these solutions still have limitations: First, they rely solely on static metadata for matching, failing to consider dynamic information such as runtime access behavior of fields and cross-system dependencies, resulting in low accuracy in identifying unannotated or poorly named fields. Second, they lack a unified cross-source semantic mapping mechanism, leading to high conflict rates during cross-system semantic fusion and hindering the semantic unification of multi-source data. Third, they do not support dynamic incremental updates; when new data sources are added or the business structure changes, a complete re-examination of all data is required to reconstruct the ontology, making it impossible to respond to business changes in real time and resulting in extremely high ontology evolution costs.

[0005] In summary, existing domain ontology construction solutions, whether using purely manual or semi-automatic modes, cannot simultaneously meet the requirements of low labor costs, high recognition accuracy, and dynamic incremental evolution in large-scale heterogeneous data scenarios. There is an urgent need for a new domain ontology construction technology to solve these problems. Summary of the Invention

[0006] To address the problems existing in the aforementioned background technologies, this invention proposes an automatic construction and dynamic evolution method for domain ontology based on proactive metadata and semantic bridging. Through a cross-source proactive metadata collection mechanism, a semantic bridging graph construction method based on multi-feature fusion, a confidence-based hierarchical ontology automatic generation logic with low manual intervention, and a locally updated dynamic incremental evolution strategy for ontology, this method aims to reduce the manual cost of domain ontology construction, improve the accuracy of cross-source semantic unification, and enable the ontology to dynamically evolve with changes in business, thereby supporting large-scale data governance and the large-scale implementation of knowledge graphs.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging includes the following steps: S1. Connect to several distributed heterogeneous data sources and collect active metadata from each data source; S2. Extract semantic features from the active metadata, generate semantic vectors corresponding to each field, and obtain a candidate concept set based on semantic vector clustering; S3. Using the fields, tables, and candidate concepts of each data source as core nodes, and the semantic similarity mapping relationship, structural similarity mapping relationship, and behavioral association mapping relationship between nodes as core edges, construct a semantic bridging graph; S4. Extract the mapping relationships in the semantic bridging graph that meet the preset confidence requirements, and generate a domain ontology that includes concept classes, attributes, associations and hierarchical structures; S5. Filter the mapping relationships in the semantic bridging graph whose confidence level is in the preset low confidence interval and push them to manual verification. Update the mapping relationship weights and structure of the semantic bridging graph according to the manual verification results. S6. When a change in the data source structure or the addition of a new data source is detected, the semantic bridging graph is locally updated, and the domain ontology structure related to the changed content is incrementally reconstructed.

[0008] Specifically, step S1 involves: connecting to several distributed heterogeneous data sources through a virtualized federated unified access layer, wherein the distributed heterogeneous data sources include at least one of relational databases, NoSQL databases, data files, and application programming interfaces; collecting active metadata from each data source, wherein the active metadata includes four categories: static structural information, field semantic information, runtime access behavior information, and data characteristic information.

[0009] Specifically, step S2 includes: S21. Extract the field name, field annotation, and a preset number of field data samples corresponding to each field from the active metadata, and perform semantic vectorization transformation on each field to obtain field name vector, field annotation vector, and data sample vector. S22. The field name vector, field annotation vector, and data sample vector are weighted and fused to obtain the semantic vector of the corresponding field. S23. Based on the precondition of data type matching, the semantic vectors of all fields are clustered. Each cluster corresponds to a candidate concept. After naming standardization and semantic vector generation of all candidate concepts, a set of candidate concepts is obtained.

[0010] Specifically, in step S3, the generation rule for semantic similarity mapping relationship is: when the semantic vector similarity of two nodes is greater than the first preset similarity threshold, a semantic similarity mapping relationship is generated between fields or between a field and a candidate concept, and its weight is set as the semantic vector similarity value of the two nodes. The specific rules for generating structural similarity mapping relationships are as follows: determine whether two fields satisfy any of the following conditions: a. The two fields have the same data type and the semantic vector similarity of the field names is greater than the second preset similarity threshold; b. Both fields are either primary keys or both are foreign keys; c. The similarity of the table-level semantic vectors of the two fields is greater than the preset table similarity threshold, and the logical function groups to which the semantic labels of the two fields belong are consistent; When two fields meet any of the conditions, a structural similarity mapping relationship is generated, and the weighted sum of type matching degree, naming similarity degree, and key attribute matching degree is set as the weight. The specific rules for generating behavior association mapping relationships are as follows: when the number of times two fields co-occur in the same SQL statement or the same service call chain within a preset statistical period is greater than a preset threshold, a behavior association mapping relationship is generated between the two fields, and its weight is set as the ratio of the number of co-occurrences to the total access frequency of the two fields within the preset statistical period.

[0011] Specifically, the semantic bridging graph also includes cross-data source synonym mapping relationships. The generation rule for cross-data source synonym mapping relationships is as follows: when the weight of the semantic similarity mapping relationship between two nodes belonging to different data sources is greater than a preset semantic threshold, and the weight of the structural similarity mapping relationship is greater than a preset structural threshold, a cross-data source synonym mapping relationship between the two nodes is generated. The initial confidence calculation rule for all mapping relationships in the semantic bridging graph is as follows: multiply the weights of the semantic similarity mapping relationship, the structural similarity mapping relationship, and the behavioral association mapping relationship with the preset semantic weight coefficient, preset structural weight coefficient, and preset behavioral weight coefficient respectively, and then sum them.

[0012] Specifically, the construction of the semantic bridging graph in step S3 also includes a confidence propagation mechanism: (1) For two nodes that are indirectly associated through at least two consecutive edges, the mapping confidence of each indirect association path is calculated as the product of the mapping relation weights corresponding to all edges on the path. The maximum mapping confidence of all indirect association paths is taken as the initial confidence of the indirect association mapping between the two nodes. (2) Update the initial confidence of the mapping between each field node and the corresponding candidate concept according to the preset iteration rule. The preset iteration rule is: the initial confidence of the mapping of the field node in the (t+1)th iteration is equal to the sum of the weights of all adjacent edges connected to the field node and the initial confidence of the mapping of the candidate concept corresponding to the adjacent node in the tth iteration. (3) Normalize the initial confidence of the mapping between all field nodes and corresponding candidate concepts so that the value of the initial confidence of all mappings is in the range of 0 to 1; (4) Based on the updated initial confidence of the mapping, re-determine the candidate concepts to which the field node belongs, and take the candidate concept with the highest initial confidence of the mapping to the corresponding field node and greater than the preset belonging threshold as the final belonging concept of the field node.

[0013] Specifically, step S4 includes: S41. Perform multi-dimensional fusion calculation on each mapping relationship in the semantic bridging graph to obtain the final confidence of each mapping relationship. The multi-dimensional fusion calculation process specifically involves weighted summation of the initial confidence of the corresponding mapping relationship with the data statistical mapping weight. S42. The mapping relationships in the semantic bridging graph are graded according to the final confidence level: a. Mapping relationships with a final confidence level not less than the first preset confidence threshold are determined to be high-confidence mappings and are directly applied to domain ontology construction; b. The mapping relationship between the final confidence level and the first preset confidence threshold and the second preset confidence threshold is determined to be a medium-to-high confidence mapping relationship and participates in the construction of candidate ontology structures; c. The mapping relationship between the final confidence level and the second and third preset confidence thresholds is determined as a medium confidence mapping relationship, which is used for relationship completion and inference; d. Mapping relationships with a final confidence level less than the third preset confidence threshold are determined to be low-confidence mapping relationships and will not participate in ontology construction; S43. Based on the filtered mapping relationship, generate a domain ontology that includes concept classes, attributes, relationships and hierarchical structure.

[0014] Specifically, the generation rule for concept classes in the domain ontology is as follows: when the number of fields contained in the cluster corresponding to a candidate concept is greater than the preset minimum cluster size, and the average semantic similarity of the fields within the cluster is greater than the preset average cluster similarity threshold, the candidate concept is mapped to a concept class. The generation rule for attributes in the domain ontology is as follows: based on the high confidence mapping relationship between fields and concept classes, fields with a mapping confidence of greater than a preset attribute threshold are used as attributes of the corresponding concept classes. Among them, fields belonging to the primary key are mapped as identifier attributes, and the remaining fields are mapped as data attributes. The generation rules for relationships in the domain ontology are as follows: filter high-confidence mapping relationships. When there is a foreign key dependency, co-occurrence relationship between SQL statements and call chains, or semantic similarity mapping between fields corresponding to two concept classes, generate a relationship between the two concept classes. The generation rule for the hierarchical structure in the domain ontology is as follows: if all corresponding fields of the first concept class belong to the corresponding fields of the second concept class, or if the first concept class is a subordinate semantic of the second concept class, then the first concept class is determined to be a subclass of the second concept class.

[0015] Specifically, step S5 includes: S51. Filter the mapping relationships that require manual verification. The mapping relationships include two types: (1) The mapping relationship where the final confidence level is in the preset low confidence interval; (2) Mapping relationships with mapping conflicts, including multi-attribution conflicts where a single field is simultaneously mapped to multiple candidate concepts, conflicts where there is circular dependency between concepts, and bidirectional conflicts in the direction of association. S52. Receive manually input structured verification operation, wherein the structured verification operation and corresponding processing rules are as follows: a. Confirmation operation: Set the weight of the corresponding mapping relationship to 1 and mark it as a positive training sample; b. Rejection operation: Set the weight of the corresponding mapping relationship to 0 and mark it as a negative training sample; c. Redirection operation: Adjust the candidate concepts of the corresponding field and mark them as corrected training samples; S53. Based on the training samples generated by all structured verification operations, update the mapping relationship weights and topology of the semantic bridging graph, and adjust the semantic vector calculation parameters and mapping threshold parameters.

[0016] Specifically, step S6 includes: S61. Poll or receive event trigger signals according to a preset period to detect the status of distributed heterogeneous data sources. When a change in the structure of an existing data source or the access of a new data source is detected, collect the corresponding active metadata. S62. Calculate the mapping relationship between the newly added or changed node and the existing nodes in the semantic bridging graph that have semantic similarity, structural association or behavioral association, and the corresponding initial confidence of the mapping. Then, execute the local confidence propagation mechanism on the updated initial confidence of the mapping and locally update the mapping relationship weight and topology of the semantic bridging graph based on the calculation results. S63. Adjust the concept classes, attributes, relationships, and hierarchical structure related to the changed content in the domain ontology, and incrementally reconstruct to obtain the updated domain ontology.

[0017] In summary, the beneficial technical effects of the present invention are as follows: 1. Reduced manual costs and time required for ontology construction: This invention eliminates the need for manual sorting of heterogeneous data source structures through a virtualized federated proactive metadata collection mechanism. Combined with a confidence leveling mechanism, only low-confidence mappings are pushed to manual verification, eliminating the need for manual verification of all mapping relationships. Compared to traditional manual construction solutions, both manual costs and the domain ontology construction cycle are reduced, adapting to the needs of rapid business iteration.

[0018] 2. Improved semantic recognition accuracy and coverage of cross-source heterogeneous data: This invention constructs a semantic bridging graph by fusing semantic, structural, and behavioral features, combined with a confidence propagation inference mechanism. It can achieve accurate recognition of cross-source synonymous fields without relying on field annotations, and can cover all common heterogeneous data source types without the risk of omissions.

[0019] 3. Solves the problem of semantic inconsistency across systems: This invention achieves global semantic mapping of all heterogeneous data sources through a unified semantic bridging graph, effectively avoiding semantic conflicts caused by differences in understanding among different experts, and providing an unambiguous semantic foundation for cross-system data interoperability.

[0020] 4. Achieves dynamic incremental evolution of the ontology: This invention adopts a local incremental update mechanism. When a new data source is added or the existing business structure changes, it is not necessary to fully reconstruct the entire ontology. Only the metadata of the changed part needs to be collected and the semantic bridging graph is locally updated. Dynamic incremental evolution reduces evolution costs. Attached Figure Description

[0021] Figure 1 This is a flowchart of the overall method of the present invention. Detailed Implementation

[0022] To make the technical means, creative features, objectives and effects of this invention clearer and easier to understand, the invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0023] Example like Figure 1 As shown, the present invention provides a method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging, comprising: S1. Connect to several distributed heterogeneous data sources through a virtualized federated unified access layer. The distributed heterogeneous data sources include at least one of relational databases, NoSQL databases, data files, and application programming interfaces. The relational databases are connected through JDBC / R2DBC drivers, the NoSQL databases are connected through the corresponding data source APIs, the data files are connected through object storage interfaces, and the application programming interfaces are connected through API gateway interfaces. Subsequently, active metadata from each data source was collected, including four categories: static structure information, field semantic information, runtime access behavior information, and data characteristic information.

[0024] Specifically, static structure information includes table name, field name, field type, index, partition information, constraints such as NOT NULL / unique / default value, and explicit foreign key dependencies; it is obtained through JDBC / R2DBC standard metadata interfaces or database built-in system tables (such as MySQL's INFORMATION_SCHEMA and Oracle's SYS.TABLES), and the validity of constraints is statically parsed and verified on DDL statements; Field semantic information includes the business semantic meaning, semantic tags, and usage context information of the field. It is obtained by combining existing data dictionaries and pre-annotated business models, using natural language processing technology to parse field names and field annotations, and extracting the field usage context from SQL query logs.

[0025] Runtime access behavior information includes field access frequency, table access frequency, field query co-occurrence relationship, data access path, and cross-service / cross-system data flow; it is obtained by bypassing the monitoring of database audit logs, query engine logs, and API call chain logs. Data feature information includes field null value rate, maximum / minimum value, enumeration value distribution, numerical range, and sampled data samples; statistical features are obtained through SQL aggregation calculations, and non-null sample data of the fields are extracted according to a preset sampling ratio.

[0026] S2. Extract semantic features from the active metadata to generate semantic vectors for each field, and obtain a candidate concept set based on semantic vector clustering; this step specifically includes: S21. Extract the field name, field annotation, and a preset number of field data samples corresponding to each field from the active metadata, and perform semantic vectorization transformation on each field to obtain field name vector, field annotation vector, and data sample vector. Specifically, this step first preprocesses the text, splits the field names into camelCase and underscore names to restore them to natural language semantic representations, directly retains the original natural language descriptions of the field annotations, selects the Top-K high-frequency values ​​of the field data samples and concatenates them into a text sequence, and then uses the existing mainstream pre-trained models to achieve semantic encoding, generating field name vectors, field annotation vectors and data sample vectors respectively. S22. The field name vector, field annotation vector, and data sample vector for the same field are weighted and fused to obtain the semantic vector for the corresponding field. The weighting formula is: ; in, , , The weighting coefficients are adjustable and dynamically adjusted based on data quality. For regular fields containing field comments, the preferred weighting coefficients are 0.4, 0.4, and 0.2. For legacy system fields without field comments, the coefficients can be dynamically adjusted to... , No need to rely on annotation information.

[0027] S23. Based on the pre-constraint of data type matching, the semantic vectors of all fields are clustered. Each cluster corresponds to a candidate concept. After naming standardization and semantic vector generation of all candidate concepts, a set of candidate concepts is obtained. Specifically, this step first executes the pre-constraint of data type matching, only including fields with the same data type into the same cluster candidate set, to avoid similar named fields of different data types being incorrectly clustered; Next, the semantic vectors of all fields are clustered. The appropriate clustering algorithm can be selected according to the business scenario. For scenarios with large data scale and predictable number of concepts, K-Means clustering is used, which has high computational efficiency. For scenarios that need to generate a tree-like semantic structure, hierarchical clustering is used, which can directly output the initial hierarchical relationship. For scenarios with uneven field distribution and abnormal noise fields, DBSCAN clustering is used, which can automatically identify the number of clusters and filter noise points. Each cluster corresponds to a candidate concept. The business term that appears most frequently within the cluster is taken as the standardized name of the candidate concept, and the center vector of the cluster is taken as the semantic vector of the candidate concept. All candidate concepts form a candidate concept set.

[0028] S3. Using fields, tables, and candidate concepts from each data source as core nodes of the semantic bridging graph, and semantic similarity mapping relationships, structural similarity mapping relationships, and behavioral association mapping relationships between nodes as core edges, a semantic bridging graph is constructed; specifically including: S31. Construct three types of core nodes, where each field corresponds to a unique field node. The node attributes include the field's unique identifier, field name, data type, table ID, the field semantic vector generated in step S2, the field null value rate, the number of deduplicated values, the access frequency, the sampled data, and the semantic label, which is the finest unit of semantic analysis. Each data table corresponds to a unique table node. The node attributes include the table's unique identifier, table name, data source identifier, number of fields, primary key set, foreign key set, and table-level semantic vector. The table-level semantic vector is the average of the semantic vectors of all field nodes under that table. Each candidate concept generated in step S2 corresponds to a unique concept node. The node attributes include a unique concept identifier, a standardized concept name, a semantic vector (cluster center vector), a set of synonyms, and a business domain label.

[0029] S32. Generate three types of core basic edges. The specific rules for generating semantic similarity mapping relationships are as follows: use cosine similarity to calculate the semantic vector similarity between fields and fields, and between fields and any two nodes in the candidate concept. When the similarity is greater than the first preset similarity threshold, it is preferred to set the range to 0.6~0.8. Generate semantic similarity mapping relationships between the two nodes, and the weight of the corresponding edge is equal to the cosine similarity value of the semantic vectors of the two nodes. The specific rules for generating structural similarity mapping relationships are as follows: determine whether two fields satisfy any of the following conditions: a. The two fields have the same data type and the semantic vector similarity of the field names is greater than the second preset similarity threshold, preferably set in the range of 0.5~0.7; b. Both fields are either primary keys or both are foreign keys; c. The similarity of the table-level semantic vectors of the two fields is greater than the preset table similarity threshold. The threshold is preferably set in the range of 0.6 to 0.8, and the semantic tags of the two fields belong to the same logical function group. When two fields meet any of the conditions, a structural similarity mapping relationship is generated, and the weighted sum of type matching degree, naming similarity degree, and key attribute matching degree is set as the weight. The calculation formula is as follows: ; in, , , Here are adjustable weighting coefficients, and `type_match` is the type matching score, where 1 represents a match and 0 represents a non-match. This is the cosine similarity value of the field name vectors. key_similarity is the key attribute matching score, which is 1 if they are both primary keys / foreign keys, and 0 otherwise. The specific rules for generating behavior association mapping relationships are as follows: Statistically analyze the co-occurrence of two fields within a preset statistical period. When the number of times two fields co-occur in the same SQL statement or the same service call chain within the preset statistical period is greater than a preset threshold, a behavior association mapping relationship is generated between the two fields. The edge weight is equal to the ratio of the number of co-occurrences to the maximum total access frequency of the two fields within the statistical period.

[0030] S33. Generate synonymous mappings across data sources and calculate the initial confidence of the mappings: The specific rules for generating cross-data source synonym mapping relationships are as follows: when the weight of the semantic similarity mapping relationship between two nodes belonging to different data sources is greater than the preset semantic threshold, and the weight of the structural similarity mapping relationship is greater than the preset structural threshold, a cross-data source synonym mapping relationship is generated between the two nodes to identify equivalent fields and concepts between different systems.

[0031] The initial confidence scores of all mapping relationships in the semantic bridging graph are uniformly calculated using a weighted fusion method. ; in , , These are adjustable weighting coefficients. , , These represent the weights of semantic similarity mapping, structural similarity mapping, and behavioral association mapping, respectively. The weights are 0 when there is no corresponding edge.

[0032] S34. Based on the generated mapping relationships and initial confidence levels, the synonym mappings of indirect associations are mined through a confidence propagation mechanism to improve the accuracy of field-concept attribution. The specific steps are as follows: (1) Derivation of indirect association confidence: For two nodes that are indirectly associated through at least two consecutive edges, the mapping confidence of each indirect association path is calculated as the product of the mapping relation weights corresponding to all edges on the path. The maximum value of the mapping confidence of all indirect association paths is taken as the initial confidence of the indirect association mapping between the two nodes. (2) Iterative update of initial confidence of mapping: The label propagation algorithm or PageRank algorithm is used to update the initial confidence of mapping between each field node and the corresponding candidate concept according to the preset iteration rules. The preset iteration rules are as follows: the initial confidence of mapping of field node in the (t+1)th iteration is equal to the sum of the weights of all adjacent edges connected to the field node and the initial confidence of mapping of the candidate concept corresponding to the adjacent node in the tth iteration; the iteration stops when the difference between the initial confidence of mapping of two consecutive iterations is less than the preset stability threshold. (3) Confidence normalization: The initial confidence of the mapping between all field nodes and corresponding candidate concepts is normalized so that the value of the initial confidence of all mappings is in the range of 0 to 1. (4) Field concept attribution optimization: Based on the updated initial confidence of the mapping, the candidate concepts to which the field node belongs are re-determined, and the candidate concept with the highest initial confidence of the mapping with the corresponding field node and greater than the preset attribution threshold is taken as the final attribution concept of the field node.

[0033] Step S3 above constructs a semantic bridging graph that integrates semantic, structural, and behavioral features to achieve automatic semantic alignment of fields across systems. Combined with a confidence propagation mechanism, it mines indirect synonym mappings, enabling semantic unification of multi-source heterogeneous data without manual sorting, thus providing a reliable mapping foundation for subsequent ontology generation.

[0034] S4. Extract mapping relationships from the semantic bridging graph that meet preset confidence requirements, and generate a domain ontology containing concept classes, attributes, relationships, and hierarchical structures; this step specifically includes: S41. Perform multi-dimensional fusion calculation on each mapping relationship in the semantic bridging graph to obtain the final confidence level of each mapping relationship. The multi-dimensional fusion calculation process specifically involves weighted summation of the initial confidence level of the corresponding mapping relationship with the data statistical mapping weights. The calculation formula is as follows: ; in, The initial confidence level of the mapping generated in step S3; The data statistical mapping weights are calculated based on the data feature information collected in step S1. The calculation dimensions include field value distribution similarity, enumeration overlap, and numerical range consistency. The weight values ​​range from 0 to 1, and the higher the feature matching degree, the greater the weight. , It is an adjustable weighting coefficient that can be dynamically adjusted according to the scenario.

[0035] S42. The mapping relationships in the semantic bridging graph are graded according to the final confidence level. All thresholds can be flexibly adjusted according to the business scenario. The default grading rules are as follows: a. Mapping relationships with a final confidence level not less than the first preset confidence threshold (preferably 0.85) are determined to be high-confidence mappings and are directly applied to the construction of the domain ontology; b. The mapping relationship between the final confidence level and the first preset confidence threshold and the second preset confidence threshold (preferably 0.7) is determined to be a medium-to-high confidence mapping relationship and participates in the construction of candidate ontology structures; c. The mapping relationship between the final confidence level and the second preset confidence threshold and the third preset confidence threshold (preferably 0.5) is determined as a medium confidence mapping relationship, which is used for relation completion and inference; d. Mapping relationships with a final confidence level less than the third pre-set confidence threshold are determined to be low-confidence mapping relationships and will not participate in ontology construction.

[0036] S43. Based on the filtered mapping relationship, generate a domain ontology that includes concept classes, attributes, relationships, and hierarchical structure; The generation rule for concept classes in the domain ontology is as follows: traverse all clusters corresponding to candidate concepts. When the number of fields contained in the cluster corresponding to the candidate concept is greater than the preset minimum cluster size, and the average semantic similarity of the fields within the cluster is greater than the preset average cluster similarity threshold, the candidate concept is mapped to a concept class. The generation rule for attributes in the domain ontology is as follows: based on the high confidence mapping relationship between fields and concept classes, fields with a mapping confidence of greater than a preset attribute threshold are used as attributes of the corresponding concept classes. Among them, fields belonging to the primary key are mapped as identification attributes to uniquely identify concept instances, and the remaining fields are mapped as data attributes to describe the business characteristics of the concept. The rules for generating relationships in the domain ontology are as follows: High-confidence mapping relationships are filtered out. A relationship is generated between two concept classes if any of the following conditions are met between the fields corresponding to those two concept classes: a. There is an explicit foreign key dependency between the fields; b. The number of times the field appears in the SQL statement or service call chain within the preset statistical period meets the preset requirements; c. The semantic similarity mapping relationship between the two concept classes satisfies the confidence requirement.

[0037] The generation rule for the hierarchical structure in the domain ontology is as follows: when two concept classes satisfy any of the following conditions, the first concept class is determined to be a subclass of the second concept class: a. All corresponding fields of the first concept class belong to the corresponding field set of the second concept class; b. The semantic similarity between two concept classes is greater than the preset sub-semantic threshold, and the semantic granularity of the first concept class is finer, belonging to the sub-semantic of the second concept class; The hierarchical structure satisfies two constraints: first, the acyclic constraint (DAG), which means it must be a directed acyclic structure and cyclic parent-child relationships are prohibited; second, the maximum depth limit, which means the maximum depth of the hierarchy does not exceed a preset depth threshold to avoid the structure from being too complex.

[0038] The above step S4 avoids low-quality mapping from interfering with the quality of ontology generation by using confidence-based grading and screening. The generation rules of all ontology elements are based entirely on the objective mapping results of the semantic bridging graph, without the need for manual definition of concepts, attributes and relationships.

[0039] S5. Filter mapping relationships in the semantic bridging graph whose confidence level is in a preset low confidence interval and push them to manual verification. Update the mapping relationship weights and structure of the semantic bridging graph based on the manual verification results; specifically including: S51. Filter mapping relationships that require manual verification, including two categories: (1) The mapping relationship with the final confidence level in the preset low confidence interval is automatically judged as having high system uncertainty and requires manual confirmation. (2) Mapping relationships with mapping conflicts exist. Mapping conflicts include: a. Multi-attribution conflict where a single field is simultaneously mapped to multiple candidate concepts; b. There is a conflict of circular dependencies between concepts, such as A is the parent class of B, B is the parent class of C, and C is the parent class of A; c. Bidirectional conflict in the direction of the association, such as the simultaneous existence of high-confidence association mappings A→B and B→A.

[0040] The selection process employs an active learning priority ranking rule: prioritizing models with the smallest difference between their confidence score and 0.5 for manual verification, thereby maximizing model performance improvement with minimal manual annotation.

[0041] S52. Receive manually input structured verification operations. All operations are converted into structured training samples, eliminating the need for manual input of unstructured feedback. The structured verification operations and corresponding processing rules are as follows: a. Confirmation operation: The correctness of the current mapping relationship is manually verified, the weight of the corresponding mapping relationship is set to 1, it is marked as a positive training sample, and the corresponding sample label is 1; b. Rejection operation: Manually deny the correctness of the current mapping relationship, set the weight of the corresponding mapping relationship to 0, mark it as a negative training sample, and set the corresponding sample label to 0; c. Redirection operation: Manually adjust the candidate concepts of field affiliation, set the weight of the original mapping relationship to 0, set the weight of the new mapping relationship to 1, mark it as a corrected training sample, and the corresponding sample label is -1.

[0042] S53. Based on the training samples generated from all structured verification operations, perform three types of optimization operations: (1) Update the mapping weights and topology of the semantic bridging graph: delete rejected mappings, add redirected mappings, and adjust the weights of confirmed mappings to a fixed value of 1. Subsequent iterations will not modify these weights. (2) Adjust the semantic vector calculation parameters and mapping threshold parameters: Add the labeled samples to the semantic model fine-tuning training set. When the cumulative number of labeled samples reaches the preset number, fine-tune the semantic vectorization model parameters. At the same time, dynamically adjust the core thresholds such as semantic similarity threshold and clustering parameters according to the verification results. (3) Recalculate the initial and final confidence of the affected mapping relationships: Only recalculate the confidence of the mappings that are directly or indirectly related to the verification object, without recalculating the entire mapping, so as to avoid affecting the confirmed high-confidence mappings.

[0043] S6. When a change in the data source structure or the addition of a new data source is detected, the semantic bridging graph is partially updated, and the domain ontology structure related to the changed content is incrementally reconstructed; the specific steps include: S61. Poll or receive event trigger signals according to a preset period to detect the status of distributed heterogeneous data sources. When a change in the structure of an existing data source or the access of a new data source is detected, collect the corresponding active metadata. S62. Calculate the mapping relationship between newly added or changed nodes and existing nodes in the semantic bridging graph that have semantic similarity, structural association or behavioral association, and the corresponding initial confidence of the mapping, without recalculating the mapping relationship of all nodes. A local confidence propagation mechanism is executed on the initial confidence of the updated mapping. The confidence iterative propagation is only performed within the 3rd order neighborhood of the newly added / changed nodes, without the need for full iteration. After the update is completed, the mapping relationship weights and topology of the semantic bridging graph are locally adjusted. If the automatically calculated mapping relationship conflicts with the historical manual verification results, the historical manual verification results shall be used as the final mapping result to avoid overwriting the confirmed correct mapping.

[0044] S63. Adjust the concept classes, attributes, relationships, and hierarchical structure related to the changed content in the domain ontology, and incrementally reconstruct to obtain the updated domain ontology.

[0045] Therefore, this invention provides a method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging. It employs a core process of virtualized federated collection of full-dimensional proactive metadata, construction of a semantic bridging graph integrating semantic, structural, and behavioral features to achieve cross-source semantic alignment, and human-machine collaborative verification and local incremental updates based on confidence levels to achieve dynamic ontology evolution. This realizes a closed-loop domain ontology construction process encompassing full-dimensional proactive collection, cross-domain semantic mapping, automated ontology generation, and dynamic incremental maintenance. It addresses the core pain points of traditional domain ontology construction schemes, such as high costs and long cycles due to reliance on manual sorting, low accuracy in identifying unannotated legacy fields based solely on static metadata, high cross-system semantic conflict rates, and high costs associated with full-scale reconstruction and evolution during business changes. This significantly reduces the human and time costs of domain ontology construction, ensures semantic consistency and business adaptability of the domain ontology, and ultimately provides a standardized and lightweight semantic foundation support solution for enterprise-level multi-source heterogeneous data governance, cross-system data interoperability, and large-scale deployment of industry knowledge graphs through low-manual dependence and highly scalable automated construction capabilities.

[0046] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging, characterized in that, Includes the following steps: S1. Connect to several distributed heterogeneous data sources and collect active metadata from each data source; S2. Extract semantic features from the active metadata, generate semantic vectors corresponding to each field, and obtain a candidate concept set based on semantic vector clustering; S3. Using the fields, tables, and candidate concepts of each data source as core nodes, and the semantic similarity mapping relationship, structural similarity mapping relationship, and behavioral association mapping relationship between nodes as core edges, construct a semantic bridging graph; S4. Extract the mapping relationships in the semantic bridging graph that meet the preset confidence requirements, and generate a domain ontology that includes concept classes, attributes, associations and hierarchical structures; S5. Filter the mapping relationships in the semantic bridging graph whose confidence level is in the preset low confidence interval and push them to manual verification. Update the mapping relationship weights and structure of the semantic bridging graph according to the manual verification results. S6. When a change in the data source structure or the addition of a new data source is detected, the semantic bridging graph is locally updated, and the domain ontology structure related to the changed content is incrementally reconstructed.

2. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging as described in claim 1, characterized in that, Step S1 specifically involves: connecting to several distributed heterogeneous data sources through a virtualized federated unified access layer. The distributed heterogeneous data sources include at least one of relational databases, NoSQL databases, data files, and application programming interfaces; collecting active metadata from each data source, which includes four categories: static structural information, field semantic information, runtime access behavior information, and data characteristic information.

3. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging as described in claim 1, characterized in that, Step S2 specifically includes: S21. Extract the field name, field annotation, and a preset number of field data samples corresponding to each field from the active metadata, and perform semantic vectorization transformation on each field to obtain field name vector, field annotation vector, and data sample vector. S22. The field name vector, field annotation vector, and data sample vector are weighted and fused to obtain the semantic vector of the corresponding field. S23. Based on the precondition of data type matching, the semantic vectors of all fields are clustered. Each cluster corresponds to a candidate concept. After naming standardization and semantic vector generation of all candidate concepts, a set of candidate concepts is obtained.

4. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging as described in claim 3, characterized in that, In step S3, the specific rules for generating semantic similarity mapping relationships are as follows: when the semantic vector similarity between two nodes is greater than the first preset similarity threshold, a semantic similarity mapping relationship is generated between fields or between a field and a candidate concept, and its weight is set as the semantic vector similarity value between the two nodes. The specific rules for generating structural similarity mapping relationships are as follows: determine whether two fields satisfy any of the following conditions: a. The two fields have the same data type and the semantic vector similarity of the field names is greater than the second preset similarity threshold; b. Both fields are either primary keys or both are foreign keys; c. The similarity of the table-level semantic vectors of the two fields is greater than the preset table similarity threshold, and the logical function groups to which the semantic labels of the two fields belong are consistent; When two fields meet any of the conditions, a structural similarity mapping relationship is generated, and the weighted sum of type matching degree, naming similarity degree, and key attribute matching degree is set as the weight. The specific rules for generating behavior association mapping relationships are as follows: when the number of times two fields co-occur in the same SQL statement or the same service call chain within a preset statistical period is greater than a preset threshold, a behavior association mapping relationship is generated between the two fields, and its weight is set as the ratio of the number of co-occurrences to the total access frequency of the two fields within the preset statistical period.

5. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 4, characterized in that, The semantic bridging graph also includes cross-data source synonym mapping relationships. The specific rules for generating cross-data source synonym mapping relationships are as follows: when the weight of the semantic similarity mapping relationship between two nodes belonging to different data sources is greater than a preset semantic threshold, and the weight of the structural similarity mapping relationship is greater than a preset structural threshold, a cross-data source synonym mapping relationship between the two nodes is generated. The initial confidence calculation rule for all mapping relationships in the semantic bridging graph is as follows: multiply the weights of the semantic similarity mapping relationship, the structural similarity mapping relationship, and the behavioral association mapping relationship with the preset semantic weight coefficient, preset structural weight coefficient, and preset behavioral weight coefficient respectively, and then sum them.

6. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 5, characterized in that, Step S3, the construction of the semantic bridging graph, also includes a confidence propagation mechanism, specifically: (1) For two nodes that are indirectly associated through at least two consecutive edges, the mapping confidence of each indirect association path is calculated as the product of the mapping relation weights corresponding to all edges on the path. The maximum mapping confidence of all indirect association paths is taken as the initial confidence of the indirect association mapping between the two nodes. (2) Update the initial confidence of the mapping between each field node and the corresponding candidate concept according to the preset iteration rule. The preset iteration rule is: the initial confidence of the mapping of the field node in the (t+1)th iteration is equal to the sum of the weights of all adjacent edges connected to the field node and the initial confidence of the mapping of the candidate concept corresponding to the adjacent node in the tth iteration. (3) Normalize the initial confidence of the mapping between all field nodes and corresponding candidate concepts so that the value of the initial confidence of all mappings is in the range of 0 to 1; (4) Based on the updated initial confidence of the mapping, re-determine the candidate concepts to which the field node belongs, and take the candidate concept with the highest initial confidence of the mapping to the corresponding field node and greater than the preset belonging threshold as the final belonging concept of the field node.

7. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 6, characterized in that, Step S4 specifically includes: S41. Perform multi-dimensional fusion calculation on each mapping relationship in the semantic bridging graph to obtain the final confidence of each mapping relationship. The multi-dimensional fusion calculation process specifically involves weighted summation of the initial confidence of the corresponding mapping relationship with the data statistical mapping weight. S42. The mapping relationships in the semantic bridging graph are graded according to the final confidence level: a. Mapping relationships with a final confidence level not less than the first preset confidence threshold are determined to be high-confidence mappings and are directly applied to domain ontology construction; b. The mapping relationship between the final confidence level and the first preset confidence threshold and the second preset confidence threshold is determined to be a medium-to-high confidence mapping relationship and participates in the construction of candidate ontology structures; c. The mapping relationship between the final confidence level and the second and third preset confidence thresholds is determined as a medium confidence mapping relationship, which is used for relationship completion and inference; d. Mapping relationships with a final confidence level less than the third preset confidence threshold are determined to be low-confidence mapping relationships and will not participate in ontology construction; S43. Based on the filtered mapping relationship, generate a domain ontology that includes concept classes, attributes, relationships and hierarchical structure.

8. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 7, characterized in that, The generation rule for concept classes in the domain ontology is as follows: when the number of fields contained in the cluster corresponding to a candidate concept is greater than the preset minimum cluster size, and the average semantic similarity of the fields within the cluster is greater than the preset average cluster similarity threshold, the candidate concept is mapped to a concept class. The generation rule for attributes in the domain ontology is as follows: based on the high confidence mapping relationship between fields and concept classes, fields with a mapping confidence of greater than a preset attribute threshold are used as attributes of the corresponding concept classes. Among them, fields belonging to the primary key are mapped as identifier attributes, and the remaining fields are mapped as data attributes. The generation rules for relationships in the domain ontology are as follows: filter high-confidence mapping relationships. When there is a foreign key dependency, co-occurrence relationship between SQL statements and call chains, or semantic similarity mapping between fields corresponding to two concept classes, generate a relationship between the two concept classes. The generation rule for the hierarchical structure in the domain ontology is as follows: if all corresponding fields of the first concept class belong to the corresponding fields of the second concept class, or if the first concept class is a subordinate semantic of the second concept class, then the first concept class is determined to be a subclass of the second concept class.

9. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 7, characterized in that, Step S5 specifically includes: S51. Filter the mapping relationships that require manual verification. The mapping relationships include two types: (1) The mapping relationship where the final confidence level is in the preset low confidence interval; (2) Mapping relationships with mapping conflicts, including multi-attribution conflicts where a single field is simultaneously mapped to multiple candidate concepts, conflicts where there is circular dependency between concepts, and bidirectional conflicts in the direction of association. S52. Receive manually input structured verification operation, wherein the structured verification operation and corresponding processing rules are as follows: a. Confirmation operation: Set the weight of the corresponding mapping relationship to 1 and mark it as a positive training sample; b. Rejection operation: Set the weight of the corresponding mapping relationship to 0 and mark it as a negative training sample; c. Redirection operation: Adjust the candidate concepts of the corresponding field and mark them as corrected training samples; S53. Based on the training samples generated by all structured verification operations, update the mapping relationship weights and topology of the semantic bridging graph, and adjust the semantic vector calculation parameters and mapping threshold parameters.

10. The method for automatic construction and dynamic evolution of domain ontology based on proactive metadata and semantic bridging according to claim 7, characterized in that, Step S6 specifically includes: S61. Poll or receive event trigger signals according to a preset period to detect the status of distributed heterogeneous data sources. When a change in the structure of an existing data source or the access of a new data source is detected, collect the corresponding active metadata. S62. Calculate the mapping relationship between the newly added or changed node and the existing nodes in the semantic bridging graph that have semantic similarity, structural association or behavioral association, and the corresponding initial confidence of the mapping. Then, execute the local confidence propagation mechanism on the updated initial confidence of the mapping and locally update the mapping relationship weight and topology of the semantic bridging graph based on the calculation results. S63. Adjust the concept classes, attributes, relationships, and hierarchical structure related to the changed content in the domain ontology, and incrementally reconstruct to obtain the updated domain ontology.

Images