An ontology-enhanced metadata intelligent management method based on semantic linkage

Abstract

The application provides a semantic linkage-based ontology-enhanced metadata intelligent management method, and belongs to the field of data governance and metadata management. The method solves the problem that traditional schemes cannot meet the high-precision, high-efficiency and high-reliability requirements in current data governance scenarios. The method comprises the following steps: obtaining metadata to be managed; constructing a domain ontology; establishing a semantic mapping of the metadata to the domain ontology; based on the semantic mapping, using semantic knowledge of the domain ontology to automatically annotate the metadata to generate annotation labels; in response to a change event of the domain ontology or a change event of the metadata, executing a bidirectional linkage update protocol, propagating changes according to an influence range of the semantic mapping, synchronously updating metadata annotation or feeding back adjustment of the domain ontology; and based on a preset consistency rule set, detecting and updating a consistency state between the metadata and the domain ontology. The method is mainly used in the field of large-scale data asset management.

Description

Technical Field

[0001] This invention belongs to the field of data governance and metadata management, and in particular relates to an ontology-enhanced metadata intelligent management method based on semantic linkage. Background Technology

[0002] As enterprises advance their digital transformation, data platforms, data lakes, and enterprise-level data asset catalogs have become core infrastructures for various organizations to unlock the value of their data. Metadata, as data describing data, carries the technical characteristics, business meaning, and management attributes of the data, and is a core tool for data governance. In large-scale data asset management scenarios, the efficient management and accurate maintenance of metadata directly determine the availability, reliability, and compliance of data assets.

[0003] Current mainstream metadata extraction tools generally rely on database system catalogs to obtain technical metadata, automatically collecting technical information such as table names, field names, data types, length constraints, and primary / foreign key relationships. However, the collection of business and management metadata still heavily depends on manual input. Business metadata includes the business meaning of fields, their data domain, sensitivity level, and data responsibility, while management metadata includes data quality level, update time, and compliance status. Given the scale of enterprise data warehouses, which often manage tens of thousands of tables and hundreds of thousands of fields, manual input is not only extremely inefficient but also highly prone to labeling errors. For example, in the customer information management scenario of the financial industry, fields with the same name but different meanings may be assigned completely different business meanings in different business tables. If confusion occurs during manual input, it can lead to deviations in downstream data analysis results, or even more serious risks such as sensitive data leakage and failure to pass compliance audits.

[0004] Existing metadata annotation primarily employs automated methods based on keyword matching, failing to consider the true meaning of fields within their context. The same field name often corresponds to entirely different semantics in different business contexts. For example, the ID field might represent an order number in an order table, a customer number in a customer table, and a product number in a product table. Keyword matching alone cannot identify such semantic differences. Furthermore, variations in field value encoding rules further exacerbate annotation errors. For instance, the gender field might use different encoding methods such as male / female, M / F, or 1 / 0. Keyword matching methods cannot establish a correspondence between these encodings and standardized semantic concepts, ultimately leading to a disconnect between the annotation results and the actual business semantics of the fields. This makes it difficult for downstream data users to accurately understand the data's meaning based on the metadata.

[0005] In current data governance systems, metadata management and domain ontology management are typically maintained by independent tools, lacking an effective linkage mechanism. Domain ontology, as the semantic foundation for accumulating industry knowledge and business rules, defines a standardized conceptual system, attribute constraints, and relationships between concepts. Metadata, as the carrier describing specific data resources, has long suffered from a disconnect in semantic association with the ontology. When conceptual definitions or attribute constraints in the ontology change, the associated metadata tags cannot be updated synchronously; similarly, when the actual data characteristics reflected by the metadata change, they cannot be promptly fed back to the ontology for iterative optimization. For example, when the ontology undergoes a version upgrade, subdividing the original mobile phone number attribute into domestic and overseas mobile phone numbers, all metadata tags associated with this attribute will still use the old annotation results, leading to a long-term misalignment between metadata semantics and ontology definitions.

[0006] Because metadata and ontology are maintained independently, there is a lack of a systematic mechanism to ensure consistency between them. Existing solutions lack unified verification rules for the existence of overlay annotations, type compatibility, value range matching, and relational integrity, and also lack a dynamic inconsistency detection and repair process. When metadata values ​​violate the constraints defined in the ontology—for example, if the ontology defines the legal value range for a gender attribute as male, female, or unknown, but a gender field actually stores values ​​of M, F, or N / A—such semantic conflicts often go undetected. These problems only surface when data extraction, transformation, or loading operations fail, or when downstream data analysis shows significant deviations, severely impacting the stability of the data chain and the reliability of data applications. Furthermore, when the actual data distribution shifts—for example, a field that previously only stored domestic mobile phone numbers begins to include overseas mobile phone numbers—such changes cannot reverse the iteration of the ontology's constraint rules, causing the semantic definition of the ontology to gradually decouple from the actual data characteristics.

[0007] Existing metadata management solutions lack a systematic quality evaluation mechanism, making it impossible to quantitatively assess the completeness, accuracy, and timeliness of metadata. Managers struggle to grasp the overall quality level of metadata through a unified indicator system, and are unable to accurately pinpoint problematic metadata entries. In most scenarios, metadata quality can only be roughly assessed through manual sampling, which cannot support continuous optimization of the entire dataset. Ultimately, this leads to a gradual deterioration of metadata quality, resulting in a loss of its descriptive and guiding function for data assets.

[0008] The core root of the above problems lies in the fact that existing technologies have not established a semantic linkage mechanism between metadata and domain ontology. The processing logic of each link is independent of each other. Faced with the management needs of massive enterprise-level metadata, the maintenance cost of traditional solutions is increasing exponentially, and they can no longer meet the requirements of high precision, high efficiency and high reliability in the current data governance scenario. Summary of the Invention

[0009] In view of this, the present invention aims to propose an ontology-enhanced metadata intelligent management method based on semantic linkage, in order to solve the problem that traditional solutions cannot meet the requirements of high precision, high efficiency and high reliability in the current data governance scenario.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: an ontology-enhanced metadata intelligent management method based on semantic linkage, the method comprising: Obtain the metadata to be managed, including technical metadata, business metadata, and management metadata; Construct a domain ontology, which is defined using description logic as a quintuple. ,in, For a set of concepts, For attribute set, For the axiom set, For instance set, For the range type set; Establish a semantic mapping from metadata to the domain ontology. The semantic mapping For a ternary relation ,in, For the ontology concept of mapping, For the metadata element to be mapped, For semantic similarity scores, For the set of ontology concepts, A collection of metadata elements Semantic similarity is calculated by comprehensively considering naming similarity, semantic similarity, structural compatibility, and value range matching. Based on the semantic mapping The semantic knowledge of the domain ontology is used to automatically annotate the metadata and generate annotation tags; In response to a change event in the domain ontology or a change event in the metadata, a bidirectional linkage update protocol is executed, based on the semantic mapping. The scope of influence changes, and the metadata annotation is updated synchronously or the domain ontology is adjusted accordingly. Based on a preset set of consistency rules, the consistency status between the updated metadata and the domain ontology is detected and updated.

[0011] Furthermore, a preferred method is proposed, wherein the semantic mapping is used as a basis for... The steps for automatically annotating metadata using the semantic knowledge of the domain ontology specifically include: Extracting multidimensional feature vectors from metadata elements to be labeled The multidimensional feature vector includes naming similarity features, semantic similarity features, structural compatibility features, value range matching features, and ontology constraint features; Calculate the probability distribution of the metadata elements belonging to each candidate ontology tag. :

[0012] in, For the domain ontology, For the feature weight vector, As candidate tags, A collection of ontology concept tags. For the feature function vector, Candidate ontology labels to be labeled Any element in the set of ontology concept tags; Select the label with the highest probability as the best label. : .

[0013] Furthermore, a preferred approach is proposed, in which the automatic annotation step further includes context-aware collaborative annotation: Construct a collaborative annotation objective function for a set of fields in the same data table. :

[0014] in, The strength of the relationship between the tag and the ontology. For the first One ontology tag, For the first One ontology tag, , For tag indexing; For labeled probabilities; The strength of the association between fields is determined based on foreign key dependencies or coexistence relationships within the same table. For the associated first Each metadata field For the associated first Each metadata field.

[0015] Furthermore, a preferred embodiment is proposed, wherein the step of executing the bidirectional linkage update protocol specifically includes: Define linkage rules ,in, Event types include ontology change events, metadata annotation completion events, data drift events, and consistency anomaly events; This is the trigger condition for the rule; A set of operations to be performed; For synchronization direction; When the trigger direction is from ontology to metadata, update the tag name and constraint description of the associated metadata according to the changed content, or downgrade the metadata label associated with the deleted concept to a pending confirmation status. When the trigger direction is from metadata to ontology, suggestions for ontology concept subdivision, constraint modification, or addition of concepts are generated based on the statistical regularity of metadata annotation, value range distribution drift, or common characteristics of unannotated fields.

[0016] Furthermore, a preferred approach is proposed, in which the scope of impact of the change is defined in the step of synchronously updating metadata annotation:

[0017] in, For subject to changes in the subject The set of metadata affected Metadata m and ontology changes The relationships between metadata tags; these relationships are propagated through semantic mapping, where the distance between the concept corresponding to the metadata tag and the changed concept is considered. If the data is found to be affected metadata, then k is a preset propagation depth threshold. , These are concept nodes within the ontology; Furthermore, a preferred approach is proposed, wherein the step of detecting and updating the consistency state between the metadata and the domain ontology specifically includes: The detection is performed using a formally defined set of predicate logic rules, which includes: Label Existence Rules : ; Data type compatibility rules : , The actual type of metadata m must be a member of the value range defined by tag(m), or a subtype of a certain type in that value range; Value range constraint satisfies rules : , This indicates that the actual value of metadata m must belong to or be a subtype of the value range corresponding to tag(m) in Value(m); Namespace Consistency Rules : , The namespace of the metadata tag(m) must be consistent with the current ontology version. The namespaces are consistent; Relationship integrity rules: , Representing concepts in the ontology and The attribute relationships between them; Metadata The annotation tags correspond to the ontology concept. ; Metadata The annotation tags correspond to the ontology concept. .

[0018] Furthermore, a preferred approach is proposed, which includes a business metadata derivation step after obtaining the metadata to be managed: The field names in the technical metadata are segmented and mapped to the concept tags of the domain ontology; Based on the value distribution characteristics and pattern matching results of the field, the business type of the field is inferred; Based on the keyword pattern of field names and the uniqueness of value ranges, the sensitivity level in management metadata is derived.

[0019] Furthermore, a preferred embodiment is proposed, wherein the method further includes a metadata quality measurement step: Construct a five-dimensional quality assessment system and calculate the comprehensive quality score:

[0020] in, The weight coefficient for the k-th quality dimension; The quality score of metadata MD in the k-th dimension; k This is the preset propagation depth threshold.

[0021] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes an ontology-enhanced metadata intelligent management method based on semantic linkage as described in any of the preceding claims.

[0022] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of an ontology-enhanced metadata intelligent management method based on semantic linkage as described above.

[0023] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a semantic linkage management mechanism that deeply couples metadata and domain ontology. Its core lies in breaking down the traditional separation between metadata management and ontology management, and incorporating discrete metadata entries into a unified domain knowledge framework through formal semantic mapping. First, a rigorous mathematical definition of the domain ontology is established using descriptive logic, clarifying the concept set, attribute set, axiom set, and value range constraints, forming a semantic foundation with logical reasoning capabilities. Based on this, a multi-dimensional mapping relationship is established between metadata elements and ontology concepts. This mapping is not a simple keyword correspondence, but a multi-dimensional matching process that comprehensively considers naming features, semantic context, data structure compatibility, and actual value range distribution, thereby achieving precise anchoring from surface data features to underlying business semantics. In the annotation stage, this invention introduces a context-aware collaborative mechanism. Unlike processing individual fields in isolation, this method treats the set of fields within the same data entity as a whole, utilizing the predefined inter-concept relationship strength in the ontology, combined with the physical association features between fields, to globally optimize the initial annotation results. This collaborative mechanism effectively resolves semantic ambiguity of a single field, ensuring that the annotation results of each field within the same data table are semantically consistent and conform to the pattern constraints defined in the ontology. More importantly, this invention designs a bidirectional, driven, and interconnected update protocol. On one hand, when the conceptual definitions and attribute constraints of the domain ontology change, the system can automatically define the affected scope based on semantic mapping relationships and propagate the changes to all associated metadata instances, achieving synchronous iteration of metadata annotation. On the other hand, when a drift in the actual data distribution is detected, or when a large number of unannotated fields exhibit stable common characteristics, the system can feed these changes back to the ontology side to assist in decision-making regarding concept subdivision or the expansion of new concepts. This bidirectional feedback loop enables metadata and ontology to co-evolve, maintaining semantic consistency at all times. Finally, this invention establishes a consistency maintenance system based on predicate logic. By defining a formal rule set covering annotation existence, type compatibility, value domain constraint satisfaction, and relational integrity, the matching status of metadata and ontology is continuously verified. Once an inconsistency is detected, the system can automatically execute repairs or trigger manual intervention processes according to preset strategies, ensuring the accuracy and reliability of metadata semantics from a mechanism perspective.

[0024] This invention significantly improves the automation level and semantic accuracy of metadata management. By introducing the semantic knowledge of the domain ontology into the metadata extraction and annotation process, it greatly reduces the reliance on manual input and effectively avoids the subjective arbitrariness and error risks brought about by manual operation. The annotation model based on multi-dimensional feature fusion can accurately identify the true business meaning of fields. Even when faced with fields with the same name but different meanings or complex encoding rules, it can make correct judgments based on context and data features, fundamentally solving the problem of low accuracy in traditional keyword matching methods.

[0025] This invention achieves dynamic consistency maintenance between metadata and ontology. The bidirectional, linked update mechanism breaks down the static isolation between the two, ensuring that changes on one are reflected in the other promptly and accurately. This not only eliminates semantic conflicts caused by the disconnect between definitions and instances but also significantly reduces the metadata maintenance costs associated with ontology version iterations. Data administrators no longer need to manually correct affected metadata entries one by one; the system can automatically complete most of the synchronization work, ensuring the real-time nature and authority of the data asset catalog.

[0026] This invention enhances the system's adaptability and evolution capabilities. By summarizing new knowledge from metadata schemas and feeding it back into the ontology, the system possesses a certain degree of self-improvement ability. When business expansion brings new data features, the system can promptly perceive and suggest expanding or refining the ontology's conceptual framework, enabling the domain knowledge base to continuously evolve with business development and avoiding the problem of traditional static ontology gradually becoming rigid and ineffective.

[0027] This invention improves the stability of the data link and the reliability of data applications. Strict consistency verification rules can identify and address potential problems such as metadata and ontology definition conflicts and value range constraint violations before issues are discovered at the data consumption end. This effectively reduces downstream data processing failures or analysis result deviations caused by misleading metadata, enhancing the robustness of the entire data governance system. Attached Figure Description

[0028] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a system architecture diagram of the ontology-enhanced metadata intelligent management method described in this invention; Figure 2 This is a schematic diagram of the ontology-metadata association model described in this invention; Figure 3 This is a schematic diagram of the semantic linkage propagation mechanism described in this invention; Figure 4 This is a comparative diagram illustrating the metadata enhancement effects described in this invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other, and the described embodiments are only some embodiments of the present invention, not all embodiments.

[0030] Implementation Method 1: This implementation method proposes an ontology-enhanced metadata intelligent management method based on semantic linkage. The method includes: Obtain the metadata to be managed, including technical metadata, business metadata, and management metadata; Construct a domain ontology, which is defined using description logic as a quintuple. ,in, For a set of concepts, For attribute set, For the axiom set, For instance set, For the range type set; Establish a semantic mapping from metadata to the domain ontology. The semantic mapping For a ternary relation ,in, For the ontology concept of mapping, For the metadata element to be mapped, For semantic similarity scores, For the set of ontology concepts, A collection of metadata elements Semantic similarity is calculated by comprehensively considering naming similarity, semantic similarity, structural compatibility, and value range matching. Based on the semantic mapping The semantic knowledge of the domain ontology is used to automatically annotate the metadata and generate annotation tags; In response to a change event in the domain ontology or a change event in the metadata, a bidirectional linkage update protocol is executed, based on the semantic mapping. The scope of influence changes, and the metadata annotation is updated synchronously or the domain ontology is adjusted accordingly. Based on a preset set of consistency rules, the consistency status between the updated metadata and the domain ontology is detected and updated.

[0031] In this embodiment, the semantic mapping-based The steps for automatically annotating metadata using the semantic knowledge of the domain ontology specifically include: Extracting multidimensional feature vectors from metadata elements to be labeled The multidimensional feature vector includes naming similarity features, semantic similarity features, structural compatibility features, value range matching features, and ontology constraint features; Calculate the probability distribution of the metadata elements belonging to each candidate ontology tag. :

[0032] in, For the domain ontology, For the feature weight vector, As candidate tags, A collection of ontology concept tags. For the feature function vector, , These are candidate ontology labels to be labeled.

[0033] Select the label with the highest probability as the best label. : .

[0034] Furthermore, the automatic annotation step also includes context-aware collaborative annotation: Construct a collaborative annotation objective function for a set of fields in the same data table. :

[0035] in, The strength of the relationship between the tag and the ontology. , For different ontology tags; , For tag indexing; For labeled probabilities; The strength of the association between fields is determined based on foreign key dependencies or coexistence relationships within the same table. , For associated metadata fields; Furthermore, the step of executing the bidirectional linkage update protocol specifically includes: Define linkage rules ,in, Event types include ontology change events, metadata annotation completion events, data drift events, and consistency anomaly events; This is the trigger condition for the rule; A set of operations to be performed; For synchronization direction; When the trigger direction is from ontology to metadata, update the tag name and constraint description of the associated metadata according to the changed content, or downgrade the metadata label associated with the deleted concept to a pending confirmation status. When the trigger direction is from metadata to ontology, suggestions for ontology concept subdivision, constraint modification, or addition of concepts are generated based on the statistical regularity of metadata annotation, value range distribution drift, or common characteristics of unannotated fields.

[0036] Furthermore, in the step of synchronously updating metadata annotations, the scope of the change's impact is defined:

[0037] in, For subject to changes in the subject The set of metadata affected Metadata m and ontology changes The relationships between metadata tags; these relationships are propagated through semantic mapping, where the distance between the concept corresponding to the metadata tag and the changed concept is considered. If so, it is determined to be affected metadata. k The preset propagation depth threshold, , These are concept nodes within the ontology; In this embodiment, the step of detecting and updating the consistency status between the metadata and the domain ontology specifically includes: The detection is performed using a formally defined set of predicate logic rules, which includes: Label Existence Rules : ; Data type compatibility rules : , The actual type of metadata m must be The members of the defined value range, or a subtype of a certain type in the value range; Value range constraint satisfies rules : , The actual value Value(m) of the metadata m must belong to the value range defined by tag(m), or be a subtype instance of the type within that value range; Namespace Consistency Rules : , The namespace representing the metadata tag(m) Compared to the current version namespace ; Relationship integrity rules: , Representing concepts in the ontology and Attribute relationships between Metadata The annotation tags correspond to the ontology concept. Metadata The annotation tags correspond to the ontology concept. .

[0038] In this embodiment, after obtaining the metadata to be managed, a business metadata derivation step is also included: The field names in the technical metadata are segmented and mapped to the concept tags of the domain ontology; Based on the value distribution characteristics and pattern matching results of the field, the business type of the field is inferred; Based on the keyword pattern of field names and the uniqueness of value ranges, the sensitivity level in management metadata is derived.

[0039] In this embodiment, the method further includes a metadata quality measurement step: Construct a five-dimensional quality assessment system and calculate the comprehensive quality score:

[0040] in, The weight coefficient for the k-th quality dimension; The quality score of metadata MD in the k-th dimension; k This is the preset propagation depth threshold.

[0041] Implementation Method 2, see below Figures 1 to 4 This embodiment describes a complete implementation of the ontology-enhanced metadata intelligent management method based on semantic linkage described in Embodiment 1, including: This embodiment is applied to the metadata management scenario of a data platform in a large manufacturing enterprise. This enterprise needs to perform full lifecycle metadata management on 1217 business tables and 23142 fields under its core business database, solving the problems of high manual maintenance costs, low annotation accuracy, and inconsistencies with the semantics of the original metadata. The specific implementation steps of this embodiment are as follows: First, the domain ontology is constructed, and its formal definition is achieved using descriptive logic. Definition 1 gives the five-tuple structure of the domain ontology: Domain Ontology It is a quintuple. ,in A set of concepts, representing the entity types in the domain, where each concept... Corresponding to a unary relation, , , Each represents an entity type; For a set of attributes, each attribute Has a domain Sum range A is the set of axioms, containing the axioms of inclusion. Equivalence axioms Mutually exclusive axioms Constraint Axioms , The size of the instance collection; V is the instance set; V is the value range type set, containing , , Types such as Ontology Knowledge Base ,in For the terminology level, For the assertion layer, Definition 2 gives the criterion for ontology consistency: an ontology O is consistent if and only if there exists an interpretation I such that all TBox axioms and ABox assertions are simultaneously true, i.e. In the customer management ontology constructed in this embodiment, C contains three core concepts: customer, order, and product. P contains attributes such as hasCustomerID, hasName, hasIDCardNumber, hasMobilePhone, and hasGender. Here, hasCustomerID represents whether the customer has been assigned a number, hasName represents the customer name, hasIDCardNumber represents the customer ID number, hasMobilePhone represents the customer's mobile phone number, and hasGender represents the customer's gender. The value range constraint of the hasGender attribute is { Male, Female Unknown}, and define Male, Female, and Unknown as mutually exclusive through the mutual exclusion axiom.

[0042] Subsequently, technical metadata extraction was performed. The metadata directory of the enterprise's core business database was accessed through the database connection interface. The field definitions of all business tables were traversed, and the table name, field name, data type, maximum character length, nullability, default value, and field comment information were obtained sequentially. Examples of extraction results are as follows: Table name t_customer, field name customer_id, data type BIGINT, no length constraint, nullability not allowed, comment "Customer primary key"; Table name t_customer, field name full_name, data type VARCHAR, length 100, nullability not allowed, comment "Customer name"; Table name t_customer, field name id_card_no, data type VARCHAR, length 18, nullability not allowed, comment "ID card number"; Table name t_customer, field name mobile_phone, data type VARCHAR, length 11, nullability allowed, comment "Mobile phone number"; Table name t_customer, field name register_date, data type DATE, no length constraint, nullability not allowed, comment "Registration date". The extraction logic of technical metadata can be formally represented as follows: ,in, For column names; , indicates the data type; , indicating a length constraint; , indicating emptiness; This indicates whether it is a primary key; This indicates a foreign key reference (table name or null); , indicates the default value.

[0043] Next, intelligent derivation of business metadata and management metadata is performed. First, named semantic analysis is executed, and a word segmentation function is defined. ,in, , , These represent the tokens after word segmentation. For example, the field name `cust_birth_dt` is decomposed into `[cust,birth,dt]`, which maps to the business meaning of "customer's date of birth"; the mapping logic from naming to ontology concepts is as follows: ,in Let C be the set of labels for concept C in the ontology. Next, perform data feature inference and define the range feature function: ,in, Set the cardinality for the field's values ​​(the number of distinct values). For unique value proportion, For regular expression pattern matching degree, This refers to the field length distribution characteristics. For example, if a VARCHAR field has only 18 digits and meets the ID card verification rules, it is inferred to be an "ID card number". The inference logic is as follows: , This is a function for defining the column field schema. The relation schema inference is performed again, inferring business relationships based on foreign key relationships. For example, if the `customer_id` field in the `orders` table is a foreign key related to the `id` field in the `customers` table, the business relationship can be inferred to be "orders belong to customers". This can be formally represented as... ,in, For fields The foreign key constraint points to a value in another table; The target field for the foreign key association (the field in the referenced table); For fields The corresponding ontology concept after semantic mapping For fields The ontology concept corresponding to semantic mapping. Taking the id_card_no field as an example, the word segmentation result is [id,card,no], which matches the semantics of an ID card number; all values ​​are 18-digit numbers that conform to the validation rules, further confirming that the business type is an ID card number; the sensitivity level is passed. Calculate, where, This indicates keyword sensitivity based on field name, obtained by calculating the maximum similarity between the field name and the set of sensitive keywords; This represents a predefined set of sensitive keywords; Indicates the original name of the metadata field; This indicates the sensitivity based on value characteristics, evaluated by the uniqueness ratio (cardinality / total number of rows); This indicates the total number of rows (records) in the data table. Because the field name contains the keyword "card" and its values ​​are highly unique, the final sensitivity level was determined to be P4 - Strict Confidential, and the data owner is the Information Security Department. The complete derivation logic for managing metadata is as follows: ,in, The data sensitivity level is indicated by four levels (P1 to P4), where P1 is public and P4 is strictly confidential. Indicates the data quality level (assessed through dimensions such as completeness and accuracy); This indicates the freshness of metadata (the time span between the last update and the current time). This refers to data (the responsible department or individual, such as the information security department).

[0044] After extracting the metadata, a semantic mapping from the metadata to the ontology is established. Definition 4 provides a formal definition of the semantic mapping: Semantic mapping It is a ternary relation: ,in The semantic similarity function integrates naming similarity, semantic similarity, structural compatibility, and value range matching. The calculation logic is as follows: .

[0045] Ontology-driven automatic metadata annotation is carried out based on semantic mapping. The annotation process is modeled as an ontology-based semantic matching problem for metadata elements. Its label The probability distribution is ,in, The feature function vector includes: For naming similarity features, a weighted combination of improved edit distance and word vector cosine similarity is used. ,Right now ,in, Metadata elements The original field name, For the ontology tag l The standard name, The weighted combination coefficients (balancing the weights of edit distance and word vector similarity) ), For string and The minimum edit distance (Levenshtein distance) between them. , The two strings to be compared are usually the tokens after the field name is segmented. , For string , Compiled using a pre-trained word vector model; For semantic similarity features, cosine similarity is based on context embedding vectors. Calculation, i.e. ,in, Metadata elements Contextual information (such as the table name, adjacent field names, comment documentation, etc.). For the ontology tag Definition description (the textual definition of the concept / attribute in the ontology). For context text Embeddings encoded by the BERT pre-trained model Define text Embeddings encoded by a BERT pre-trained model; For structural compatibility features, the computational logic is as follows: ,in, For metadata Technical attributes (including data type, length, whether nullable, etc.) For the ontology tag The defined data type requirements, For metadata The actual data type (such as INT, VARCHAR). For the ontology tag Required data type; For range matching features, the calculation logic is as follows: ,in, For metadata The actual value distribution (statistical distribution of field values). For the ontology tag Defined range of values ​​(allowed range of values ​​or enumeration set). For tags Upper limit / boundary of the value range For metadata The set of value distributions; For ontology-constrained features, the optimal annotation label is: ,in, For the ontology tag The set of associated constraints (including data type constraints, value range constraints, namespace constraints, relational integrity constraints, etc.).

[0046] The annotation results for the t_customer table are as follows: customer_id (identifier attribute) is annotated as hasCustomerID, with a confidence level of 0.97; full_name (description attribute) is annotated as hasName, with a confidence level of 0.95; id_card_no (identifier attribute) is annotated as hasIDCardNumber, with a confidence level of 0.98; mobile_phone (phone number) is annotated as hasMobilePhone, with a confidence level of 0.94; and register_date (registration date) is annotated as hasRegistrationDate, with a confidence level of 0.93.

[0047] To further eliminate semantic ambiguity, context-aware collaborative annotation is performed. This is applied to the table. field collection in The objective function for collaborative annotation is ,in, For tags and Relationship strength in the ontology: ; This represents the shortest path distance between the two concepts in the ontology hierarchy. Co-occurrence frequency; For fields and Association strength: .

[0048] Theorem 1 provides a proof of consistency in collaborative annotation: Let... For a consistent entity, For the optimal solution of collaborative annotation, then for any field Its label Satisfy ontology consistency constraints: The proof is as follows: By the definition of the objective function, when hour, The ontology constraint features must be satisfied. ,Right now , This represents the truth value (TRUE), indicating the consistency constraint of collaborative annotation. Optimal solution. The maximum probability value is selected from all candidates, therefore the constraint must be satisfied. In this embodiment, the collaborative annotation score of the t_customer table is... If the preset confirmation threshold of 1.0 is exceeded, the annotation result will automatically take effect.

[0049] To achieve the co-evolution of metadata and ontology, a two-way linkage update mechanism is established. First, the linkage rules are defined: , in, , where is the event type; condition is the rule trigger condition, expressed using a predicate logic formula; action is the set of operations to be executed; For synchronization direction, This is a bidirectional synchronization identifier, indicating that metadata and ontology can be updated in a bidirectional manner (O→MD and MD→O are performed simultaneously). Metadata refers to the collection of metadata such as technical attributes, business annotations, and management information for each field in a data warehouse. This refers to the domain ontology, which is a formal conceptual model of business domain knowledge (such as the concept set, attribute set, and axiom set in a customer management domain ontology). There are four synchronization rules from ontology to metadata: The first rule is triggered when the ontology terminology layer changes and the metadata tag is associated with the changed concept; the operation updates the metadata tag to the new concept, with the synchronization direction being O→MD. The second rule is triggered when the ontology attribute value range expands; the operation updates the constraint description on the metadata side, with the synchronization direction being O→MD. The third rule is triggered when a new concept is added to the ontology; the operation scans all unannotated metadata and matches the corresponding annotations, with the synchronization direction being O→MD. The fourth rule is triggered when a concept is deleted from the ontology; the operation downgrades the metadata annotation associated with that concept to a pending confirmation state, with the synchronization direction being O→MD. There are three feedback rules from metadata to ontology: The first rule is triggered when the number of annotated fields for the same ontology concept exceeds a preset threshold N and the variance of the field values ​​is greater than a preset threshold. θ The recommended operation is to further subdivide the corresponding concepts of the ontology, with the synchronization direction being MD→O; the second rule is triggered when the frequency of mismatch between the annotation labels and the actual field values ​​exceeds a preset threshold. τ The recommended operation is to modify the corresponding constraint rules of the ontology, with the synchronization direction being MD→O; the third rule is triggered when high-frequency unlabeled fields have common characteristics, and the recommended operation is to add the corresponding concept in the ontology, with the synchronization direction being MD→O.

[0050] During the synchronization process, the scope of the change's impact is first defined. The following formula provides a formal definition of the scope of impact: ,in, This indicates a relationship, which is propagated through annotation mapping, i.e. k is the propagation depth threshold. To change the semantic mapping triples of the ontology, where This is an event involving changes to the entity itself. For the changed ontology concept, For associated metadata elements, For semantic similarity scores, ontology concept and The semantic distance (the shortest path length in the ontology hierarchy) is used to measure the degree of semantic association between two concepts. The following formula gives the definition of the synchronization propagation closure: Metadata update set Defined as a propagation closure, That is, all metadata updates reachable from the changed ontology through rules, where, For the synchronization rules, the specific logic for triggering metadata updates when the ontology changes are defined. For rules In the case of changes and metadata The application result, i.e., the metadata update operation after rule deduction, For the updated metadata element (m' is the result of the rule application) (updated version) These are ontology change events, including concept addition, deletion, modification, and attribute expansion. The synchronization logic from ontology to metadata is as follows: The feedback logic from metadata to ontology is as follows: ,in, For synchronization operation pairs, representing metadata Label changes (e.g., updated to be marked as a new concept). For feedback operation pairs, represent ontology concepts. And optimization suggestions from the metadata side (such as suggestions for concept subdivision). For metadata change mode, from metadata change Common feature patterns extracted from them The affected ontology concepts (ontology concepts that need to be evaluated due to feedback on metadata changes).

[0051] Taking the customer management ontology version upgrade scenario as an example, this change splits the original attribute hasMobilePhone into hasMobilePhone (domestic mobile phone number) and hasOverseasPhone (overseas mobile phone number), and refines the value range constraint of hasMobilePhone from xsd:string to pattern="1[3-9]\d{9}". After the event is triggered, the system first identifies the affected scope as all 156 fields labeled as hasMobilePhone, and then scans the actual value distribution of each field: the values ​​of 152 fields all conform to the domestic mobile phone number format, the label is automatically updated to the new hasMobilePhone, and the confidence level remains unchanged; the values ​​of 4 fields contain overseas formats, the label is downgraded to a pending confirmation status, and pushed to the manual end for differentiation; the constraint description on the metadata side is synchronously updated to "domestic mobile phone number, 11 digits, starting with 1". After the update, a consistency check is performed, and all 152 automatically updated fields pass the verification, and the synchronous consistency measure is The preset synchronization event types include five categories: The first category is EVT_ONTO_CHANGE, triggered by a change in the ontology TBox, with the synchronization direction being O→MD, and the operation being updating associated metadata annotations; the second category is EVT_ONTO_ADD, triggered by adding an ontology element, with the synchronization direction being O→MD, and the operation being triggering a re-annotation scan; the third category is EVT_META_TAGGED, triggered by the completion of metadata annotation, with the synchronization direction being MD→O, and the operation being recording annotation statistics; the fourth category is EVT_DATA_DRIFT, triggered by data distribution drift, with the synchronization direction being MD→O, and the operation being suggesting ontology constraint adjustments; the fifth category is EVT_INCONSISTENCY, triggered by a consistency detection alarm, with the synchronization direction being bidirectional, and the operation being initiating a conflict resolution process.

[0052] The system also supports ontology-based formal reasoning. Deductive reasoning is implemented using a table deduction algorithm, which includes three core rules: R1 represents a concept containing transitive rules: ; R2 represents the attribute domain / range derivation rule: ; R3 specifies the metadata type inheritance rules:

[0053] in, It serves as the source concept in the ontology and as the domain of the attribute relationship. The target concept in the ontology serves as the value domain for attribute relationships; It is an intermediate concept in the ontology, used to convey inclusion relationships; For attributes In the example and Specific relationship assertions between them (facts in the ABox); For attributes The domain of an attribute is the scope of the main concepts to which it applies. For attributes The range of values ​​for an attribute is the scope of the object concept from which that attribute can take its value. For example Belongs to the concept (Conceptual assertion); For example Belongs to the concept (Conceptual assertion); For metadata The annotation labels in the ontology (i.e.) The ontology concept or attribute to which it is mapped.

[0054] Inductive reasoning extracts new knowledge from metadata schemas: the concept discovery logic is as follows That is, when the number of unlabeled metadata with common feature patterns exceeds a preset threshold, it is recommended to add an ontology concept; the attribute constraint induction logic is as follows: That is, to summarize the attribute constraint range from the actual data value range; among which, For metadata The feature vector (including naming features, type features, value distribution features, etc.); A Common Feature Pattern (CFR) is a set of feature templates shared by metadata. This is an unlabeled state (metadata has not yet been mapped to any ontology concept); For metadata The actual set of values ​​(enumeration or distribution of field values); This refers to the ontological property, which is the target property of inductive reasoning.

[0055] To quantify metadata quality, a five-dimensional quality assessment system is constructed, with the comprehensive quality scoring logic as follows: The weights satisfy and The calculation logic for the five dimensions is as follows: completeness ,in, , This represents the integrity determination function for metadata m (returns true when all attributes attr are not empty). Metadata m A specific attribute (such as business meaning, data type, length constraint, etc.); Metadata mThe set of attributes (all metadata attribute items that should be filled in); This indicates a null value (meaning the attribute is missing). accuracy , where GroundTruth represents the standard truth value of domain expert annotations (a benchmark reference used to evaluate the accuracy of automatic annotation), which is obtained through domain expert annotations; consistency ,in, , For a set of consistency rules, Metadata (MD) and Ontology The set of inconsistencies (metadata that violates consistency rules). Representing consistency rules r metadata and ontology The judgment result (false if violated, true if satisfied), timeliness λ is the attenuation coefficient, and T is the timestamp. The timestamp of the last time the data was changed. The timestamp of the last update of the metadata; semantic richness Update the consistency metric to From an information theory perspective, the quality score Q(MD) can be interpreted as the mutual information of metadata MD with respect to ontology O, i.e. ,in, The information entropy of ontology O (a measure of the uncertainty of ontology concepts); Given the metadata MD, the conditional entropy of ontology O is (the remaining uncertainty of the ontology after the metadata is known). A high score indicates that the metadata effectively reduces uncertainty in ontology concepts. Using 5000 fields from the enterprise's data warehouse as the evaluation object, the weights for each dimension were set as follows: completeness 0.25, accuracy 0.30, consistency 0.25, timeliness 0.10, and semantic richness 0.10. The results showed: Completeness score was 0.82, weighted score 0.205, with the main issue being 18% of fields lacking business meaning descriptions; Accuracy score was 0.91, weighted score 0.273, with a labeling bias rate of 9%; Consistency score was 0.88, weighted score 0.220, with 12% of fields exhibiting ontology consistency violations; Timeliness score was 0.75, weighted score 0.075, with 25% of field metadata not updated for more than 3 months; Semantic richness score was 0.64, weighted score 0.064, covering only 42% of the ontology concepts. The overall quality score was 0.857, rated as good, with significant improvement needed in timeliness and semantic richness.

[0056] This implementation method continuously maintains the semantic consistency between metadata and ontology through a set of consistency rules. The consistency rules are defined as predicate logic formulas. It contains five core rules: CON-01 is the tag existence rule, tag(m)∈C∪P; CON-02 is the data type compatibility rule, CON-03 indicates that the range constraint satisfies the rule. CON-04 is a namespace consistency rule. CON-05 is a relation integrity rule. ,in, The current version of the main body The namespace is used to ensure that the ontology version referenced by the metadata tags is consistent; For metadata and Foreign key relationships between them; It is an ontology attribute relation (Relationship), belonging to a certain relation type in the ontology attribute set P; For the concept and The relationship between their intrinsic attributes For metadata Labels in the ontology.

[0057] Theorem 2 gives the complexity of consistency determination: For an ALC-described logical ontology, the determination... It is PSPACE-complete. The proof summary is as follows: For the upper bound, construct a nondeterministic Turing machine and use the Tableau algorithm to determine consistency in polynomial space; for the lower bound, reduce the ALC ontology consistency (known to be PSPACE-complete) to... Problem identification. The strategy for resolving inconsistency issues is divided into five levels: First, periodically scan the consistency status of metadata and ontology, applying the CON-01 to CON-05 rule sets to detect anomalies; second, classify anomalies into three levels based on their impact: Critical level affects data production use, Warning level may cause misunderstandings but does not block the process, and Info level provides suggested improvements that do not affect current use; then, send tiered notifications to the data owner and ontology administrator; next, perform remediation operations, automatically remediating the CON-04 namespace inconsistency scenario by directly updating references, and generating remediation suggestions for manual confirmation in other scenarios; finally, re-execute consistency checks after remediation to confirm that the problem is resolved. The formal description of the inconsistency problem handling process is as follows: ,in, This is a resolution function for inconsistency issues, based on the inconsistency terms. Type selection of resolution strategy; This is an automatic fix function, suitable for inconsistencies that can be automatically corrected; The suggestion generation function generates repair suggestions for inconsistencies that require manual confirmation. The human confirmation step involves the data owner or ontology administrator reviewing and confirming the system's recommendations.

[0058] The method architecture proposed in this embodiment is as follows: Figure 1 As shown, it comprises four core components: a semantic metadata intelligent extraction engine, an ontology-driven metadata automatic annotation module, a metadata-ontology linkage update module, and a metadata consistency management module. Each module interacts with the domain ontology knowledge base to achieve end-to-end metadata management. The association model between ontology and metadata is as follows: Figure 2 As shown, the terminology layer and assertion layer of the domain ontology correspond to the business semantic constraints and instance annotation results of the metadata, respectively, and the relationship between the two is established through semantic mapping. The semantic linkage and propagation mechanism is as follows: Figure 3 As shown, ontology change events and metadata change events are transmitted to the linkage propagation processing engine via the event bus. After the impact scope is calculated, synchronization operations are performed or ontology optimization suggestions are generated, ultimately outputting the updated metadata and feedback results. The metadata enhancement effect is as follows: Figure 4 As shown, four types of performance indicators are presented: automation rate improvement, five-dimensional quality score distribution, comparison of annotation accuracy for different field types, and linkage synchronization coverage.

[0059] Implementation Method 3: This implementation method proposes a computer device, including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes an ontology-enhanced metadata intelligent management method based on semantic linkage as described in any one of Implementation Methods 1 to 2.

[0060] Implementation Method 4: This implementation method proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a semantic linkage-based ontology-enhanced metadata intelligent management method as described in any one of Implementation Methods 1 to 2.

[0061] The specific embodiments of the present invention disclosed above are merely illustrative of the invention. These embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A semantically linked ontology-enhanced metadata intelligent management method, characterized in that, The method includes: Obtain the metadata to be managed, including technical metadata, business metadata, and management metadata; Construct a domain ontology, which is defined using description logic as a quintuple. ,in, For a set of concepts, For attribute set, For the axiom set, For instance set, For the range type set; Establish a semantic mapping from metadata to the domain ontology. The semantic mapping For a ternary relation ,in, For the ontology concept of mapping, For the metadata element to be mapped, For semantic similarity scores, For the set of ontology concepts, A collection of metadata elements Semantic similarity is calculated by comprehensively considering naming similarity, semantic similarity, structural compatibility, and value range matching. Based on the semantic mapping The semantic knowledge of the domain ontology is used to automatically annotate the metadata and generate annotation tags; In response to a change event in the domain ontology or a change event in the metadata, a bidirectional linkage update protocol is executed, based on the semantic mapping. The scope of influence changes, and the metadata annotation is updated synchronously or the domain ontology is adjusted accordingly. Based on a preset set of consistency rules, the consistency status between the updated metadata and the domain ontology is detected and updated.

2. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 1, characterized in that, The semantic mapping The steps for automatically annotating metadata using the semantic knowledge of the domain ontology specifically include: Extracting multidimensional feature vectors from metadata elements to be labeled The multidimensional feature vector includes naming similarity features, semantic similarity features, structural compatibility features, value range matching features, and ontology constraint features; Calculate the probability distribution of the metadata elements belonging to each candidate ontology tag. : in, For the domain ontology, For the feature weight vector, As candidate tags, A collection of ontology concept tags. For the feature function vector, Candidate ontology labels to be labeled Any element in the set of ontology concept tags; Select the label with the highest probability as the best label. : 。 3. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 2, characterized in that, The automatic annotation step also includes context-aware collaborative annotation: Construct a collaborative annotation objective function for a set of fields in the same data table. : in, The strength of the relationship between the tag and the ontology. For the first One ontology tag, For the first One ontology tag, Tag index; Labeled probabilities; The strength of the association between fields is determined based on foreign key dependencies or coexistence relationships within the same table. For the associated first Each metadata field For the associated first Each metadata field.

4. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 1, characterized in that, The steps for executing the bidirectional linkage update protocol specifically include: Define linkage rules ,in, Event types include ontology change events, metadata annotation completion events, data drift events, and consistency anomaly events; This is the trigger condition for the rule; A set of operations to be performed; For synchronization direction; When the trigger direction is from ontology to metadata, update the tag name and constraint description of the associated metadata according to the changed content, or downgrade the metadata label associated with the deleted concept to a pending confirmation status. When the trigger direction is from metadata to ontology, suggestions for ontology concept subdivision, constraint modification, or addition of concepts are generated based on the statistical regularity of metadata annotation, value range distribution drift, or common characteristics of unannotated fields.

5. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 4, characterized in that, In the step of synchronously updating metadata annotations, the scope of impact of the changes is defined: in, For subject to change The set of metadata affected Metadata m and ontology changes The relationships between metadata tags; these relationships are propagated through semantic mapping, where the distance between the concept corresponding to the metadata tag and the changed concept is considered. If so, it is determined to be affected metadata. k The preset propagation depth threshold, , These are concept nodes within the ontology.

6. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 1, characterized in that, The step of detecting and updating the consistency status between the metadata and the domain ontology specifically includes: The detection is performed using a formally defined set of predicate logic rules, which includes: Label Existence Rules : ; Data type compatibility rules : , The actual type of metadata m must be The members of the defined value range, or a subtype of a certain type in the value range; Value range constraint satisfaction rules : , The actual value Value(m) of the metadata m must belong to the value range defined by tag(m), or be a subtype instance of the type within that value range; Namespace Consistency Rules : , The namespace representing the metadata tag(m) Compared to the current version namespace ; Relationship integrity rules: , Representing concepts in an ontology and Attribute relationships between Metadata The annotation tags correspond to the ontology concept. Metadata The annotation tags correspond to the ontology concept. .

7. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 1, characterized in that, Following the acquisition of the metadata to be managed, the process also includes a business metadata derivation step: The field names in the technical metadata are segmented and mapped to the concept tags of the domain ontology; Based on the value distribution characteristics and pattern matching results of the field, the business type of the field is inferred; Based on the keyword pattern of field names and the uniqueness of value ranges, the sensitivity level in management metadata is derived.

8. The ontology-enhanced metadata intelligent management method based on semantic linkage according to claim 1, characterized in that, The method also includes a metadata quality measurement step: Construct a five-dimensional quality assessment system and calculate the comprehensive quality score: in, The weight coefficient for the k-th quality dimension; The quality score of metadata MD in the k-th dimension; k This is the preset propagation depth threshold.

9. A computer device, characterized in that: The system includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes an ontology-enhanced metadata intelligent management method based on semantic linkage as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the ontology-enhanced metadata intelligent management method based on semantic linkage as described in any one of claims 1-8.

Images