A metadata management method and system

Abstract

This application discloses a metadata management method and system. The management method generates a category table based on a set of metadata categories, and generates a correlation table based on the relationships between the categories in the category table. The database retrieves related metadata simultaneously for retrieval based on the correlation table. The management system sends instructions to the database via a user terminal to retrieve metadata and data ontology. The database then sends related metadata and corresponding data ontology for staff to retrieve, greatly optimizing the data management system's data analysis capabilities.

Description

Technical Field

[0001] This invention relates to the field of metadata application technology, and in particular to a metadata management method and system. Background Technology

[0002] Metadata, also known as intermediary data or relay data, is data about data. It primarily describes data properties and supports functions such as indicating storage location, historical data, resource lookup, and file records. Metadata can be considered a type of electronic catalog. To achieve the purpose of cataloging, it is necessary to describe and collect the content or characteristics of the data, thereby assisting in data retrieval.

[0003] Although many enterprises already have metadata management systems, most of these metadata management systems can only interpret and organize metadata. For metadata corresponding to different data, the above-mentioned metadata management systems cannot effectively associate metadata, and therefore cannot cooperate with data management systems to achieve correlation analysis of different data. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for correlation analysis of metadata.

[0005] Therefore, this application discloses a metadata management method, which uses a database, and the method includes:

[0006] The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables.

[0007] If the database receives an instruction to retrieve metadata, it determines the category table to which the metadata belongs based on the category of the metadata, and simultaneously retrieves several other category tables associated with the category table for retrieval based on the association table.

[0008] In some embodiments of this application, in order to enable the database to generate a relational table, a method for presetting a relational index is disclosed. The method for presetting the relational index includes:

[0009] The correlation index between the category tables is determined based on the collaborative relationship between the metadata categories corresponding to the category tables.

[0010] In some embodiments of this application, a method is provided for determining the collaborative relationship between the category tables, wherein the collaborative relationship is the relationship between metadata within the same event;

[0011] In the same event relationship, if a certain data ontology is indispensable, the correlation index of the metadata corresponding to that data ontology is considered to be high. For events, the work experience of the staff is used to determine whether a certain data ontology is indispensable.

[0012] In some embodiments of this application, the contents of the category table are provided, and the categories of the category table include:

[0013] Storage category, including storage location, storage time, space occupied, and storage type;

[0014] Content categories, including keywords, titles, and content summaries.

[0015] In some embodiments of this application, in order to retrieve the corresponding data ontology based on the correlation of metadata for staff to consult and search, the method for retrieving the data ontology corresponding to the metadata from the database includes:

[0016] Each metadata in the category table is associated with a unique identifier, and the corresponding data ontology is also associated with the same identifier.

[0017] The database obtains the identification code and generates an identification code table. Each identification code in the identification code table is marked with the corresponding storage location of the data body.

[0018] After receiving the instruction to retrieve the data ontology corresponding to the metadata, the database first obtains the identification code of the metadata in the category table, and then retrieves the data ontology according to the storage location indicated by the identification code in the identification code table.

[0019] This application discloses a metadata management method that generates a category table based on a set of metadata categories, and generates an association table based on the relationships between the categories in the category table. The database then uses the association table to simultaneously retrieve related metadata for retrieval.

[0020] A metadata management system, the system comprising:

[0021] Databases are used to store metadata and data ontology;

[0022] The user terminal is used to retrieve the metadata and the data ontology;

[0023] The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables.

[0024] If the database receives an instruction from the user terminal to retrieve metadata, it determines the category table to which the metadata belongs based on the category, and retrieves several other category tables associated with the category table based on the association table and sends them to the user terminal for staff to retrieve.

[0025] In some embodiments of this application, a method for sending correlation indices to the database is provided, and the system further includes a control terminal;

[0026] The control terminal is used to send the correlation index between the category tables to the database, so that the database can determine the correlation relationship between the category tables based on the correlation index and generate the correlation table.

[0027] In some embodiments of this application, in order to enable the database to retrieve the corresponding data ontology based on metadata, the content of the category table has been enriched, and the category table further includes:

[0028] The identification code corresponds one-to-one with the metadata. The database obtains the identification code and generates an identification code table. Each identification code in the identification code table is associated with a storage location of the data ontology corresponding to the metadata.

[0029] In some embodiments of this application, a method for retrieving a data ontology using an identification code is disclosed. The method for retrieving the data ontology corresponding to the metadata includes:

[0030] If the user terminal sends an instruction to the database to retrieve the data ontology corresponding to the metadata, the database determines the identification code corresponding to the metadata according to the category table, and determines the storage location of the data ontology corresponding to the metadata according to the identification code table, so as to retrieve the data ontology and send it to the user terminal.

[0031] This application also discloses a metadata management system, in which the user terminal sends an instruction to the database to retrieve metadata and data ontology, and the database sends interrelated metadata and corresponding data ontology for staff to retrieve.

[0032] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the method used to construct a metadata management system in the implementation of this application. Detailed Implementation

[0034] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0036] Example:

[0037] Metadata, also known as intermediary data or relay data, is data that describes data, mainly information describing data properties.

[0038] To facilitate the explanation of the technical solutions of this application, the types of metadata are now disclosed. Metadata includes physical metadata, data source metadata, storage metadata, computing metadata, quality metadata, cost metadata, standard metadata, security metadata, shared metadata, model metadata, application metadata, analysis metadata, and operation and maintenance metadata.

[0039] in:

[0040] Physical metadata describes the metadata of physical resources, such as the location of servers, computer rooms, and other information.

[0041] Data source metadata describes the metadata of the data source and typically includes four types of information: data source address (e.g., IP, PORT), physical topology (e.g., master / slave, roles), permissions (e.g., username, password), and database name, version, domain name, etc.

[0042] The metadata describing the object storage typically refers to metadata in a "narrow" sense, including several main types: management attributes (e.g., creator, application system, business department, business owner, etc.), lifecycle (e.g., creation time, DDL time, version information, etc.), storage attributes (e.g., location, physical size, etc.), data characteristics (e.g., data skew, average length, etc.), usage characteristics (e.g., DML, refresh rate, etc.), data structures (tables / partitions, e.g., name, type, comments, etc.), columns (e.g., name, type, length, precision, etc.), indexes (e.g., name, type, field, etc.), and constraints (e.g., type, field, etc.).

[0043] Metadata describing the data computation process can generally be divided into two types of computation: data extraction (ETL) or data processing (JOB). Each type of computation can be further subdivided into control metadata (e.g., configuration properties, scheduling policies, etc.) and process metadata (e.g., dependencies, execution status, execution logs, etc.).

[0044] Quality metadata is metadata that describes the quality of data. Typically, data quality is reflected by defining a series of quality metrics.

[0045] Cost metadata describes the metadata related to data storage and computation costs. This includes computation costs (e.g., CPU, MEM, etc.) and storage costs (e.g., space, compression ratio, etc.).

[0046] Standard metadata describes the metadata of standardized data content. Code management (e.g., transformation rules, external interfaces, etc.) and mapping management (e.g., styles, rules, semantics, units, etc.) are used to manage and display data.

[0047] Security metadata describes the metadata related to the security content of the data. Security level: Data sensitivity (e.g., whether it is sensitive, de-identification algorithms, etc.).

[0048] Shared metadata describes how data is shared, such as interface methods, format, and content.

[0049] Data modeling, or metadata modeling, describes the business and allows for a better understanding of it. Common modeling methods include paradigmatic models and dimensional models. Examples include business lines, sectors, process data domains, subject domains, dimensions, attributes, facts, metrics, markets, and applications.

[0050] Application metadata refers to metadata that describes the type of data application. For example, application scenarios, etc.

[0051] Metadata analysis refers to describing business metadata from a data analysis perspective. Examples include data domains, subject areas, product lines, modules, business processes, business rules, etc.

[0052] Operational metadata includes data processing logs, operational status data, scheduling frequency, access records, etc.

[0053] In data management systems, metadata is needed to retrieve the desired data ontology during the data retrieval process. In order to retrieve related data ontology based on the retrieval intent, the metadata management method needs to be improved.

[0054] Therefore, this application discloses a metadata management method, which uses a database, and the method includes:

[0055] The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables.

[0056] If the database receives an instruction to retrieve metadata, it determines the category table to which the metadata belongs based on the category of the metadata, and simultaneously retrieves several other category tables associated with the category table for retrieval based on the association table.

[0057] In some embodiments of this application, in order to enable the database to generate a relational table, a method for presetting a relational index is disclosed. The method for presetting the relational index includes:

[0058] The correlation index between the category tables is determined based on the collaborative relationship between the metadata categories corresponding to the category tables.

[0059] In some embodiments of this application, a method is provided for determining the collaborative relationship between the category tables, wherein the collaborative relationship is the relationship between metadata within the same event.

[0060] In the same event relationship, if a certain data ontology is indispensable, the correlation index of the metadata corresponding to that data ontology is considered to be high. For events, the work experience of the staff is used to determine whether a certain data ontology is indispensable.

[0061] In some embodiments of this application, the contents of the category table are provided, and the categories of the category table include:

[0062] Storage category, including storage location, storage time, space occupied, and storage type.

[0063] Content categories, including keywords, titles, and content summaries.

[0064] In some embodiments of this application, in order to retrieve the corresponding data ontology based on the correlation of metadata for staff to consult and search, the method for retrieving the data ontology corresponding to the metadata from the database includes:

[0065] Each metadata in the category table is accompanied by a unique identifier, and the corresponding data ontology is also accompanied by the same identifier.

[0066] The database obtains the identification code and generates an identification code table, where each identification code in the identification code table is marked with the corresponding storage location of the data body.

[0067] After receiving the instruction to retrieve the data ontology corresponding to the metadata, the database first obtains the identification code of the metadata in the category table, and then retrieves the data ontology according to the storage location indicated by the identification code in the identification code table.

[0068] This application discloses a metadata management method that generates a category table based on a set of metadata categories, and generates an association table based on the relationships between the categories in the category table. The database then uses the association table to simultaneously retrieve related metadata for retrieval.

[0069] A metadata management system, the system comprising a database and a client.

[0070] in:

[0071] The database is used to store metadata and data ontology.

[0072] The client is used to retrieve the metadata and the data ontology.

[0073] The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables.

[0074] If the database receives an instruction from the user terminal to retrieve metadata, it determines the category table to which the metadata belongs based on the category, and retrieves several other category tables associated with the category table based on the association table and sends them to the user terminal for staff to retrieve.

[0075] In some embodiments of this application, a method for sending correlation indices to the database is provided, and the system further includes a control terminal.

[0076] The control terminal is used to send the correlation index between the category tables to the database, so that the database can determine the correlation relationship between the category tables based on the correlation index and generate the correlation table.

[0077] In some embodiments of this application, in order to enable the database to retrieve the corresponding data ontology based on metadata, the content of the category table has been enriched, and the category table further includes:

[0078] The identification code corresponds one-to-one with the metadata. The database obtains the identification code and generates an identification code table. Each identification code in the identification code table is associated with a storage location of the data ontology corresponding to the metadata.

[0079] In some embodiments of this application, a method for retrieving a data ontology using an identification code is disclosed. The method for retrieving the data ontology corresponding to the metadata includes:

[0080] If the user terminal sends an instruction to the database to retrieve the data ontology corresponding to the metadata, the database determines the identification code corresponding to the metadata according to the category table, and determines the storage location of the data ontology corresponding to the metadata according to the identification code table, so as to retrieve the data ontology and send it to the user terminal.

[0081] This application also discloses a metadata management system, in which the user terminal sends an instruction to the database to retrieve metadata and data ontology, and the database sends interrelated metadata and corresponding data ontology for staff to retrieve.

[0082] To further illustrate the technical solution of this application, a specific application event using the method and system of this application is hereby disclosed.

[0083] We need to build a metadata management system that can retrieve multiple related metadata entries simultaneously based on their relationships and display them to the data management system or staff, facilitating data analysis.

[0084] To build this metadata management system, the following construction method and process are disclosed:

[0085] S1: The database generates category tables based on the categories of metadata.

[0086] S2: The database generates an association table based on a preset association index between the category tables;

[0087] S3: If the database receives an instruction to call metadata, it determines the category table to which the metadata belongs based on the category of the metadata.

[0088] S4: The database simultaneously retrieves several other category tables associated with the category table for retrieval based on the association table.

[0089] Based on the above construction method, a metadata management system is constructed, which includes a database, a user terminal, and a control terminal.

[0090] The database is used to store metadata and data ontology; the user terminal is used to retrieve metadata and data ontology; the control terminal is used to send the correlation index between the category tables to the database, and the correlation index is determined based on the collaborative relationship between the metadata categories corresponding to the category tables.

[0091] The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables (the association index is sent by the control terminal and is determined based on the system relationship between the metadata). The association tables are used to record the association relationships between the category tables.

[0092] If the database receives an instruction from the user terminal to retrieve metadata, it determines the category table to which the metadata belongs based on the category, and retrieves several other category tables associated with the category table based on the association table and sends them to the user terminal for staff to retrieve.

[0093] The category table also includes identification codes, each corresponding one-to-one with metadata. The database obtains these identification codes and generates an identification code table. Each identification code in the identification code table is associated with a storage location for the corresponding data ontology. If the user client sends an instruction to the database to retrieve the data ontology corresponding to the metadata, the database determines the identification code corresponding to the metadata based on the category table and the storage location of the corresponding data ontology based on the identification code table, then retrieves the data ontology and sends it to the user client.

[0094] This application discloses a metadata management method and system. The management method generates a category table based on a set of metadata categories, and generates a correlation table based on the relationships between the categories in the category table. The database retrieves related metadata simultaneously for retrieval based on the correlation table. The management system, guided by the management method, sends instructions to the database via the user terminal to retrieve metadata and data ontology. The database then sends related metadata and corresponding data ontology for staff to retrieve, greatly optimizing the data management system's data analysis capabilities.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A metadata management method, characterized in that, Applied to a database, the method includes: The correlation index between the category tables is determined based on the collaborative relationship between the metadata categories corresponding to the category tables; The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables. If the database receives an instruction to call metadata, it determines the category table to which the metadata belongs based on the category of the metadata, and simultaneously retrieves several other category tables associated with the category table for retrieval based on the association table. The metadata in the category table is equipped with a unique identification code, and the data ontology corresponding to the metadata is also equipped with the same identification code. The database obtains the identification code and generates an identification code table. Each identification code in the identification code table is marked with the corresponding storage location of the data body. After receiving the instruction to retrieve the data ontology corresponding to the metadata, the database first obtains the identification code of the metadata in the category table, and then retrieves the data ontology according to the storage location indicated by the identification code in the identification code table.

2. The metadata management method according to claim 1, characterized in that, The collaborative relationship refers to the relationship between metadata within the same event.

3. The metadata management method according to claim 1, characterized in that, The categories in the category table include: Storage category, including storage location, storage time, space occupied, and storage type; Content categories, including keywords, titles, and content summaries.

4. A metadata management system, characterized in that, The system includes: Databases are used to store metadata and data ontology; The user-side is used to retrieve metadata and data ontology; The control terminal is used to send the correlation index between category tables to the database, so that the database can determine the correlation relationship between the category tables based on the correlation index and generate a correlation table. The database generates category tables based on the categories of metadata, and generates association tables based on a preset association index between the category tables. The association tables are used to record the association relationships between the category tables. If the database receives an instruction from the user terminal to retrieve metadata, it determines the category table to which the metadata belongs based on the category, and retrieves several other category tables associated with the category table based on the association table and sends them to the user terminal for staff to retrieve. The category table includes an identification code, which corresponds one-to-one with metadata. The database obtains the identification code and generates an identification code table. Each identification code in the identification code table is associated with a storage location of the data ontology corresponding to the metadata. If the user terminal sends an instruction to the database to retrieve the data ontology corresponding to the metadata, the database determines the identification code corresponding to the metadata according to the category table, and determines the storage location of the data ontology corresponding to the metadata according to the identification code table, so as to retrieve the data ontology and send it to the user terminal.

Images