A method and system for metadata maintenance and management

Abstract

This invention provides a metadata maintenance and management method and system, relating to the field of data management technology. The method includes: extracting target metadata from a data source based on metadata model-defined collection rules, and integrating the extracted target metadata to obtain a first metadata set; indexing and storing the metadata based on preset indexing technology, and determining data retrieval keywords for the target metadata based on real-time user needs; determining metadata retrieval and analysis tools based on real-time user needs, thereby performing data analysis on the integrated metadata in the metadata center; performing quality checks on the metadata, and determining metadata optimization schemes based on the quality check results, thereby formulating improvement plans and achieving real-time improvement of the metadata management method. By integrating, indexing, storing, and analyzing metadata, the efficiency and quality of metadata management can be improved, the efficiency of metadata updates and the value of metadata can be increased, and the accuracy and security of data can be ensured.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a method and system for metadata maintenance and management. Background Technology

[0002] Currently, metadata management refers to the process of effectively managing, maintaining, and utilizing metadata. Metadata management plays a crucial role in data governance and data lifecycle management.

[0003] However, existing metadata management methods have some problems, such as incomplete data lineage tracing, inadequate data quality monitoring, and untimely metadata updates. These problems affect the effective use of data and the accuracy of enterprise decision-making.

[0004] Therefore, the present invention provides a method and system for metadata maintenance and management. Summary of the Invention

[0005] This invention provides a metadata maintenance and management method and system to solve problems such as untimely and incomplete data tracking, inadequate data quality monitoring, and untimely and inaccurate updates in the prior art.

[0006] This invention provides a metadata maintenance and management method, comprising:

[0007] Step 1: Based on the metadata model, define the collection rules to extract target metadata from the data source, and integrate the extracted target metadata to obtain the first metadata set;

[0008] Step 2: Store metadata indexes based on preset indexing technology, and determine data retrieval keywords for target metadata based on real-time user needs;

[0009] Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then conduct data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool;

[0010] Step 4: When conducting data analysis, it is also necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

[0011] According to the metadata model defined by the present invention, target metadata is extracted from the data source by defining collection rules, and the extracted target metadata is integrated to obtain a first metadata set, including:

[0012] Step 11: Determine the data collection rules for extracting target metadata from the target data source based on the target data source metadata model;

[0013] Step 12: Based on the data collection rules, extract the target metadata from the target data source in real time to obtain the initial metadata;

[0014] Step 13: Used to clean, transform, and associate the extracted initial metadata to obtain the first metadata set.

[0015] The present invention provides a method for storing metadata indexes based on preset indexing technology and determining data retrieval keywords for target metadata based on real-time user needs, including:

[0016] Step 21: Determine the indexing technique for indexing the metadata of the target data source based on the data type of the target data source, and obtain the first indexing technique;

[0017] Step 22: Store the first metadata set according to the first indexing technique to obtain the first storage result;

[0018] Step 23: Obtain the real-time user needs of the target users, and perform demand breakdown and demand analysis to determine the demand fields of the target users, and determine the data retrieval keywords of the target metadata based on the demand fields.

[0019] According to the present invention, the method for acquiring real-time user needs of target users, performing need decomposition and need analysis to determine the need fields of target users, and determining data retrieval keywords for target metadata based on the need fields includes:

[0020] Step 231: Obtain the target user's real-time user needs, and at the same time obtain the target user's historical user needs and historical behavior data;

[0021] Step 232: Based on the historical behavior data of the target users, the corresponding historical user needs are broken down to obtain the first set of historical needs;

[0022] Among them, each subset of the first historical demand set corresponds to the same historical behavior data;

[0023] Step 233: Based on the demand splitting schemes of each subset of the first historical demand set, synthesize them to obtain the reference demand splitting scheme for the target user;

[0024] Step 234: Used to obtain the urgency level of each sub-requirement in the real-time user requirements, and sort each sub-requirement of the real-time user requirements based on the urgency level to obtain the first ordered requirements;

[0025] Step 235: Optimize the reference requirement splitting scheme based on the first ordered requirement to obtain the first requirement splitting scheme set;

[0026] Step 236: Based on the first set of demand decomposition schemes, the real-time user demand is decomposed, and based on the demand decomposition results, each sub-demand is analyzed to obtain the first demand analysis results of the target user.

[0027] Step 237: Based on the results of the first requirement analysis, determine the data fields of each sub-requirement to obtain the first requirement fields. At the same time, obtain the field parameters of each metadata field in the metadata model of the target data source.

[0028] Step 238: Match the first requirement field with the field parameters of each metadata field in the metadata model to obtain keywords that meet the user needs of the target user, and evaluate the importance of each keyword to obtain the first keyword that meets the user needs.

[0029] Step 239: Combine each first keyword that meets the user's needs to obtain the first keyword set of the target metadata, and optimize the first keywords in the first keyword set to obtain the data retrieval keywords of the target metadata.

[0030] The field parameters of the metadata field provided by the present invention include: field meaning, data type, and value range.

[0031] According to the metadata retrieval and analysis tool provided by the present invention, which determines metadata based on real-time user needs, data analysis is performed on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool, including:

[0032] Step 31: Determine the target metadata retrieval and analysis tools based on the user needs of the target users and the data characteristics of the target data source;

[0033] Step 32: Connect the first metadata set to the metadata center to obtain the initial integrated metadata;

[0034] Step 33: Based on the data themes and meanings in the metadata center, integrate and optimize the initial integrated metadata to obtain integrated metadata;

[0035] Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval and analysis tools to obtain the first data analysis result.

[0036] According to the present invention, when performing data analysis, it is also necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement schemes and realizing real-time improvement of metadata management methods, including:

[0037] Step 41: Perform a quality check on the integrated metadata to obtain the first quality check result;

[0038] Step 42: Based on the results of the first quality inspection, determine the data problems in the integrated metadata, and formulate corresponding metadata optimization solutions based on the data problems;

[0039] Step 43: Transform the metadata optimization scheme into a data improvement scheme for the target metadata, and improve the target metadata one by one according to the data improvement scheme, and monitor the improvement effect in real time.

[0040] Step 44: Based on the improvement effect of real-time monitoring, adjust the metadata management method of the corresponding target metadata in real time, thereby improving the data quality and data value of the metadata.

[0041] This invention provides a metadata maintenance and management system, comprising:

[0042] The data acquisition and integration module is used to extract target metadata from the data source based on the data acquisition rules defined by the metadata model, and to integrate the extracted target metadata to obtain the first metadata set.

[0043] Intelligent retrieval module: used to index and store metadata based on preset indexing technology, and to determine data retrieval keywords for target metadata based on real-time user needs;

[0044] Application Analysis Module: Used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools;

[0045] Inspection and Improvement Module: When conducting data analysis, it is necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows: The metadata maintenance and management method and system provided by the present invention can improve the efficiency and quality of metadata management, improve the efficiency and value of metadata updates, and ensure the accuracy and security of data by integrating, indexing and storing, and analyzing metadata. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a metadata maintenance and management method provided in an embodiment of the present invention;

[0049] Figure 2 This is a structural diagram of a metadata maintenance and management system provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] Example 1:

[0052] This invention provides a metadata maintenance and management method, such as... Figure 1 As shown, it includes:

[0053] Step 1: Based on the metadata model, define the collection rules to extract target metadata from the data source, and integrate the extracted target metadata to obtain the first metadata set;

[0054] Step 2: Store metadata indexes based on preset indexing technology, and determine data retrieval keywords for target metadata based on real-time user needs;

[0055] Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then conduct data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool;

[0056] Step 4: When conducting data analysis, it is also necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

[0057] In this embodiment, the metadata model is the definition and description of the structure, attributes, and relationships of metadata. It defines how metadata is organized, stored, and managed so that it can be retrieved, analyzed, and used effectively.

[0058] In this embodiment, the collection rules refer to the guidelines or standards followed when extracting metadata from the data source. The collection rules define which metadata should be extracted, how to extract it, and how to process the extracted data.

[0059] In this embodiment, metadata is data used to describe data, including information such as data attributes, structure, source, and relationships.

[0060] In this embodiment, metadata resources refer to information related to data resources, such as the data's structure, content, source, format, quality, and access permissions.

[0061] In this embodiment, data integration is the process of merging data from different data sources into a consistent and complete dataset, including data cleaning, transformation, and association. In metadata management, data integration typically involves merging multiple metadata sets into a more comprehensive metadata set.

[0062] In this embodiment, the first metadata set is the metadata set obtained after data cleaning, data transformation, and data integration of the collected real-time metadata.

[0063] In this embodiment, the preset indexing technology is a technique for quickly retrieving and querying large amounts of data. For example, preset indexing technologies include inverted indexes, B-tree indexes, etc.

[0064] In this embodiment, metadata index storage is the process of storing metadata and its index information in a specific location or database, which helps to quickly locate and retrieve metadata.

[0065] In this embodiment, real-time user demand refers to a user's current need or query requirements for metadata. Understanding and meeting real-time user demand is crucial for providing high-quality services in metadata management.

[0066] In this embodiment, the data retrieval keyword refers to the keyword corresponding to the data retrieval of metadata. For example, if the metadata is abcdefg, where 'a' is the data retrieval keyword of the metadata, then entering the data 'a' in the metadata center will yield the corresponding abcdefg.

[0067] In this embodiment, the metadata center is a system or platform that centrally stores and manages metadata. It typically provides functions such as metadata querying, analysis, quality checking, and optimization.

[0068] In this embodiment, the metadata retrieval and analysis tool is a software tool used to retrieve, analyze, and visualize metadata.

[0069] In this embodiment, data analysis includes processes such as statistical analysis, data mining, and graphical display, which can intuitively and accurately display the relationships and characteristics between metadata.

[0070] In this embodiment, quality checking is the process of evaluating the accuracy, completeness, consistency, and other quality attributes of metadata. This helps ensure the quality and reliability of metadata.

[0071] In this embodiment, the metadata optimization scheme refers to a series of improvement measures or suggestions proposed based on the metadata quality inspection results.

[0072] The beneficial effects of the above technical solution are: by integrating, indexing, storing and analyzing metadata, the efficiency and quality of metadata management can be improved, the efficiency of metadata updates and the value of metadata can be increased, and the accuracy and security of data can be ensured.

[0073] Example 2:

[0074] Based on Example 1, target metadata is extracted from the data source according to the collection rules defined by the metadata model, and the extracted target metadata is integrated to obtain a first metadata set, including:

[0075] Step 11: Determine the data collection rules for extracting target metadata from the target data source based on the target data source metadata model;

[0076] Step 12: Based on the data collection rules, extract the target metadata from the target data source in real time to obtain the initial metadata;

[0077] Step 13: Used to clean, transform, and associate the extracted initial metadata to obtain the first metadata set.

[0078] In this embodiment, the metadata model is the definition and description of the structure, attributes, and relationships of metadata. It defines how metadata is organized, stored, and managed so that it can be retrieved, analyzed, and used effectively.

[0079] In this embodiment, data collection rules refer to the guidelines or standards followed when extracting metadata from a data source. The collection rules define which metadata should be extracted, how to extract it, and how to process the extracted data.

[0080] In this embodiment, metadata is data used to describe data, including information such as data attributes, structure, source, and relationships.

[0081] In this embodiment, the initial metadata refers to the target metadata extracted in real time from the target data source.

[0082] In this embodiment, the first metadata set is the metadata set obtained after data cleaning, data transformation, and data association of the collected initial metadata.

[0083] The beneficial effects of the above technical solution are: by determining the data collection rules, initial metadata is extracted, and data is cleaned, transformed, and associated to obtain the first metadata set, thereby realizing data integration of metadata, enabling more accurate indexing, storage, and data analysis, and ensuring data accuracy.

[0084] Example 3:

[0085] Based on Example 2, metadata is indexed and stored using a preset indexing technique, and data retrieval keywords for the target metadata are determined based on real-time user needs, including:

[0086] Step 21: Determine the indexing technique for indexing the metadata of the target data source based on the data type of the target data source, and obtain the first indexing technique;

[0087] Step 22: Store the first metadata set according to the first indexing technique to obtain the first storage result;

[0088] Step 23: Obtain the real-time user needs of the target users, and perform demand breakdown and demand analysis to determine the demand fields of the target users, and determine the data retrieval keywords of the target metadata based on the demand fields.

[0089] In this embodiment, different data types exist depending on the classification scheme. For example, according to different application areas, functions, and descriptive objects, the data types of the target data source include business metadata, technical metadata, and operational metadata.

[0090] In this embodiment, the first indexing technology refers to the data indexing technology that extracts data that matches the metadata of the target data source based on the data type of the target data source. For example, the first indexing technology includes inverted index, B-tree index, etc.

[0091] In this embodiment, storing according to the first indexing technology is the process of storing metadata and its index information in a specific location or database, which helps to quickly locate and retrieve metadata.

[0092] In this embodiment, the first storage result refers to the storage result obtained after storing the first metadata set according to the first indexing technology.

[0093] In this embodiment, real-time user demand refers to a user's current need or query requirements for metadata. Understanding and meeting real-time user demand is crucial for providing high-quality services in metadata management.

[0094] In this embodiment, demand splitting refers to splitting real-time user demands according to different demand types.

[0095] In this embodiment, requirement analysis refers to performing requirement analysis on each sub-requirement based on the requirement decomposition results, thereby determining the requirement fields corresponding to each sub-requirement.

[0096] In this embodiment, the data retrieval keyword refers to the keyword corresponding to the data retrieval of metadata. For example, if the metadata is abcdefg, where 'a' is the data retrieval keyword of the metadata, then entering the data 'a' in the metadata center will yield the corresponding abcdefg.

[0097] The beneficial effects of the above technical solution are: by determining the indexing technology of metadata, the metadata can be indexed and stored, thereby enabling data analysis of the stored results, improving the efficiency and quality of metadata management, increasing the value of metadata, and ensuring the accuracy of data.

[0098] Example 4:

[0099] Based on Example 3, the data retrieval keywords for the target metadata are determined, including:

[0100] Step 231: Obtain the target user's real-time user needs, and at the same time obtain the target user's historical user needs and historical behavior data;

[0101] Step 232: Based on the historical behavior data of the target users, the corresponding historical user needs are broken down to obtain the first set of historical needs;

[0102] Among them, each subset of the first historical demand set corresponds to the same historical behavior data;

[0103] Step 233: Based on the demand splitting schemes of each subset of the first historical demand set, synthesize them to obtain the reference demand splitting scheme for the target user;

[0104] Step 234: Used to obtain the urgency level of each sub-requirement in the real-time user requirements, and sort each sub-requirement of the real-time user requirements based on the urgency level to obtain the first ordered requirements;

[0105] Step 235: Optimize the reference requirement splitting scheme based on the first ordered requirement to obtain the first requirement splitting scheme set;

[0106] Step 236: Based on the first set of demand decomposition schemes, the real-time user demand is decomposed, and based on the demand decomposition results, each sub-demand is analyzed to obtain the first demand analysis results of the target user.

[0107] Step 237: Based on the results of the first requirement analysis, determine the data fields of each sub-requirement to obtain the first requirement fields. At the same time, obtain the field parameters of each metadata field in the metadata model of the target data source.

[0108] Step 238: Match the first requirement field with the field parameters of each metadata field in the metadata model to obtain keywords that meet the user needs of the target user, and evaluate the importance of each keyword to obtain the first keyword that meets the user needs.

[0109] Step 239: Combine each first keyword that meets the user's needs to obtain the first keyword set of the target metadata, and optimize the first keywords in the first keyword set to obtain the data retrieval keywords of the target metadata.

[0110] In this embodiment, real-time user demand refers to a user's current need or query requirements for metadata. Understanding and meeting real-time user demand is crucial for providing high-quality services in metadata management.

[0111] In this embodiment, historical behavior data refers to the target user's past behavior records, such as browsing history, purchase history, click history, etc., which can reflect the user's preferences and habits.

[0112] In this embodiment, historical user needs refer to the needs that the target user has raised in the past and that have been recorded.

[0113] In this embodiment, requirement decomposition refers to breaking down complex or comprehensive user requirements into multiple smaller requirements that are more specific, easier to implement, or easier to analyze.

[0114] In this embodiment, the first historical demand set is a set obtained by splitting the historical user demand of the target user, wherein each first historical demand subset corresponds to the same historical behavior data.

[0115] In this embodiment, the reference demand splitting scheme is a comprehensive solution based on the demand splitting schemes of each first historical demand subset in the first historical demand set, and is used to guide subsequent demand splitting and analysis.

[0116] In this embodiment, the urgency of the demand refers to the priority of each sub-demand in the real-time user demand.

[0117] In this embodiment, the first ordered demand is a demand sequence obtained by sorting each sub-demand of the real-time user demand according to the urgency of the demand.

[0118] In this embodiment, the first demand splitting scheme set is a set of multiple demand splitting schemes obtained by optimizing the reference demand splitting scheme based on the first ordered demand.

[0119] In this embodiment, the demand splitting result is obtained by splitting real-time user demands according to the first demand splitting scheme set.

[0120] In this embodiment, the sub-requirement is each sub-requirement in the result of the requirement splitting.

[0121] In this embodiment, requirements analysis refers to conducting in-depth research on the sub-requirements and clarifying their specific requirements, objectives, constraints, etc.

[0122] In this embodiment, the first requirement analysis result is the result obtained after performing requirement analysis on the sub-requirements.

[0123] In this embodiment, the requirement field is a data field used to describe or represent the sub-requirements to be split.

[0124] In this embodiment, the first requirement field is a data field for each sub-requirement determined based on the first requirement analysis results.

[0125] In this embodiment, the metadata model is a model used to describe the data structure, including data fields, field types, field relationships, etc.

[0126] In this embodiment, the metadata field refers to the data field in the metadata model.

[0127] In this embodiment, the field parameters are information used to describe the attributes or characteristics of the metadata field, including: field meaning, data type, and value range.

[0128] In this embodiment, the first keyword is a keyword that meets the needs of the target user.

[0129] In this embodiment, the first keyword set refers to the keyword set that includes all first keywords.

[0130] In this embodiment, the data retrieval keyword refers to the keyword corresponding to the data retrieval of metadata. For example, if the metadata is abcdefg, where 'a' is the data retrieval keyword of the metadata, then entering the data 'a' in the metadata center will yield the corresponding abcdefg.

[0131] The beneficial effects of the above technical solution are: by indexing and storing metadata, data analysis can be performed on the stored results, thereby improving the efficiency and quality of metadata management, increasing the value of metadata, and ensuring the accuracy of data.

[0132] Example 5:

[0133] Based on Example 4, the field parameters of the metadata field include: field meaning, data type, and value range.

[0134] The beneficial effects of the above technical solution are: by indexing and storing metadata, data analysis can be performed on the stored results, thereby improving the efficiency and quality of metadata management, increasing the value of metadata, and ensuring the accuracy of data.

[0135] Example 6:

[0136] Based on Example 3, data analysis is performed on the integrated metadata of the metadata center, including:

[0137] Step 31: Determine the target metadata retrieval and analysis tools based on the user needs of the target users and the data characteristics of the target data source;

[0138] Step 32: Connect the first metadata set to the metadata center to obtain the initial integrated metadata;

[0139] Step 33: Based on the data themes and meanings in the metadata center, integrate and optimize the initial integrated metadata to obtain integrated metadata;

[0140] Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval and analysis tools to obtain the first data analysis result.

[0141] In this embodiment, the metadata center is a system or platform that centrally stores and manages metadata. It typically provides functions such as metadata querying, analysis, quality checking, and optimization.

[0142] In this embodiment, the metadata retrieval and analysis tool is a software tool used to retrieve, analyze, and visualize metadata.

[0143] In this embodiment, the initial integrated metadata refers to the integrated data obtained after the first data set is connected to the metadata center and processed by data normalization and data reorganization.

[0144] In this embodiment, integrated metadata refers to the integrated metadata obtained by integrating and optimizing the initial integrated metadata based on the data themes and meanings of the metadata center.

[0145] In this embodiment, data analysis includes processes such as statistical analysis, data mining, and graphical display, which can intuitively and accurately display the relationships and characteristics between metadata.

[0146] In this embodiment, the first data analysis result refers to the data analysis result obtained by combining the integrated metadata in the metadata center with the metadata retrieval and analysis tool.

[0147] The beneficial effects of the above technical solution are: by performing data analysis on metadata, the value of metadata can be determined more accurately and clearly, improving the efficiency and quality of metadata management and ensuring the accuracy of metadata.

[0148] Example 7:

[0149] Based on Example 6, real-time improvements to the metadata management method are implemented, including:

[0150] Step 41: Perform a quality check on the integrated metadata to obtain the first quality check result;

[0151] Step 42: Based on the results of the first quality inspection, determine the data problems in the integrated metadata, and formulate corresponding metadata optimization solutions based on the data problems;

[0152] Step 43: Transform the metadata optimization scheme into a data improvement scheme for the target metadata, and improve the target metadata one by one according to the data improvement scheme, and monitor the improvement effect in real time.

[0153] Step 44: Based on the improvement effect of real-time monitoring, adjust the metadata management method of the corresponding target metadata in real time, thereby improving the data quality and data value of the metadata.

[0154] In this embodiment, quality checking is the process of evaluating the accuracy, completeness, consistency, and other quality attributes of metadata, which helps to ensure the quality and reliability of metadata.

[0155] In this embodiment, the first quality check result refers to the quality check result obtained after performing a quality check on the integrated metadata. The quality check refers to the process of evaluating the accuracy, completeness, consistency and other quality attributes of the metadata.

[0156] In this embodiment, data issues refer to any problems that do not conform to expectations or specifications discovered during the quality check of integrated metadata, such as missing data, duplicate data, incorrect data format, data inconsistency, etc.

[0157] In this embodiment, the metadata optimization scheme is a solution developed to address data problems discovered during metadata integration. The metadata optimization scheme typically includes steps such as data cleaning, data transformation, data merging, and data verification.

[0158] In this embodiment, scheme transformation is the process of converting metadata optimization schemes into specific data improvement schemes.

[0159] In this embodiment, the data improvement scheme is the specific operational steps and strategies obtained after the scheme is transformed. It is used to improve the target metadata one by one. The data improvement scheme usually includes specific data processing tasks, task execution order, data processing parameters and other information.

[0160] In this embodiment, real-time monitoring of the improvement effect refers to the process of tracking and evaluating the improvement of the target metadata in real time during the execution of the data improvement plan.

[0161] In this embodiment, metadata management methods refer to the strategies and processes for managing, maintaining, optimizing, and using metadata. These methods typically include metadata definition, metadata collection, metadata storage, metadata querying, and metadata updating. Based on real-time monitoring of improvement effects, metadata management methods are adjusted in real time according to actual conditions to improve the data quality and value of metadata.

[0162] The beneficial effects of the above technical solution are: by improving the metadata management method in real time, the management efficiency of metadata can be improved, thereby enabling timely processing and updating of metadata and increasing the value of metadata.

[0163] Example 8:

[0164] This invention provides a metadata maintenance and management system, such as... Figure 2 As shown, it includes:

[0165] The data acquisition and integration module is used to extract target metadata from the data source based on the data acquisition rules defined by the metadata model, and to integrate the extracted target metadata to obtain the first metadata set.

[0166] Intelligent retrieval module: used to index and store metadata based on preset indexing technology, and to determine data retrieval keywords for target metadata based on real-time user needs;

[0167] Application Analysis Module: Used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools;

[0168] Inspection and Improvement Module: When conducting data analysis, it is necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

[0169] The beneficial effects of the above technical solution are: by integrating, indexing, storing and analyzing metadata, the efficiency and quality of metadata management can be improved, the efficiency of metadata updates and the value of metadata can be increased, and the accuracy and security of data can be ensured.

[0170] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A metadata maintenance and management method, characterized in that, include: Step 1: Determine the data collection rules for extracting target metadata from the target data source based on the target data source's metadata model; According to the data collection rules, target metadata is extracted from the target data source in real time to obtain initial metadata; Used to clean, transform, and associate the initial metadata to obtain the first metadata set; Step 2: Determine the first indexing technique for indexing the metadata of the target data source based on the data type of the target data source; The first metadata set is stored using a first indexing technique to obtain a first storage result; real-time user needs, historical user needs, and historical behavior data of the target user are obtained; based on the historical behavior data of the target user, the historical user needs are split to obtain a first historical needs set; wherein each first historical needs subset in the first historical needs set corresponds to the same historical behavior data; the demand splitting schemes of each first historical needs subset in the first historical needs set are combined to obtain a reference demand splitting scheme for the target user; the urgency of each sub-demand in the real-time user needs is obtained, and each sub-demand in the real-time user needs is sorted based on the urgency to obtain a first ordered demand; the reference demand splitting scheme is optimized based on the first ordered demand to obtain a first demand... Calculate a set of splitting schemes; based on the first set of splitting schemes, split the real-time user needs, and analyze each sub-needle based on the splitting results to obtain the first needs analysis results for the target user; based on the first needs analysis results, determine the data fields for each sub-needle to obtain the first needs fields, and obtain the field parameters of each metadata field in the metadata model of the target data source; match the first needs fields with the field parameters of each metadata field in the metadata model, and evaluate the importance of each keyword to obtain the first keywords that meet the user needs; synthesize each first keyword that meets the user needs to obtain the first keyword set for the target metadata, and optimize the first keywords in the first keyword set to obtain the data retrieval keywords for the target metadata; Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then conduct data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool; Step 4: When conducting data analysis, it is also necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

2. The metadata maintenance and management method according to claim 1, characterized in that, The field parameters of metadata fields include: field meaning, data type, and value range.

3. The metadata maintenance and management method according to claim 1, characterized in that, Based on real-time user needs, a metadata retrieval and analysis tool is determined. This tool is then used to perform data analysis on the integrated metadata from the metadata center, including: Step 31: Determine the target metadata retrieval and analysis tools based on the user needs of the target users and the data characteristics of the target data source; Step 32: Connect the first metadata set to the metadata center to obtain the initial integrated metadata; Step 33: Based on the data themes and meanings in the metadata center, integrate and optimize the initial integrated metadata to obtain integrated metadata; Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval and analysis tools to obtain the first data analysis result.

4. The metadata maintenance and management method according to claim 3, characterized in that, When conducting data analysis, it is also necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results. This leads to the development of improvement plans, enabling real-time improvements to metadata management methods, including: Step 41: Perform a quality check on the integrated metadata to obtain the first quality check result; Step 42: Based on the results of the first quality inspection, determine the data problems in the integrated metadata, and formulate corresponding metadata optimization solutions based on the data problems; Step 43: Transform the metadata optimization scheme into a data improvement scheme for the target metadata, and improve the target metadata one by one according to the data improvement scheme, and monitor the improvement effect in real time. Step 44: Based on the improvement effect of real-time monitoring, adjust the metadata management method of the corresponding target metadata in real time, thereby improving the data quality and data value of the metadata.

5. A metadata maintenance and management system, characterized in that, include: Data Acquisition and Integration Module: Used to determine data acquisition rules for extracting target metadata from the target data source based on the target data source's metadata model; According to the data collection rules, target metadata is extracted from the target data source in real time to obtain initial metadata; Used to clean, transform, and associate the initial metadata to obtain the first metadata set; Intelligent retrieval module: A first indexing technology used to determine the metadata of the target data source based on the data type of the target data source; The first metadata set is stored using a first indexing technique to obtain a first storage result; real-time user needs, historical user needs, and historical behavior data of the target user are obtained; based on the historical behavior data of the target user, the historical user needs are split to obtain a first historical needs set; wherein each first historical needs subset in the first historical needs set corresponds to the same historical behavior data; the demand splitting schemes of each first historical needs subset in the first historical needs set are combined to obtain a reference demand splitting scheme for the target user; the urgency of each sub-demand in the real-time user needs is obtained, and each sub-demand in the real-time user needs is sorted based on the urgency to obtain a first ordered demand; the reference demand splitting scheme is optimized based on the first ordered demand to obtain a first demand... Calculate a set of splitting schemes; based on the first set of splitting schemes, split the real-time user needs, and analyze each sub-needle based on the splitting results to obtain the first needs analysis results for the target user; based on the first needs analysis results, determine the data fields for each sub-needle to obtain the first needs fields, and obtain the field parameters of each metadata field in the metadata model of the target data source; match the first needs fields with the field parameters of each metadata field in the metadata model, and evaluate the importance of each keyword to obtain the first keywords that meet the user needs; synthesize each first keyword that meets the user needs to obtain the first keyword set for the target metadata, and optimize the first keywords in the first keyword set to obtain the data retrieval keywords for the target metadata; Application Analysis Module: Used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools; Inspection and Improvement Module: When conducting data analysis, it is necessary to perform quality checks on metadata and determine metadata optimization schemes based on the quality check results, thereby formulating improvement plans and realizing real-time improvement of metadata management methods.

Images