Metadata maintenance management method and system

Abstract

The invention provides a metadata maintenance management method and system, and relates to the technical field of data management, and the method comprises the steps: defining an acquisition rule based on a metadata model to extract target metadata from a data source, and carrying out the data integration of the extracted target metadata to obtain a first metadata set; performing metadata index storage based on a preset index technology, and determining a data retrieval keyword of the target metadata based on a real-time user demand; determining a metadata retrieval analysis tool based on real-time user requirements, so as to perform data analysis on the integrated metadata of the metadata center; according to the method, the metadata is subjected to quality inspection, and a metadata optimization scheme is determined based on a quality inspection result, so that an improvement scheme is formulated, and real-time improvement of the metadata management method is realized. By performing data integration, index storage and data analysis on the metadata, the efficiency and the quality of metadata management can be improved, the metadata updating efficiency and the metadata value are improved, and the accuracy and the safety of the data are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a metadata maintenance management method and system. Background Art

[0002] At present, metadata management refers to the process of effectively managing, maintaining and utilizing metadata. Metadata management plays a vital role in data governance and data lifecycle management.

[0003] However, existing metadata management methods have some problems, such as incomplete data lineage tracking, inadequate data quality monitoring, and untimely metadata updates. These problems affect the effective use of data and the accuracy of corporate decision-making.

[0004] Therefore, the present invention provides a metadata maintenance management method and system. Summary of the invention

[0005] The present invention provides a metadata maintenance management method and system, which are used to solve the problems of untimely and comprehensive data tracking, inadequate data quality monitoring, and untimely and inaccurate updating in the prior art.

[0006] The present invention provides a metadata maintenance management method, comprising: Step 1: extract target metadata from the data source based on the metadata model definition collection rule, and integrate the extracted target metadata to obtain a first metadata set; Step 2: Metadata indexing and storage is performed based on preset indexing technology, and data retrieval keywords of target metadata are determined based on real-time user needs; Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool; Step 4: When conducting data analysis, it is also necessary to conduct a quality check on the metadata and determine the metadata optimization plan based on the quality check results, so as to develop an improvement plan to achieve real-time improvement of the metadata management method.

[0007] According to the metadata model-based collection rule provided by the present invention, target metadata is extracted from a data source, and the extracted target metadata is integrated to obtain a first metadata set, including: Step 11: Determine a data collection rule for extracting target metadata from the target data source based on the metadata model of the target data source; Step 12: According to the data collection rules, the target metadata is extracted from the target data source in real time to obtain the initial metadata; Step 13: Cleaning, converting and associating the extracted initial metadata to obtain a first metadata set.

[0008] The metadata index storage based on the preset index technology provided by the present invention and the determination of data retrieval keywords of the target metadata based on real-time user needs include: Step 21: determining an indexing technology for performing data indexing on metadata of the target data source based on the data type of the target data source, and obtaining a first indexing technology; Step 22: storing the first metadata set according to the first indexing technology to obtain a first storage result; Step 23: Acquire the real-time user needs of the target user, and perform demand segmentation and demand analysis to determine the target user's demand fields, and determine the data retrieval keywords of the target metadata based on the demand fields.

[0009] According to the present invention, the real-time user needs of the target user are obtained, and the needs are split and analyzed to determine the need fields of the target user, and the data retrieval keywords of the target metadata are determined based on the need fields, including: Step 231: obtaining the real-time user demand of the target user, and simultaneously obtaining the historical user demand and historical behavior data of the target user; Step 232: splitting the corresponding historical user demands based on the historical behavior data of the target user, thereby obtaining a first historical demand set; Wherein, each first historical demand subset in the first historical demand set corresponds to the same historical behavior data; Step 233: synthesizing the demand splitting schemes of each first historical demand subset in the first historical demand set, thereby obtaining a reference demand splitting scheme for the target user; Step 234: obtaining the urgency of each sub-demand in the real-time user demand, and sorting each sub-demand of the real-time user demand based on the urgency of the demand to obtain a first ordered demand; Step 235: Optimizing the reference demand splitting solution based on the first ordered demand to obtain a first demand splitting solution set; Step 236: split the real-time user demand based on the first demand splitting solution set, and perform demand analysis on each split sub-demand based on the demand splitting result, so as to obtain the first demand analysis result of the target user; Step 237: Determine the data field of each corresponding split sub-demand based on the first demand analysis result to obtain the first demand field, and at the same time, obtain the field parameters of each metadata field in the metadata model of the target data source; Step 238: Match the first requirement field with the field parameters of each metadata field in the metadata model to obtain keywords that meet the user requirements of the target user, and evaluate the importance of each keyword to obtain the first keyword that meets the user requirements; Step 239: synthesize each first keyword that meets the user's needs to obtain a first keyword set of target metadata, and optimize the first keyword in the first keyword set to obtain a data retrieval keyword of the target metadata.

[0010] The field parameters of the metadata field provided by the present invention include: field meaning, data type, and value range.

[0011] According to the present invention, a metadata retrieval and analysis tool is determined based on real-time user needs, thereby performing data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool, including: Step 31: Determine a retrieval and analysis tool for target metadata based on the user needs of target users and the data characteristics of target data sources; Step 32: Connect the first metadata set to the metadata center to obtain initial integrated metadata; Step 33: Integrate and optimize the initial integrated metadata based on the data subject and data meaning of the metadata center to obtain integrated metadata; Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval analysis tool to obtain a first data analysis result.

[0012] According to the present invention, when performing data analysis, it is also necessary to perform a quality check on the metadata, and determine a metadata optimization plan based on the quality check result, so as to formulate an improvement plan, thereby achieving real-time improvement of the metadata management method, including: Step 41: Performing a quality check on the integrated metadata, thereby obtaining a first quality check result; Step 42: determining data problems of the integrated metadata based on the first quality inspection result, and formulating corresponding metadata optimization solutions based on the data problems; Step 43: converting the metadata optimization scheme into a scheme, thereby obtaining a data improvement scheme for the target metadata, and improving the target metadata one by one according to the data improvement scheme, and monitoring the improvement effect in real time; Step 44: Based on the real-time monitoring improvement effect, the metadata management method corresponding to the target metadata is adjusted in real time, thereby improving the data quality and data value of the metadata.

[0013] The present invention provides a metadata maintenance and management system, comprising: Collection and integration module: used to extract target metadata from the data source based on the metadata model definition collection rule, and perform data integration on the extracted target metadata to obtain a first metadata set; Intelligent retrieval module: used to index and store metadata based on preset indexing technology, and determine data retrieval keywords for target metadata based on real-time user needs; Application analysis module: used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools; Inspection and Improvement Module: It is used to perform quality checks on metadata during data analysis, and determine metadata optimization plans based on the quality check results, thereby formulating improvement plans to achieve real-time improvements in metadata management methods.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: the metadata maintenance management method and system provided by the present invention can improve the efficiency and quality of metadata management, improve the metadata update efficiency and metadata value, and ensure the accuracy and security of data by performing data integration, index storage and data analysis on metadata. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 is a flowchart of a metadata maintenance and management method provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a metadata maintenance and management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1: The embodiment of the present invention provides a metadata maintenance management method, such as Figure 1 As shown, including: Step 1: extract target metadata from the data source based on the metadata model definition collection rule, and integrate the extracted target metadata to obtain a first metadata set; Step 2: Metadata indexing and storage is performed based on preset indexing technology, and data retrieval keywords of target metadata are determined based on real-time user needs; Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool; Step 4: When conducting data analysis, it is also necessary to conduct a quality check on the metadata and determine the metadata optimization plan based on the quality check results, so as to develop an improvement plan to achieve real-time improvement of the metadata management method.

[0019] In this embodiment, the metadata model is a definition and description of the structure, attributes and relationships of metadata, and defines how to organize, store and manage metadata so that it can be effectively retrieved, analyzed and used.

[0020] In this embodiment, the collection rule refers to a criterion or standard followed when extracting metadata from a data source. The collection rule defines which metadata should be extracted, how to extract it, and how to process the extracted data.

[0021] In this embodiment, metadata is data used to describe data, including information such as attributes, structure, source, and relationship of the data.

[0022] In this embodiment, metadata resources refer to relevant information of data resources, such as data structure, content, source, format, quality, access rights, etc.

[0023] In this embodiment, data integration is the process of merging data from different data sources into a consistent and complete data set, including data cleaning, transformation, and association processes. In metadata management, data integration usually involves merging multiple metadata sets into a more comprehensive metadata set.

[0024] In this embodiment, the first metadata set is a metadata set obtained by performing data cleaning, data conversion, and data integration on the collected real-time metadata.

[0025] In this embodiment, the preset index technology is a technology for quickly retrieving and querying a large amount of data. For example, the preset index technology includes an inverted index, a B-tree index, and the like.

[0026] In this embodiment, metadata index storage is a process of storing metadata and its index information in a specific location or database, which helps to quickly locate and retrieve metadata.

[0027] In this embodiment, the real-time user demand refers to the user's demand or query requirement for metadata at the current moment. In metadata management, understanding and satisfying the real-time user demand is the key to providing high-quality services.

[0028] In this embodiment, the data search keyword refers to the keyword corresponding to the metadata when performing data search. For example, the metadata is abcdefg, where a is the data search keyword of the metadata. Then, by inputting data a in the metadata center, the corresponding abcdefg can be obtained.

[0029] In this embodiment, the metadata center is a system or platform that centrally stores and manages metadata, and generally provides functions such as metadata query, analysis, quality inspection, and optimization.

[0030] In this embodiment, the metadata retrieval and analysis tool is a software tool for retrieving, analyzing and visualizing metadata.

[0031] In this embodiment, data analysis includes statistical analysis, data mining, graphical display and other processes, so as to intuitively and accurately display the relationship and characteristics between metadata.

[0032] In this embodiment, quality checking is the process of evaluating the accuracy, completeness, consistency and other quality attributes of metadata, which helps to ensure the quality and reliability of metadata.

[0033] In this embodiment, the metadata optimization solution refers to a series of improvement measures or suggestions proposed based on the metadata quality inspection results.

[0034] The beneficial effects of the above technical solution are: by integrating, indexing and analyzing metadata, the efficiency and quality of metadata management can be improved, the metadata update efficiency and metadata value can be improved, and the accuracy and security of data can be ensured.

[0035] Embodiment 2: Based on Example 1, target metadata is extracted from a data source based on a metadata model definition collection rule, and the extracted target metadata is integrated to obtain a first metadata set, including: Step 11: Determine a data collection rule for extracting target metadata from the target data source based on the metadata model of the target data source; Step 12: According to the data collection rules, the target metadata is extracted from the target data source in real time to obtain the initial metadata; Step 13: Cleaning, converting and associating the extracted initial metadata to obtain a first metadata set.

[0036] In this embodiment, the metadata model is a definition and description of the structure, attributes and relationships of metadata, and defines how to organize, store and manage metadata so that it can be effectively retrieved, analyzed and used.

[0037] In this embodiment, the data collection rule refers to a criterion or standard followed when extracting metadata from a data source. The collection rule defines which metadata should be extracted, how to extract it, and how to process the extracted data.

[0038] In this embodiment, metadata is data used to describe data, including information such as attributes, structure, source, and relationship of the data.

[0039] In this embodiment, the initial metadata refers to target metadata extracted in real time from the target data source.

[0040] In this embodiment, the first metadata set is a metadata set obtained by performing data cleaning, data conversion, and data association on the collected initial metadata.

[0041] The beneficial effect of the above technical solution is: by determining the data collection rules, the initial metadata is extracted, and the data is cleaned, converted, and associated to obtain the first metadata set, thereby realizing data integration of the metadata, so that index storage and data analysis can be performed more accurately to ensure the accuracy of the data.

[0042] Embodiment 3: Based on Example 2, metadata index storage is performed based on a preset index technology, and data retrieval keywords of target metadata are determined based on real-time user needs, including: Step 21: determining an indexing technology for performing data indexing on metadata of the target data source based on the data type of the target data source, and obtaining a first indexing technology; Step 22: storing the first metadata set according to the first indexing technology to obtain a first storage result; Step 23: Acquire the real-time user needs of the target user, and perform demand segmentation and demand analysis to determine the target user's demand fields, and determine the data retrieval keywords of the target metadata based on the demand fields.

[0043] In this embodiment, different data types exist according to different division schemes. For example, according to different application fields, functions and description objects, the data types of the target data source include business metadata, technical metadata, operation metadata, etc.

[0044] In this embodiment, the first indexing technology refers to a data indexing technology that is extracted according to the data type of the target data source and is consistent with the metadata of the target data source. For example, the first indexing technology includes an inverted index, a B-tree index, and the like.

[0045] In this embodiment, storing according to the first indexing technology is a process of storing metadata and its index information in a specific location or database, which helps to quickly locate and retrieve metadata.

[0046] In this embodiment, the first storage result refers to a storage result obtained after storing the first metadata set according to the first indexing technology.

[0047] In this embodiment, the real-time user demand refers to the user's demand or query requirement for metadata at the current moment. In metadata management, understanding and satisfying the real-time user demand is the key to providing high-quality services.

[0048] In this embodiment, demand splitting refers to splitting the real-time user demands according to different demand types of the real-time user demands.

[0049] In this embodiment, the demand analysis refers to performing demand analysis on each sub-demand based on the demand splitting result, so as to determine the demand field corresponding to each sub-demand.

[0050] In this embodiment, the data search keyword refers to the keyword corresponding to the metadata when performing data search. For example, the metadata is abcdefg, where a is the data search keyword of the metadata. Then, by inputting data a in the metadata center, the corresponding abcdefg can be obtained.

[0051] The beneficial effect of the above technical solution is: by determining the indexing technology of metadata, the metadata is indexed and stored, so that data analysis can be performed on the storage results, the efficiency and quality of metadata management are improved, the value of metadata is increased, and the accuracy of data is ensured.

[0052] Embodiment 4: Based on Example 3, the data search keywords of the target metadata are determined, including: Step 231: obtaining the real-time user demand of the target user, and simultaneously obtaining the historical user demand and historical behavior data of the target user; Step 232: splitting the corresponding historical user demands based on the historical behavior data of the target user, thereby obtaining a first historical demand set; Wherein, each first historical demand subset in the first historical demand set corresponds to the same historical behavior data; Step 233: synthesizing the demand splitting schemes of each first historical demand subset in the first historical demand set, thereby obtaining a reference demand splitting scheme for the target user; Step 234: obtaining the urgency of each sub-demand in the real-time user demand, and sorting each sub-demand of the real-time user demand based on the urgency of the demand to obtain a first ordered demand; Step 235: Optimizing the reference demand splitting solution based on the first ordered demand to obtain a first demand splitting solution set; Step 236: split the real-time user demand based on the first demand splitting solution set, and perform demand analysis on each split sub-demand based on the demand splitting result, so as to obtain the first demand analysis result of the target user; Step 237: Determine the data field of each corresponding split sub-demand based on the first demand analysis result to obtain the first demand field, and at the same time, obtain the field parameters of each metadata field in the metadata model of the target data source; Step 238: Match the first requirement field with the field parameters of each metadata field in the metadata model to obtain keywords that meet the user requirements of the target user, and evaluate the importance of each keyword to obtain the first keyword that meets the user requirements; Step 239: synthesize each first keyword that meets the user's needs to obtain a first keyword set of target metadata, and optimize the first keyword in the first keyword set to obtain a data retrieval keyword of the target metadata.

[0053] In this embodiment, the real-time user demand refers to the user's demand or query requirement for metadata at the current moment. In metadata management, understanding and satisfying the real-time user demand is the key to providing high-quality services.

[0054] In this embodiment, historical behavior data refers to the target user's past behavior records, such as browsing records, purchase records, click records, etc. These data can reflect the user's preferences and habits.

[0055] In this embodiment, historical user demands refer to demands that were raised by target users in the past and have been recorded.

[0056] In this embodiment, demand splitting refers to breaking down complex or comprehensive user demands into multiple smaller demands that are more specific and easier to implement or analyze.

[0057] In this embodiment, the first historical demand set is a set obtained by splitting the historical user demands of the target user, wherein each first historical demand subset corresponds to the same historical behavior data.

[0058] In this embodiment, the reference demand splitting scheme is obtained based on the demand splitting scheme of each first historical demand subset in the first historical demand set, and is used as a reference scheme for guiding subsequent demand splitting and analysis.

[0059] In this embodiment, the demand urgency refers to the priority of each sub-demand in the real-time user demand.

[0060] In this embodiment, the first ordered demand is a demand sequence obtained by sorting each sub-demand of the real-time user demand according to the urgency of the demand.

[0061] In this embodiment, the first demand splitting solution set is a set of multiple demand splitting solutions obtained by optimizing the reference demand splitting solution based on the first ordered demand.

[0062] In this embodiment, the demand splitting result is a result obtained after splitting the real-time user demand according to the first demand splitting solution set.

[0063] In this embodiment, the split sub-demand is each sub-demand in the requirement splitting result.

[0064] In this embodiment, demand analysis refers to conducting an in-depth study on the split sub-demands to clarify their specific requirements, goals, constraints, etc.

[0065] In this embodiment, the first demand analysis result is a result obtained after demand analysis is performed on the split sub-demands.

[0066] In this embodiment, the requirement field is a data field used to describe or represent the data required for splitting sub-requirements.

[0067] In this embodiment, the first requirement field is a data field of each split sub-requirement determined based on the first requirement analysis result.

[0068] In this embodiment, the metadata model is a model used to describe the data structure, including data fields, field types, field relationships, etc.

[0069] In this embodiment, the metadata field refers to a data field in the metadata model.

[0070] In this embodiment, the field parameters are information used to describe the attributes or characteristics of the metadata field, including: field meaning, data type, and value range.

[0071] In this embodiment, the first keyword is a keyword that meets the needs of the target user.

[0072] In this embodiment, the first keyword set refers to a keyword set including all first keywords.

[0073] In this embodiment, the data search keyword refers to the keyword corresponding to the metadata when performing data search. For example, the metadata is abcdefg, where a is the data search keyword of the metadata. Then, by inputting data a in the metadata center, the corresponding abcdefg can be obtained.

[0074] The beneficial effects of the above technical solution are: by indexing and storing metadata, data analysis can be performed on the storage results, thereby improving the efficiency and quality of metadata management, increasing the value of metadata, and ensuring the accuracy of data.

[0075] Embodiment 5: Based on Example 4, the field parameters of the metadata field include: field meaning, data type, and value range.

[0076] The beneficial effects of the above technical solution are: by indexing and storing metadata, data analysis can be performed on the storage results, thereby improving the efficiency and quality of metadata management, increasing the value of metadata, and ensuring the accuracy of data.

[0077] Embodiment 6: Based on Example 3, data analysis is performed on the integrated metadata of the metadata center, including: Step 31: Determine a retrieval and analysis tool for target metadata based on the user needs of target users and the data characteristics of target data sources; Step 32: Connect the first metadata set to the metadata center to obtain initial integrated metadata; Step 33: Integrate and optimize the initial integrated metadata based on the data subject and data meaning of the metadata center to obtain integrated metadata; Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval analysis tool to obtain a first data analysis result.

[0078] In this embodiment, the metadata center is a system or platform that centrally stores and manages metadata, and generally provides functions such as metadata query, analysis, quality inspection, and optimization.

[0079] In this embodiment, the metadata retrieval and analysis tool is a software tool for retrieving, analyzing and visualizing metadata.

[0080] In this embodiment, the initial integrated metadata refers to integrated data obtained after the first data set is connected to the metadata center and data is normalized and reorganized.

[0081] In this embodiment, the integrated metadata refers to the integrated metadata obtained by integrating and optimizing the initial integrated metadata according to the data subject and data meaning of the metadata center.

[0082] In this embodiment, data analysis includes statistical analysis, data mining, graphical display and other processes, so as to intuitively and accurately display the relationship and characteristics between metadata.

[0083] In this embodiment, the first data analysis result refers to a data analysis result obtained by performing data analysis on the integrated metadata in the metadata center in combination with a metadata retrieval analysis tool.

[0084] The beneficial effect of the above technical solution is: by performing data analysis on metadata, the value of metadata can be determined more accurately and clearly, the efficiency of metadata management and the quality of metadata can be improved, and the accuracy of metadata can be guaranteed.

[0085] Embodiment 7: Based on Example 6, a real-time improvement of the metadata management method is achieved, including: Step 41: Performing a quality check on the integrated metadata, thereby obtaining a first quality check result; Step 42: determining data problems of the integrated metadata based on the first quality inspection result, and formulating corresponding metadata optimization solutions based on the data problems; Step 43: converting the metadata optimization scheme into a scheme, thereby obtaining a data improvement scheme for the target metadata, and improving the target metadata one by one according to the data improvement scheme, and monitoring the improvement effect in real time; Step 44: Based on the real-time monitoring improvement effect, the metadata management method corresponding to the target metadata is adjusted in real time, thereby improving the data quality and data value of the metadata.

[0086] In this embodiment, quality checking is a process of evaluating the accuracy, completeness, consistency and other quality attributes of metadata, which helps to ensure the quality and reliability of metadata.

[0087] In this embodiment, the first quality check result refers to a quality check result obtained after performing a quality check on the integrated metadata, wherein performing a quality check refers to a process of evaluating the accuracy, completeness, consistency and other quality attributes of the metadata.

[0088] In this embodiment, data problems include any problems that are found during the quality check of the integrated metadata and are not in compliance with expectations or specifications, such as missing data, duplicate data, incorrect data format, inconsistent data, etc.

[0089] In this embodiment, the metadata optimization solution is a solution formulated for data problems found in the integrated metadata. The metadata optimization solution usually includes steps such as data cleaning, data conversion, data merging, and data verification.

[0090] In this embodiment, solution conversion is a process of converting a metadata optimization solution into a specific data improvement solution.

[0091] In this embodiment, the data improvement plan is the specific operation steps and strategies obtained after the plan conversion, which is used to improve the target metadata one by one. The data improvement plan usually includes specific data processing tasks, task execution order, data processing parameters and other information.

[0092] In this embodiment, real-time monitoring of improvement effects refers to a process of real-time tracking and evaluating the improvement status of target metadata during the execution of the data improvement solution.

[0093] In this embodiment, the metadata management method refers to the strategy and process for managing, maintaining, optimizing and using metadata. These methods generally include metadata definition, metadata collection, metadata storage, metadata query, metadata update and other aspects. Based on real-time monitoring of the improvement effect, the metadata management method is adjusted in real time according to the actual situation to improve the data quality and data value of the metadata.

[0094] The beneficial effect of the above technical solution is that: by improving the metadata management method in real time, the management efficiency of metadata can be improved, so that metadata processing and updating can be carried out in time, thereby improving the use value of metadata.

[0095] Embodiment 8: The embodiment of the present invention provides a metadata maintenance management system, such as Figure 2 As shown, including: Collection and integration module: used to extract target metadata from the data source based on the metadata model definition collection rule, and perform data integration on the extracted target metadata to obtain a first metadata set; Intelligent retrieval module: used to index and store metadata based on preset indexing technology, and determine data retrieval keywords for target metadata based on real-time user needs; Application analysis module: used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools; Inspection and Improvement Module: It is used to perform quality checks on metadata during data analysis, and determine metadata optimization plans based on the quality check results, thereby formulating improvement plans to achieve real-time improvements in metadata management methods.

[0096] The beneficial effects of the above technical solution are: by integrating, indexing and analyzing metadata, the efficiency and quality of metadata management can be improved, the metadata update efficiency and metadata value can be improved, and the accuracy and security of data can be ensured.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A metadata maintenance and management method, characterized in that: include: Step 1: extract target metadata from the data source based on the metadata model definition collection rule, and integrate the extracted target metadata to obtain a first metadata set; Step 2: Perform metadata index storage based on preset index technology, and determine data retrieval keywords for target metadata based on real-time user needs; Step 3: Determine the metadata retrieval and analysis tool based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tool; Step 4: When conducting data analysis, it is also necessary to conduct a quality check on the metadata and determine the metadata optimization plan based on the quality check results, so as to develop an improvement plan to achieve real-time improvement of the metadata management method.

2. A metadata maintenance and management method according to claim 1, characterized in that: The target metadata is extracted from the data source based on the metadata model definition collection rule, and the extracted target metadata is integrated to obtain a first metadata set, including: Step 11: Determine a data collection rule for extracting target metadata from the target data source based on the metadata model of the target data source; Step 12: According to the data collection rules, the target metadata is extracted from the target data source in real time to obtain the initial metadata; Step 13: Cleaning, converting and associating the extracted initial metadata to obtain a first metadata set.

3. A metadata maintenance and management method according to claim 2, characterized in that: Metadata indexing and storage are performed based on preset indexing technology, and data retrieval keywords for target metadata are determined based on real-time user needs, including: Step 21: determining an indexing technology for performing data indexing on metadata of the target data source based on the data type of the target data source, and obtaining a first indexing technology; Step 22: storing the first metadata set according to the first indexing technology to obtain a first storage result; Step 23: Acquire the real-time user needs of the target user, and perform demand segmentation and demand analysis to determine the target user's demand fields, and determine the data retrieval keywords of the target metadata based on the demand fields.

4. A metadata maintenance and management method according to claim 3, characterized in that: Obtain the real-time user needs of the target users, and perform demand segmentation and demand analysis to determine the target users' demand fields, and determine the data retrieval keywords of the target metadata based on the demand fields, including: Step 231: obtaining the real-time user demand of the target user, and simultaneously obtaining the historical user demand and historical behavior data of the target user; Step 232: splitting the corresponding historical user demands based on the historical behavior data of the target user, thereby obtaining a first historical demand set; Wherein, each first historical demand subset in the first historical demand set corresponds to the same historical behavior data; Step 233: synthesizing the demand splitting schemes of each first historical demand subset in the first historical demand set, thereby obtaining a reference demand splitting scheme for the target user; Step 234: obtaining the urgency of each sub-demand in the real-time user demand, and sorting each sub-demand of the real-time user demand based on the urgency of the demand to obtain a first ordered demand; Step 235: Optimizing the reference demand splitting solution based on the first ordered demand to obtain a first demand splitting solution set; Step 236: split the real-time user demand based on the first demand splitting solution set, and perform demand analysis on each split sub-demand based on the demand splitting result, so as to obtain the first demand analysis result of the target user; Step 237: Determine the data field of each corresponding split sub-demand based on the first demand analysis result to obtain the first demand field, and at the same time, obtain the field parameters of each metadata field in the metadata model of the target data source; Step 238: Match the first requirement field with the field parameters of each metadata field in the metadata model to obtain keywords that meet the user requirements of the target user, and evaluate the importance of each keyword to obtain the first keyword that meets the user requirements; Step 239: synthesize each first keyword that meets the user's needs to obtain a first keyword set of target metadata, and optimize the first keyword in the first keyword set to obtain a data retrieval keyword of the target metadata.

5. A metadata maintenance management method according to claim 4, characterized in that: The field parameters of the metadata field include: field meaning, data type, and value range.

6. A metadata maintenance and management method according to claim 3, characterized in that: Determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools, including: Step 31: Determine a retrieval and analysis tool for target metadata based on the user needs of target users and the data characteristics of target data sources; Step 32: Connect the first metadata set to the metadata center to obtain initial integrated metadata; Step 33: Integrate and optimize the initial integrated metadata based on the data subject and data meaning of the metadata center to obtain integrated metadata; Step 34: Perform data analysis on the integrated metadata in the metadata center based on the retrieval analysis tool to obtain a first data analysis result.

7. A metadata maintenance and management method according to claim 6, characterized in that: When conducting data analysis, it is also necessary to conduct a quality check on the metadata and determine the metadata optimization plan based on the quality check results, so as to formulate an improvement plan to achieve real-time improvement of the metadata management method, including: Step 41: Performing a quality check on the integrated metadata, thereby obtaining a first quality check result; Step 42: determining data problems of the integrated metadata based on the first quality inspection result, and formulating corresponding metadata optimization solutions based on the data problems; Step 43: converting the metadata optimization scheme into a scheme, thereby obtaining a data improvement scheme for the target metadata, and improving the target metadata one by one according to the data improvement scheme, and monitoring the improvement effect in real time; Step 44: Based on the real-time monitoring improvement effect, the metadata management method corresponding to the target metadata is adjusted in real time, thereby improving the data quality and data value of the metadata.

8. A metadata maintenance and management system, characterized in that: include: Collection and integration module: used to extract target metadata from the data source based on the metadata model definition collection rule, and perform data integration on the extracted target metadata to obtain a first metadata set; Intelligent retrieval module: used to index and store metadata based on preset indexing technology, and determine data retrieval keywords for target metadata based on real-time user needs; Application analysis module: used to determine metadata retrieval and analysis tools based on real-time user needs, and then perform data analysis on the integrated metadata of the metadata center based on the metadata retrieval and analysis tools; Inspection and Improvement Module: It is used to perform quality checks on metadata during data analysis, and determine metadata optimization plans based on the quality check results, thereby formulating improvement plans to achieve real-time improvements in metadata management methods.

Images