Data resource directory management method based on metadata driving

Abstract

The invention provides a metadata-driven data resource directory management method, which relates to the technical field of data management, and comprises the following steps of: periodically scanning a data source by using a data warehouse to obtain metadata resources, and managing the metadata resources; classifying and labeling the metadata resources based on a set classification rule; performing permission or role binding on the metadata resources to form a target data map; the data resource catalog is updated regularly, and a data service interface and a search function are provided for each data resource. Metadata resources obtained by using a data warehouse to regularly scan a data source are managed; based on a set classification rule, the metadata resources are classified and labeled; performing permission or role binding on the metadata resources to form a target data map; according to the method, the data resource catalog is periodically updated, and the data service interface and the search function are provided for each data resource, so that the utilization efficiency of the data resources and the accuracy of data resource management can be effectively improved, the data security is enhanced, and the support capability of business decision is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a data resource directory management method based on metadata drive. Background Art

[0002] In recent years, with the continuous deepening of enterprise informatization construction, the management and utilization of data resources have become increasingly important. Traditional data resource management methods often have problems such as scattered data resources and difficulty in unified management and discovery, resulting in low utilization of data resources and failure to meet the growing data needs of enterprises. Therefore, how to efficiently manage and utilize data resources has become one of the current research focuses.

[0003] Therefore, the present invention provides a data resource directory management method based on metadata drive. Summary of the invention

[0004] The present invention provides a data resource directory management method based on metadata drive, which is used to manage metadata resources obtained by regularly scanning data sources using a data warehouse; classify and label metadata resources based on set classification rules; bind metadata resources to permissions or roles to form a target data map; regularly update the data resource directory, and provide a data service interface and search function for each data resource, which can effectively improve the utilization efficiency of data resources and the accuracy of data resource governance, enhance data security, and strengthen the support capability for business decision-making.

[0005] The present invention provides a data resource directory management method based on metadata drive, comprising: Step 1: Use the target data warehouse to regularly scan the data source to obtain metadata resources and manage the metadata resources; Step 2: Based on the set classification rules, classify the metadata resources and mark the corresponding functional tags; Step 3: Bind permissions or roles to metadata resources to form a target data map; Step 4: Update the data resource directory regularly and provide data service interface and search function for each data resource.

[0006] Preferably, based on the set classification rules, the metadata resources are classified and marked with corresponding functional tags, including: Based on the set classification rules, metadata resources are classified into three levels according to business functions to obtain business-classification data; Creating corresponding business function labels for the business-classification data, and adding corresponding business function labels in the metadata management system; Perform secondary classification on the business-classification data according to the data type to obtain numerical-classification data; Creating corresponding data type labels for the number-classification data, and adding corresponding number-classification function labels in the metadata management system; Classify the classified data into one level according to the data security level to obtain the secure classified data; A corresponding data security level label is created for the security-classified data, and a level function label is added accordingly in the metadata management system.

[0007] Preferably, metadata resources are bound to permissions or roles to form a target data map, including: Using metadata resources and corresponding level function labels as matching conditions, extracting a first role and a first permission group from a set role-permission database; In the metadata management system, binding the first role and the first permission group to the current metadata resource; According to the preset map layout requirements, use the set drawing tool to draw the initial data map; Inputting the metadata resource and the corresponding function label, the first role and the first permission group into a setting drawing tool; The setting drawing tool establishes the relationship between the metadata resource, the function label, the first role and the first permission group, adjusts the layout of the initial data map, and forms the target data map.

[0008] Preferably, the data resource directory is updated regularly, and a data service interface and search function are provided for each data resource, including: Use data resources and corresponding metadata resources to establish a target data resource catalog system; Based on the monitoring characteristics of data resources, different update mechanisms are used to regularly update the metadata resources in the target data resource directory system; Import the metadata resources in the target data resource directory into a set search engine, establish a metadata index, and provide multiple set search methods, and then integrate the set search engine into the target data resource directory system; Develop the data service interface after defining its functions and parameters based on user needs and the characteristics of data resources; Regularly maintain the data service interface and take appropriate repair measures to repair loopholes when they exist; Use pre-deployed monitoring tools to monitor the usage of data service interfaces in real time and obtain interface performance data; The interface performance data is input into a pre-established anomaly detection model, and after abnormality judgment and annotation of the interface performance data, the data is output as an anomaly detection result; If there is interface performance data marked as abnormal in the abnormality detection result, it is determined that there is an interface abnormality in the current data service interface, and the interface performance data marked as abnormal is used as a matching condition to match the corresponding abnormal maintenance strategy for maintenance.

[0009] Preferably, the set search method includes four search methods: keyword search, category search, attribute search and multiple search condition combination search.

[0010] Preferably, based on the monitoring characteristics of data resources, different update mechanisms are used to regularly update the metadata resources in the target data resource directory system, including: If the monitoring characteristic of the data resource is to support automatic monitoring, the current data resource is regarded as the first data resource; Using a set monitoring tool, periodically performing change detection on the first data resource in the target data resource directory system according to a preset dynamic update cycle to obtain a change detection result; According to the change detection result, if there is a change in the first data resource, automatically updating the corresponding metadata resource of the current first data resource in the target data resource directory system according to the change detection result; If the monitoring characteristic of the data resource is that it does not support automatic monitoring, the current data resource is regarded as the second data resource; For the second data resource in the target data resource directory system, a target update request is periodically submitted according to a preset dynamic update cycle by using a set manual update strategy; The target update request is audited, and when the audit is successful, the corresponding metadata resource of the current second data resource in the target data resource directory system is updated.

[0011] Preferably, it also includes: Obtain historical change data of the current data resource within a preset time period from the data resource change record library, and input it into a pre-established change prediction model to obtain predicted change data of the current data resource; Based on the predicted change time point of the current data resource in the predicted change data as the independent variable and the predicted metadata change amount as the dependent variable, an estimated resource change sequence diagram is established; Extracting statistical analysis indicators from the estimated resource change sequence diagram, performing change trend analysis to obtain an estimated change trend coefficient of the current data resource; Based on the set business evaluation indicators, the business needs of the current data resources are evaluated to obtain the demand evaluation indicator value; The acquired demand assessment index value is combined with the estimated change trend coefficient to calculate the resource update demand coefficient; The calculation formula of resource update demand coefficient is as follows: ; In the formula, It is expressed as the resource update requirement coefficient of the current data resource; It is expressed as the metadata change of the current data resource at the current moment; It is represented as the predicted metadata change amount at the jth predicted change time point of the current data resource, where j=1, 2, 3, , n; It is expressed as the time interval between the current moment and the jth predicted change time point; It is expressed as the estimated change trend coefficient of the current data resources; It is expressed as the weight of the impact of the predicted change trend of metadata on the update cycle of the changed data resources; It is expressed as the weight of the impact of business requirements on the update cycle of changing data resources; It is represented as the i-th set business evaluation index, where i=1,2; It is represented as the contribution weight of the i-th set business evaluation indicator to the business demand for evaluating the current data resources; e is represented as a constant with a value of 2.7; When the resource update demand coefficient falls within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource does not need to be updated; When the resource update demand coefficient does not fall within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource needs to be updated; By analyzing the predicted change data of the current data resources and the historical dynamic update cycle, the real-time dynamic cycle is obtained; The real-time dynamic cycle is used to replace the current preset dynamic update cycle to achieve updating.

[0012] Preferably, the real-time dynamic cycle is obtained by analyzing the predicted change data of the current data resource and the historical dynamic update cycle, including: Extracting the historical dynamic update cycle of the current data resource within a preset time period from the data resource change record library, and outputting it as the first analysis cycle; Outputting the historical generation time point of the first analysis period as the first update time; If the first analysis cycle does not exist, the resource update demand coefficient of the current data resource is combined with the predicted change data for analysis to calculate the first analysis dynamic cycle; The calculation formula for the first analysis dynamic period is as follows: ; In the formula, It is indicated as the first analysis dynamic cycle; it is indicated as the current preset dynamic update cycle; Expressed as resource renewal demand coefficient; It is expressed as setting an upper limit on the demand threshold; The time interval between the predicted time point corresponding to the minimum value of the metadata resource amount change in the predicted change data and the current time; It is represented as the time interval between the corresponding predicted time point of the maximum value of the metadata resource amount change in the predicted change data and the current time; e is represented as a constant with a value of 2.7; It is represented as the influence weight of the resource update demand degree of the current data resource on the calculation of the first analysis dynamic cycle; It is represented as the influence weight of the resource change prediction of the current data resource on the calculation of the first analysis dynamic cycle; It is expressed as the maximum value of the change in the amount of metadata resources in the predicted change data; Represents the minimum change in the metadata resource amount in the predicted change data outputting the first analysis dynamic cycle as a real-time dynamic cycle; If there is a single first analysis cycle, the second analysis dynamic cycle is calculated based on the historical time interval between the first update moment of the current first analysis cycle and the current moment and the historical resource change amount; The calculation formula for the second analysis dynamic period is as follows: ; In the formula, It is represented as the second analysis dynamic cycle; Represents the weight of the impact of resource demand and resource change forecast on the calculation of the second analysis dynamic cycle; It is represented as the influence weight of the historical update situation of the preset dynamic update period of the current data resource on the calculation of the second analysis dynamic period; It is represented as the historical time interval between the first update time of the current first analysis period and the current time; Expressed as the forecast time period; It is expressed as the absolute difference between the metadata resource amount at the last moment of prediction and the metadata resource amount at the current moment; It is represented as the absolute difference between the amount of data resources at the first update time and the amount of data resources at the current time; outputting the second analysis dynamic period as a real-time dynamic period; If there are multiple first analysis periods, a third analysis dynamic period is calculated based on the historical update intervals between the first update moments corresponding to all first analysis periods and the historical resource changes; Among them, the calculation formula of the third analysis dynamic period is as follows: ; In the formula, It is represented as the third analysis dynamic cycle; Represented as the cth first analysis period, c=1,2, , n; n represents the total number of the first analysis cycle; It is represented as the absolute difference between the metadata resource amount at the first update time corresponding to the cth first analysis period and the metadata resource amount at the current time; The third analysis dynamic cycle is output as a real-time dynamic cycle.

[0013] Compared with the prior art, the present invention has the following beneficial effects: By managing metadata resources obtained by regularly scanning data sources using data warehouses; classifying and labeling metadata resources based on set classification rules; binding metadata resources with permissions or roles to form a target data map; regularly updating the data resource directory and providing data service interfaces and search functions for each data resource, the utilization efficiency of data resources and the accuracy of data resource governance can be effectively improved, data security can be enhanced, and the support capabilities for business decisions can be strengthened.

[0014] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0015] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 The present invention is a flowchart of a metadata-driven data resource directory management method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0018] The embodiment of the present invention provides a data resource directory management method based on metadata drive, such as Figure 1 As shown, including: Step 1: Use the target data warehouse to regularly scan the data source to obtain metadata resources and manage the metadata resources; Step 2: Based on the set classification rules, classify the metadata resources and mark the corresponding functional tags; Step 3: Bind permissions or roles to metadata resources to form a target data map; Step 4: Update the data resource directory regularly and provide data service interface and search function for each data resource.

[0019] In this embodiment, the target data warehouse is a large, centralized storage system for storing and managing data extracted, transformed, and loaded (ETL) from various data sources; the data source is predetermined, such as a database, a file system, and a data lake; metadata resources refer to relevant information of data resources, such as the structure, content, source, format, quality, access rights, etc. of the data; functional labels include three types: business function labels, number function labels, and level function labels; the target data map is a visual tool or representation that shows the relationship between metadata resources, functional labels, the first role, and the first permission group; the data resource directory is a list of data resources within an organization or institution, consisting of data resources and corresponding metadata resources.

[0020] In this embodiment, the target data warehouse is used to periodically scan all data sources to obtain metadata resources and manage the metadata resources, including: Use the target data warehouse to scan all data sources regularly according to the preset time period to obtain all data tables, fields and metadata; Annotate the obtained metadata as a metadata resource; Importing the metadata resources into a set metadata management tool for centralized storage and maintenance; The preset time period is predetermined, such as weekly; and the set metadata management tool is a predetermined tool for centrally storing and maintaining metadata resources.

[0021] The beneficial effects of the above technical solution are: by managing the metadata resources obtained by regularly scanning data sources using a data warehouse; classifying and labeling metadata resources based on set classification rules; binding metadata resources with permissions or roles to form a target data map; regularly updating the data resource directory and providing a data service interface and search function for each data resource, it can effectively improve the utilization efficiency of data resources and the accuracy of data resource governance, enhance data security, and strengthen the support capabilities for business decisions.

[0022] The embodiment of the present invention provides a data resource directory management method based on metadata drive, which classifies metadata resources and marks corresponding function tags based on set classification rules, including: Based on the set classification rules, metadata resources are classified into three levels according to business functions to obtain business-classification data; Creating corresponding business function labels for the business-classification data, and adding corresponding business function labels in the metadata management system; Perform secondary classification on the business-classification data according to the data type to obtain numerical-classification data; Creating corresponding data type labels for the number-classification data, and adding corresponding number-classification function labels in the metadata management system; Classify the classified data into one level according to the data security level to obtain the secure classified data; A corresponding data security level label is created for the security-classified data, and a level function label is added accordingly in the metadata management system.

[0023] In this embodiment, the function labels include three types: business function labels, category function labels and level function labels; business-classified data is obtained by classifying metadata resources according to business functions, wherein business functions include sales, finance, human resources, etc.; category-classified data is obtained by classifying business-classified data according to data types, wherein data types include structured, unstructured, etc.; security-classified data is obtained by classifying category-classified data according to data security levels, wherein data security levels include general level, important level and core level.

[0024] The beneficial effects of the above technical solution are: by classifying metadata resources and marking corresponding functional tags based on set classification rules, data basis can be provided for establishing data maps, and through classification and labeling, metadata resources can be organized and stored in a more orderly manner, making search and retrieval faster and more accurate, thereby improving resource management efficiency.

[0025] The embodiment of the present invention provides a data resource directory management method based on metadata drive, which performs permission or role binding on metadata resources to form a target data map, including: Using metadata resources and corresponding level function labels as matching conditions, extracting a first role and a first permission group from a set role-permission database; In the metadata management system, binding the first role and the first permission group to the current metadata resource; According to the preset map layout requirements, use the set drawing tool to draw the initial data map; Inputting the metadata resource and the corresponding function label, the first role and the first permission group into a setting drawing tool; The setting drawing tool establishes the relationship between the metadata resource, the function label, the first role and the first permission group, adjusts the layout of the initial data map, and forms the target data map.

[0026] In this embodiment, the role-setting-authority database is a pre-established database that stores different roles and the permissions possessed by these roles, such as the read, write, and delete permissions of resources; the first role refers to one or more roles extracted from the role-setting-authority database that match the current metadata resource and the corresponding level function label, and these roles represent users or user groups that can access or operate the current metadata resource; the first permission group refers to a permission set that is extracted from the role-setting-authority database corresponding to the first role and matches the current metadata resource and the corresponding level function label, such as read data, write data; the metadata management system is a system for storing, managing, and A system that maintains metadata for data resources can define, classify, annotate, and manage metadata for various data resources in order to better understand and utilize these data resources; preset map layout requirements refer to map layout requirements set according to the specific needs of an organization or institution before drawing the initial data map, such as the arrangement of data resources, color coding, and label display method; initial data map refers to the initial map drawn using a set drawing tool in accordance with the preset map layout requirements, where the set drawing tool is a predetermined tool for drawing maps; target data map refers to a map drawn using a set drawing tool that displays metadata resources and related information (such as functional labels, roles, permissions, etc.).

[0027] The beneficial effects of the above technical solution are: by binding metadata resources with permissions or roles to form a target data map, it is possible to effectively prevent data leakage and abuse, simplify data management operations, improve data quality, and promote data sharing and collaboration.

[0028] The embodiment of the present invention provides a data resource directory management method based on metadata drive, which regularly updates the data resource directory and provides a data service interface and search function for each data resource, including: Use data resources and corresponding metadata resources to establish a target data resource catalog system; Based on the monitoring characteristics of data resources, different update mechanisms are used to regularly update the metadata resources in the target data resource directory system; Import the metadata resources in the target data resource directory into a set search engine, establish a metadata index, and provide multiple set search methods, and then integrate the set search engine into the target data resource directory system; Develop the data service interface after defining its functions and parameters based on user needs and the characteristics of data resources; Regularly maintain the data service interface and take appropriate repair measures to repair loopholes when they exist; Use pre-deployed monitoring tools to monitor the usage of data service interfaces in real time and obtain interface performance data; The interface performance data is input into a pre-established anomaly detection model, and after abnormality judgment and annotation of the interface performance data, the data is output as an anomaly detection result; If there is interface performance data marked as abnormal in the abnormality detection result, it is determined that there is an interface abnormality in the current data service interface, and the interface performance data marked as abnormal is used as a matching condition to match the corresponding abnormal maintenance strategy for maintenance.

[0029] In this embodiment, the target data resource directory system refers to a system for organizing and storing data resources. By using data resources and their corresponding metadata to build a structured directory, users can more easily find, access and manage resources; the monitoring feature refers to whether automated monitoring is supported or not supported; the update mechanism refers to the rules for regularly updating metadata resources in the data resource directory system. Different monitoring features use different update mechanisms.

[0030] In this embodiment, the search engine is set to be predetermined, such as Elasticsearch and Solr; the search methods are set to include keyword search, category search, attribute search, and search combined with multiple search conditions; user needs refer to data content, format, quality, and access requirements; the characteristics of data resources refer to data types, data formats, data sources, data quality, and data access rights; the data service interface refers to an interface provided for the target data resource directory system for interacting with external systems or applications, such as an API interface; defining the functions and parameters of the data service interface refers to clarifying information such as the service types, service contents, and parameters required for the services that the data service interface can provide; repair measures refer to regular maintenance of the data service interface, and the repair and improvement measures taken when vulnerabilities or problems are found, such as updating the interface code, repairing security vulnerabilities, and optimizing interface performance.

[0031] In this embodiment, interface performance data refers to data generated during the use of the data service interface, which reflects the interface performance and stability, such as response time and throughput; the set monitoring tool refers to a software tool used to monitor the operation of the data service interface in real time; the anomaly detection model refers to a model used to identify and analyze performance anomalies of the data service interface, which is obtained by training a neural network based on historical data or predefined rules; the anomaly detection result refers to the labeling result obtained after anomaly judgment on the interface performance data; the abnormal maintenance strategy refers to the exception handling strategy matched from the set exception handling table with the interface performance data marked as abnormal as the matching condition, wherein the exception handling strategy refers to a predefined maintenance step or maintenance operation, which is used to restore the normal operation of the system when an interface abnormality is detected, such as restarting the service; the set exception handling table is composed of interface performance data and corresponding exception handling strategies.

[0032] The beneficial effect of the above technical solution is: by regularly updating the data resource directory and providing a data service interface and search function for each data resource, the timeliness of the data can be ensured and data sharing and collaboration can be promoted.

[0033] The embodiment of the present invention provides a data resource directory management method based on metadata drive, which uses different update mechanisms to regularly update metadata resources in the target data resource directory system based on the monitoring characteristics of data resources, including: If the monitoring characteristic of the data resource is to support automatic monitoring, the current data resource is regarded as the first data resource; Using a set monitoring tool, periodically performing change detection on the first data resource in the target data resource directory system according to a preset dynamic update cycle to obtain a change detection result; According to the change detection result, if there is a change in the first data resource, automatically updating the corresponding metadata resource of the current first data resource in the target data resource directory system according to the change detection result; If the monitoring characteristic of the data resource is that it does not support automatic monitoring, the current data resource is regarded as the second data resource; For the second data resource in the target data resource directory system, a target update request is periodically submitted according to a preset dynamic update cycle by using a set manual update strategy; The target update request is audited, and when the audit is successful, the corresponding metadata resource of the current second data resource in the target data resource directory system is updated.

[0034] In this embodiment, the first data resource refers to a data resource whose monitoring characteristics support automated monitoring; the preset dynamic update cycle refers to the set frequency for submitting data resource change detection or update requests; the set monitoring tool refers to a tool pre-configured or specified for regular change detection of data resources, which can be a software program, script or other automated means, used to monitor the data resources in the target data resource directory system according to the preset dynamic update cycle to detect their changes; the change detection result refers to the result obtained after regular change detection of the first data resource in the target data resource directory system, which is used to reflect the changes in the data resource over a period of time, such as addition, deletion, modification and other operations; the second data resource refers to a data resource whose monitoring characteristics do not support automated monitoring; the set manual update policy is a pre-set update policy, which is composed of the request submission method, data identifier, update requester and update content description; the target update request is composed of the update time, update content and update reason.

[0035] The beneficial effect of the above technical solution is that by adopting different update mechanisms to regularly update the metadata resources in the target data resource directory system, the scalability and flexibility of the system can be enhanced, and the management and utilization efficiency of data resources can be improved.

[0036] The embodiment of the present invention provides a data resource directory management method based on metadata drive, further comprising: Obtain historical change data of the current data resource within a preset time period from the data resource change record library, and input it into a pre-established change prediction model to obtain predicted change data of the current data resource; Based on the predicted change time point of the current data resource in the predicted change data as the independent variable and the predicted metadata change amount as the dependent variable, an estimated resource change sequence diagram is established; Extracting statistical analysis indicators from the estimated resource change sequence diagram, performing change trend analysis to obtain an estimated change trend coefficient of the current data resource; Based on the set business evaluation indicators, the business needs of the current data resources are evaluated to obtain the demand evaluation indicator value; The acquired demand assessment index value is combined with the estimated change trend coefficient to calculate the resource update demand coefficient; The calculation formula of resource update demand coefficient is as follows: ; In the formula, It is expressed as the resource update requirement coefficient of the current data resource; It is expressed as the metadata change of the current data resource at the current moment; It is represented as the predicted metadata change amount at the jth predicted change time point of the current data resource, where j=1, 2, 3, , n; It is expressed as the time interval between the current moment and the jth predicted change time point; It is expressed as the estimated change trend coefficient of the current data resources; It is expressed as the weight of the impact of the predicted change trend of metadata on the update cycle of the changed data resources; It is expressed as the weight of the impact of business requirements on the update cycle of changing data resources; It is represented as the i-th set business evaluation index, where i=1,2; It is represented as the contribution weight of the i-th set business evaluation indicator to the business demand for evaluating the current data resources; e is represented as a constant with a value of 2.7; When the resource update demand coefficient falls within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource does not need to be updated; When the resource update demand coefficient does not fall within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource needs to be updated; By analyzing the predicted change data of the current data resources and the historical dynamic update cycle, the real-time dynamic cycle is obtained; The real-time dynamic cycle is used to replace the current preset dynamic update cycle to achieve updating.

[0037] In this embodiment, the data resource change record library refers to a database used to store data resource change records, and the change records include information such as the time, type, and content of the change; the preset time period is predetermined, such as 6 months; the historical change data includes the historical change time points, historical change frequencies, and historical change amounts of the data resources, etc.; the change prediction model is a prediction model obtained by training a neural network based on training data based on historical data resource change records and the characteristics of the data resources, and is used to predict changes in data resources in the future.

[0038] In this embodiment, the predicted change data includes the predicted change time point, the predicted change frequency, the predicted change amount, etc.; the estimated resource change sequence diagram is established by taking the predicted change time point of the data resource as the independent variable and the predicted metadata change amount as the dependent variable; the statistical analysis index refers to a quantitative index used to measure and analyze the change of data resources, such as the mean value and standard deviation of the change amount; the estimated change trend coefficient is used to characterize the predicted resource change degree of the current data resource, and is obtained by weighted averaging the index values ​​of the extracted statistical analysis indexes, wherein the weight assigned to the index value is obtained by solving the matrix constructed after pairwise comparison and relative importance scoring using the hierarchical analysis method.

[0039] In this embodiment, the set business evaluation index refers to the timeliness of data resource update and the stability of data resources; the demand evaluation index value refers to the index value obtained after evaluating the business demand of the current data resource based on the set business evaluation index; the resource update demand coefficient is used to characterize the urgency of adjusting the preset dynamic update cycle of the current data resource; the set demand threshold range is predetermined, consisting of the set demand threshold upper limit and the set threshold demand lower limit, generally ; The historical dynamic update cycle refers to the preset dynamic update cycle adopted by the data resource in the past preset time period; the real-time dynamic cycle is obtained by analyzing the predicted change data of the current data resource and the historical dynamic update cycle, and is used to replace the current preset dynamic update cycle (when it is determined that the preset dynamic update cycle of the current data resource needs to be updated).

[0040] The beneficial effect of the above technical solution is: by analyzing the update and change trends of data resources and business needs, the preset dynamic update cycle is updated, which can effectively avoid data lags caused by too long an update cycle, or system burden and resource waste caused by too short an update cycle, thereby improving resource utilization efficiency and reducing operating costs.

[0041] The embodiment of the present invention provides a data resource directory management method based on metadata drive, which obtains a real-time dynamic cycle by analyzing the predicted change data of the current data resource and the historical dynamic update cycle, including: Extracting the historical dynamic update cycle of the current data resource within a preset time period from the data resource change record library, and outputting it as the first analysis cycle; Outputting the historical generation time point of the first analysis period as the first update time; If the first analysis cycle does not exist, the resource update demand coefficient of the current data resource is combined with the predicted change data for analysis to calculate the first analysis dynamic cycle; The calculation formula for the first analysis dynamic period is as follows: ; In the formula, It is indicated as the first analysis dynamic cycle; it is indicated as the current preset dynamic update cycle; Expressed as resource renewal demand coefficient; It is expressed as setting an upper limit on the demand threshold; The time interval between the predicted time point corresponding to the minimum value of the metadata resource amount change in the predicted change data and the current time; It is represented as the time interval between the corresponding predicted time point of the maximum value of the metadata resource amount change in the predicted change data and the current time; e is represented as a constant with a value of 2.7; It is represented as the influence weight of the resource update demand degree of the current data resource on the calculation of the first analysis dynamic cycle; It is represented as the influence weight of the resource change prediction of the current data resource on the calculation of the first analysis dynamic cycle; It is expressed as the maximum value of the change in the amount of metadata resources in the predicted change data; Represents the minimum change in the metadata resource amount in the predicted change data outputting the first analysis dynamic cycle as a real-time dynamic cycle; If there is a single first analysis cycle, the second analysis dynamic cycle is calculated based on the historical time interval between the first update moment of the current first analysis cycle and the current moment and the historical resource change amount; The calculation formula for the second analysis dynamic period is as follows: ; In the formula, It is represented as the second analysis dynamic cycle; Represents the weight of the impact of resource demand and resource change forecast on the calculation of the second analysis dynamic cycle; It is represented as the influence weight of the historical update situation of the preset dynamic update period of the current data resource on the calculation of the second analysis dynamic period; It is represented as the historical time interval between the first update time of the current first analysis period and the current time; Expressed as the forecast time period; It is expressed as the absolute difference between the metadata resource amount at the last moment of prediction and the metadata resource amount at the current moment; It is represented as the absolute difference between the amount of data resources at the first update time and the amount of data resources at the current time; outputting the second analysis dynamic period as a real-time dynamic period; If there are multiple first analysis periods, a third analysis dynamic period is calculated based on the historical update intervals between the first update moments corresponding to all first analysis periods and the historical resource changes; Among them, the calculation formula of the third analysis dynamic period is as follows: ; In the formula, It is represented as the third analysis dynamic cycle; Represented as the cth first analysis period, c=1,2, , n; n represents the total number of the first analysis cycle; It is represented as the absolute difference between the metadata resource amount at the first update time corresponding to the cth first analysis period and the metadata resource amount at the current time; The third analysis dynamic cycle is output as a real-time dynamic cycle.

[0042] In this embodiment, the first analysis cycle refers to the historical dynamic update cycle of the current data resource within a preset time period extracted from the data resource change record library; the historical generation time point refers to the starting time point when the first analysis cycle (i.e., the historical dynamic update cycle) is applied; the first update moment refers to the historical generation time point of the first analysis cycle; the first analysis dynamic cycle is the update cycle of the data resource calculated by combining the resource update demand coefficient of the current data resource with the predicted change data for analysis (when the first analysis cycle does not exist); the second analysis dynamic cycle is the update cycle of the data resource calculated based on the analysis of the historical time interval between the first update moment of the current first analysis cycle and the current moment and the historical resource change amount (when there is a single first analysis cycle); the third analysis dynamic cycle is the update cycle of the data resource calculated based on the analysis of the historical update intervals between the first update moments corresponding to all first analysis cycles and the historical resource change amount (when there are multiple first analysis cycles).

[0043] The beneficial effect of the above technical solution is: by analyzing the predicted change data of the current data resources and the historical dynamic update cycle, the real-time dynamic cycle is obtained to replace the current preset dynamic update cycle, so as to realize the update of the preset dynamic update cycle, thereby effectively avoiding data lag caused by too long update cycle, or system burden and resource waste caused by too short update cycle, so as to improve resource utilization efficiency.

[0044] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A metadata-driven data resource directory management method, characterized in that: include: Step 1: Use the target data warehouse to regularly scan the data source to obtain metadata resources and manage the metadata resources; Step 2: Based on the set classification rules, classify the metadata resources and label them with corresponding functional tags; Step 3: Bind permissions or roles to metadata resources to form a target data map; Step 4: Update the data resource directory regularly and provide data service interface and search function for each data resource.

2. According to the metadata-driven data resource directory management method of claim 1, it is characterized in that: Based on the set classification rules, metadata resources are classified and labeled with corresponding functional labels, including: Based on the set classification rules, metadata resources are classified into three levels according to business functions to obtain business-classification data; Creating corresponding business function labels for the business-classification data, and adding corresponding business function labels in the metadata management system; Perform secondary classification of business-classification data according to data type to obtain number-classification data; Creating corresponding data type labels for the number-classification data, and adding corresponding number-classification function labels in the metadata management system; Classify the classified data into primary categories according to the data security level to obtain the secure classified data; A corresponding data security level label is created for the security-classified data, and a level function label is added accordingly in the metadata management system.

3. The metadata-driven data resource directory management method according to claim 1, characterized in that: Bind permissions or roles to metadata resources to form a target data map, including: Using metadata resources and corresponding level function labels as matching conditions, extracting a first role and a first permission group from a set role-permission database; In the metadata management system, binding the first role and the first permission group to the current metadata resource; According to the preset map layout requirements, use the set drawing tool to draw the initial data map; Inputting the metadata resource and the corresponding function label, the first role and the first permission group into a setting drawing tool; The setting drawing tool establishes the relationship between the metadata resource, the function label, the first role and the first permission group, adjusts the layout of the initial data map, and forms a target data map.

4. The metadata-driven data resource directory management method according to claim 1, characterized in that: Regularly update the data resource catalog and provide data service interfaces and search functions for each data resource, including: Use data resources and corresponding metadata resources to establish a target data resource catalog system; Based on the monitoring characteristics of data resources, different update mechanisms are used to regularly update the metadata resources in the target data resource directory system; Import the metadata resources in the target data resource directory into a set search engine, establish a metadata index, and provide multiple set search methods, and then integrate the set search engine into the target data resource directory system; Develop the data service interface after defining its functions and parameters based on user needs and the characteristics of data resources; Regularly maintain the data service interface and take appropriate repair measures to repair loopholes when they exist; Use pre-deployed monitoring tools to monitor the usage of data service interfaces in real time and obtain interface performance data; The interface performance data is input into a pre-established anomaly detection model, and after abnormality judgment and annotation of the interface performance data, the data is output as an anomaly detection result; If there is interface performance data marked as abnormal in the abnormality detection result, it is determined that there is an interface abnormality in the current data service interface, and the interface performance data marked as abnormal is used as a matching condition to match the corresponding abnormal maintenance strategy for maintenance.

5. The metadata-driven data resource directory management method according to claim 4 is characterized in that: The set search methods include four search methods: keyword search, category search, attribute search, and multiple search condition combination search.

6. The metadata-driven data resource directory management method according to claim 4 is characterized in that: Based on the monitoring characteristics of data resources, different update mechanisms are used to regularly update the metadata resources in the target data resource directory system, including: If the monitoring characteristic of the data resource is to support automatic monitoring, the current data resource is regarded as the first data resource; Using a set monitoring tool, periodically performing change detection on the first data resource in the target data resource directory system according to a preset dynamic update cycle to obtain a change detection result; According to the change detection result, if there is a change in the first data resource, automatically updating the corresponding metadata resource of the current first data resource in the target data resource directory system according to the change detection result; If the monitoring characteristic of the data resource is that it does not support automatic monitoring, the current data resource is regarded as the second data resource; For the second data resource in the target data resource directory system, a target update request is periodically submitted according to a preset dynamic update cycle by using a set manual update strategy; The target update request is audited, and when the audit is successful, the corresponding metadata resource of the current second data resource in the target data resource directory system is updated.

7. The metadata-driven data resource directory management method according to claim 6 is characterized in that: Also includes: Obtain historical change data of the current data resource within a preset time period from the data resource change record library, and input it into a pre-established change prediction model to obtain predicted change data of the current data resource; Based on the predicted change time point of the current data resource in the predicted change data as the independent variable and the predicted metadata change amount as the dependent variable, an estimated resource change sequence diagram is established; Extracting statistical analysis indicators from the estimated resource change sequence diagram, performing change trend analysis to obtain an estimated change trend coefficient of the current data resource; Based on the set business evaluation indicators, the business needs of the current data resources are evaluated to obtain the demand evaluation indicator value; The acquired demand assessment index value is combined with the estimated change trend coefficient to calculate the resource update demand coefficient; The calculation formula of resource update demand coefficient is as follows: ; In the formula, It is expressed as the resource update requirement coefficient of the current data resource; It is expressed as the metadata change of the current data resource at the current moment; It is represented as the predicted metadata change amount at the jth predicted change time point of the current data resource, where j=1, 2, 3, , n; It is expressed as the time interval between the current moment and the jth predicted change time point; It is expressed as the estimated change trend coefficient of the current data resources; It is expressed as the weight of the impact of the predicted change trend of metadata on the update cycle of the changed data resources; It is expressed as the weight of the impact of business requirements on the update cycle of changing data resources; It is represented as the i-th set business evaluation index, where i=1,2; It is represented as the contribution weight of the i-th set business evaluation indicator to the business demand for evaluating the current data resources; e is represented as a constant with a value of 2.7; When the resource update demand coefficient falls within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource does not need to be updated; When the resource update demand coefficient does not fall within the set demand threshold range, it is determined that the preset dynamic update cycle of the current data resource needs to be updated; By analyzing the predicted change data of the current data resources and the historical dynamic update cycle, the real-time dynamic cycle is obtained; The real-time dynamic cycle is used to replace the current preset dynamic update cycle to achieve updating.

8. The metadata-driven data resource directory management method according to claim 7, characterized in that: By analyzing the predicted change data of the current data resources and the historical dynamic update cycle, the real-time dynamic cycle is obtained, including: Extracting the historical dynamic update cycle of the current data resource within a preset time period from the data resource change record library, and outputting it as the first analysis cycle; Outputting the historical generation time point of the first analysis period as the first update time; If the first analysis cycle does not exist, the resource update demand coefficient of the current data resource is combined with the predicted change data for analysis to calculate the first analysis dynamic cycle; The calculation formula for the first analysis dynamic period is as follows: ; In the formula, It is indicated as the first analysis dynamic cycle; it is indicated as the current preset dynamic update cycle; Expressed as resource renewal demand coefficient; It is expressed as setting an upper limit on the demand threshold; The time interval between the corresponding predicted time point of the minimum value of the metadata resource amount change in the predicted change data and the current time; It is represented as the time interval between the corresponding predicted time point of the maximum value of the metadata resource amount change in the predicted change data and the current time; e is represented as a constant with a value of 2.7; It is represented as the influence weight of the resource update demand degree of the current data resource on the calculation of the first analysis dynamic cycle; It is represented as the influence weight of the resource change prediction of the current data resource on the calculation of the first analysis dynamic cycle; It is expressed as the maximum value of the change in the amount of metadata resources in the predicted change data; Represents the minimum change in the metadata resource amount in the predicted change data outputting the first analysis dynamic cycle as a real-time dynamic cycle; If there is a single first analysis cycle, the second analysis dynamic cycle is calculated based on the historical time interval between the first update moment of the current first analysis cycle and the current moment and the historical resource change amount; The calculation formula for the second analysis dynamic period is as follows: ; In the formula, It is represented as the second analysis dynamic cycle; Represents the weight of the impact of resource demand and resource change forecast on the calculation of the second analysis dynamic cycle; It is represented as the influence weight of the historical update situation of the preset dynamic update period of the current data resource on the calculation of the second analysis dynamic period; It is represented as the historical time interval between the first update time of the current first analysis period and the current time; Expressed as the forecast time period; It is expressed as the absolute difference between the metadata resource amount at the last moment of prediction and the metadata resource amount at the current moment; It is represented as the absolute difference between the amount of data resources at the first update time and the amount of data resources at the current time; outputting the second analysis dynamic period as a real-time dynamic period; If there are multiple first analysis periods, a third analysis dynamic period is calculated based on the historical update intervals between the first update moments corresponding to all first analysis periods and the historical resource changes; Among them, the calculation formula of the third analysis dynamic period is as follows: ; In the formula, It is represented as the third analysis dynamic cycle; Represented as the cth first analysis period, c=1,2, , n; n represents the total number of the first analysis cycle; It is represented as the absolute difference between the metadata resource amount at the first update time corresponding to the cth first analysis period and the metadata resource amount at the current time; The third analysis dynamic cycle is output as a real-time dynamic cycle.

Images